|
SCOPE is administered by the Follett Implementation |
Go to the SCOPE Annual Report Table of Contents
During the initial period of the project, evaluation included issuing a student questionnaire and various less formal information-gathering activities in the course of other project work. Over recent months and following the Tavistock Institute guidelines, we have attempted to identify key stakeholders and in consultation with them, to determine which issues are of concern and what type of data they require from us. This has informed a comprehensive evaluation plan.
Our priorities in evaluation are to monitor our success in:
Equally important is the evaluation of product design and production, for both printed course packs and on-line delivery.
To date activities include:
This last has been the most problematic to date, primarily for two reasons:
i) given the scale of SCOPE to date, the impact is bound to be small and difficult to measure (although a few useful indicators have been produced)
ii) it was not feasible in Year 1 to evaluate the pre-SCOPE situation to provide a basis of comparison.
A special librarian's evaluation working group has been set up to consider how best to proceed. A great deal of relevant general data have been identified (eg. User Satisfaction and Book Availability Surveys). Useful pictures have been built up of the various approaches to text provision in member institutions, although it has not been possible to identify clearly relevant costs, e.g. of running short loan collections. We hope that eLib will consider a bid for funds for an evaluator for the entire on-demand and electronic reserve programme areas, and are currently working on a case.
Evidence of mobilisation comes from our negotiations with over 60 publishers, with invitations to the Copyright Officer to give presentations for Book Industry Communications and the College and University Booksellers' group of the Booksellers' Association; and from inclusion of SCOPE data in two consultants' reports on on-demand publishing. On the academic side, although our promotion of SCOPE has necessarily been low-key to date, enquiries have been received from many higher education institutions, not only in Scotland: these have come from librarians and academics in subject areas including business, law, engineering, film and media, linguistics, history, psychology, biology and health studies. Comments on sustainability and future scenarios are given in section 4.
Details of evaluation studies arising from the first rounds of course pack production are appended. Although the final report from the second round is not yet complete, the most significant points have been included in this report. From the start of the project, SCOPE was made aware of doubts and criticisms regarding its aims and particular attention has been given to seeking evidence as to their validity. Specific issues are noted, with comments, below:
Course packs have been welcomed as a pragmatic solution for large classes where students almost invariably have difficulty in obtaining core recommended reading. In every case lecturers ensured that additional references were available in the libraries to widen the range available to students still further.
Course packs can be used imaginatively by academic staff and form an integral part of course design. Selection of texts from a wide variety of sources offers students a broad frame of reference and discourages shallow learning. Where students can select from a large number of readings, they bring differing perspectives to tutorials. Some lecturers argue that possession of a course pack encourages students to study in greater depth as they are not dependent on short loans of library texts.
On the other hand, one lecturer felt that the need to restrict the number of articles provided on grounds of cost, focussed the students' reading too narrowly, even although other material was made available in the library.
SCOPE's aim is to provide a value-added service to students, and there is evidence that this is particularly appreciated by evening degree students who may have little spare time to come to the library or who find borrowing conditions inconvenient. There is also a suggestion that full-time students may value time more highly now that more are obliged to fit their studies around jobs. SCOPE will not permit copies of packs to be added to library stock (since negotiations with publishers have only been concerned with sales to students), but recommends that at least one copy of each separate text in the pack should be available for borrowing. Experience to date shows that minimal provision is sufficient, but ensures that no student is disadvantaged through not being able to purchase a pack. The attitude of individual lecturers regarding student purchase of texts has an important bearing on SCOPE sales. It is surprising that apparently few universities in Scotland have institutional policies in this regard. Further investigation is required to try to determine how much students spend on photocopying as opposed to purchasing texts.
Data show that a majority of students read 50% or less of the articles in a pack. In spite of this, a large number of students wished additional articles to be included - possibly reflecting a desire to have all the material required for an essay together. There is some evidence that students who bought SCOPE packs were the same as those who bought textbooks and who read additional material in the libraries.
We have conclusive proof from round one that this was not the case. In round two, book sales at the same university bookshop dropped considerably. However, since the lecturer sent in his reading list late, textbooks relevant for pre- and early-semester reading were not available for students to purchase when required. Other factors may also have played a part - and we must admit the possibility that the packs did affect sales, although not greatly. In both rounds, the overall volume of sales of copyright material (packs and books) rose dramatically. A bookshop manager commented 'textbook sales are so pathetic in any case, that anything which increases sales of copyright material is welcome!' It was not possible to quantify total sales at the city university which used SCOPE packs in round two as students could have purchased from a wide variety of shops. However approximately 35% (40 students) claimed to have bought at least one new textbook.
Although £17.50 was considered too expensive by a majority of students offered packs in the first round, sales were very high. Following a rise in second round prices to £20.50, sales dropped drastically. These packs were subsidised, although the average rate of copyright fee per page was reduced in the second round. It is possible that many packs will not be viable unless academics are prepared to reduce the number of commercially published texts. A move to on-line delivery would permit students to browse, select and pay only for what they want: this is eagerly awaited by a majority of lecturers and students, although a significant minority had worries about access to computers or IT skills. Academics are prepared to substitute less costly or more easily available texts and regard this in the same light as selecting a textbook for a course where prime considerations are availability and price.