This section reports on the findings which have emerged from SCOPE evaluation activities and comments in particular on any general outcomes, effects and impacts. Difficulties or delays experienced in carrying out evaluation activities are noted in Appendix XI.

3.1. Technical

Hardware
Hardware specifications are likely to be more demanding than institutions may already have in place, which will mean some necessary upgrading before an on-demand, full-text service can be successfully implemented.
Difficulties:
• One CMI found it had only one (staff) computer capable of handling the service.
• One CMI had network printing problems connected with the print server's capabilities.

Software
• Purchase and installation of software (Adobe Acrobat Exchange V3 [c £100 per 10-user licence]; the SCOPE plug-in [free]) is a requirement before delivery can be implemented. Running the software over a network has the obvious advantage of savings in staff time on installation and maintenance. However this did not always prove trouble-free.
  Difficulties:
  • One CMI had problems with their software supplier, which caused considerable delay.
  • One CMI encountered problems in running software over a network, as opposed to installing it on individual computers, which is how it was handled by another CMI.

  Mobilisation

• Institutions have been made aware that electronic delivery of full-text over a network brings with it a general requirement for higher-specification equipment than has been necessary up to now.
• Institutional networks may require upgrading in order to cater for the developing demand for this type of service.
  See Appendix XI
• The SCOPE plug-in has expanded the possible uses of commercial software (Adobe Acrobat Exchange V3)
• The problems of delivering image-based files over a network have been communicated to rightsholders and representative bodies. It is hoped that this will have an effect on the negotiation of electronic rights licences.
• A commitment to maintaining service levels will be necessary to ensure sustainability
  Difficulties:
  One CMI with low-specification equipment was obliged to allocate one staff machine to the service this year, and also had a problem with password-protected access, thus leaving the sustainability of any continuing service in doubt. However, as the project neared its termination date, news was received that access and security in this institution had been tightened and computer hardware upgraded. As a result, it is hoped that service to this CMI can be sustained.

  Cultural change
  

• The delivery of full text through a combination of software programs was innovative and presented a new concept to institutions. (Filemaker database / Acrobat Exchange /Web browser)
• Electronic delivery increased the integration of technology and associated support staff into course preparation.

  Cost-effectiveness/value-added
  The initial investment required in upgrading equipment and networks will benefit other services in the long term.

  Contribution to overall project goals

• CMIs were able to assess the feasibility of delivering materials in this way, in a low-risk environment.
• SCOPE was successful in developing user authorisation mechanisms, as well as print logging and royalty calculation.

3.2 Organisational

• Implementation ideally requires high level managerial commitment and support to ensure:
  1. strategic planning, to ensure co-operation and communication between the different departments involved, i.e.: academic, library, computer support, registry
  2. adequate allocation of funds and resources
  3. effective communication channels, which are vital to successful implementation, with regard to the timely passing of relevant information to technical support services, budget managers, etc.
• Effective delivery requires on-site administrators with sufficient time to devote to the service.
  Difficulties:
  
  • Five CMIs did not participate fully in the project, as a result of time pressures.

  Mobilisation/cultural change
  Awareness of the above requirements was raised, especially at higher levels of management.

  3.2.2 SCOPE
  The SCOPE team experienced an increased workload and pace of activity at all levels, due to the larger number of courses and institutions being serviced. Greater effort in project co-ordination and communication were essential to maintain the higher level of activity in copyright clearance, production, liaison and evaluation.

3.3 Economic

Institutional pricing mechanisms

Online
Payment of the costs of copyright fees was handled in a number of ways:

• Two CMIs paid through the library by top-slicing book funds and monitoring usage.
• One CMI planned to pass the full cost on to students.
• One CMI paid through the department, which intended to recover costs by charging students for printing via EMOS cards sold through the department.
  In the event, various difficulties of implementation meant that no CMI actually passed any charges on to students, therefore libraries/departments bore the full costs.
  Course packs
  
• One CMI distributed packs through the department rather than via a commercial outlet. This meant increased costs in staff time, and would not be practicable in the long-term or on a large scale.
• One CMI saved on departmental staff time, as the service reduced the need for copying handouts for students.

Mobilisation
Awareness was raised of issues pertaining to the cost of providing on-demand services, especially in the innovative area of online delivery.

Cultural change
The ethos of free library services is being re-examined with regard to recovering some of the costs of on-demand services, and also within the context of pending legislation on student fees.

Cost-effectiveness/value-added


• One CMI carried out a survey which demonstrated that SCOPE proved generally cheaper than the CLA Rapid Clearance Service (CLARCS) for paper equivalent. A significant percentage of material could not have been cleared through CLARCS, even for paper-to-paper permissions.
• Libraries could face a possible loss of income from photocopying if electronic delivery continues to be free of charge to students. However, where students are charged for printing, it could provide an alternative income stream.

  3.3.2 STUDENTS

• Students benefit from internal distribution of course packs through the department, as the book shop mark-up is avoided. However, this is not viable on a long-term, large-scale basis.
• Electronic delivery proved in the event to be cheaper for students than photocopying the whole article. However, if planned methods of cost-recovery by CMIs had been implemented, this would not necessarily have been the case.
• Analysis continues to show that students who buy course packs are also likely to buy books. There appears to be an inherent willingness or reluctance to spend money on reading materials, in whatever format. It cannot therefore be said that SCOPE has significantly changed students' existing attitudes towards reading materials.
  See Appendix XII

  Cultural change
  There is a perceived increase in the service-customer ethos, especially in the light of existing financial pressures on students and pending legislation on student fees. Students are looking for value for money, and are more willing to complain about inadequate or inefficient services than was previously the case. Many resent the idea of paying for course material previously made available without charge. The cost to students of electronic delivery needs to be less than that of photocopying to be attractive to students.

3.3.3 SCOPE

SCOPE established the following setup:

Payment mechanism with rightsholders:

• SCOPE and rightsholders' initial preference was for pay-per-use
• Some rightsholders required a flat fee
• A few rightsholders made no charge
• Some material was in the public domain

  Invoicing and payments mechanisms

• Invoices were raised by the finance office of the University of Stirling, according to information derived from the SCOPE database
• Payments were made by the finance office of the University of Stirling, according to information derived from the SCOPE database

  Costs to SCOPE

• Administration, production and salary costs

  Mobilisation
  The pay-per-use mechanism was shown to be impractical for online delivery, despite initial enthusiasm from all parties, and all stakeholders have investigated other options. The JISC/PA working party agreed with these findings.

  Cost-effectiveness/value-added
  Payment mechanisms with rightsholders

  pay-per-use
  

• course packs - acceptable method as costs can be guaranteed in advance.
• online - cost-effective method in that payment is made only for what is used, but there is a risk that the administrative costs could be greater than the value of usage.

  flat fee

• course packs - again, an acceptable method as costs can be guaranteed in advance.
• online - the risk of failure to recover costs through usage makes this poor value for money, but administration is simpler and therefore cheaper.

• pay-per-use administration costs may outweigh the value of transactions to all parties, especially with regard to publishers' requirement to split copyright income with authors.
• the mechanism for extracting information from the SCOPE database requires further automation to make it cost-effective, as the present method demands much manual input.

• some course packs allowed a contribution to overheads to be recovered.
• online - for the purposes of the project in its first year of online delivery, no attempt was made to recover any overhead costs. Had any charge been made for the service, apart from cost recovery, it would have been necessary to add VAT, thus increasing costs to the user.

  Demand/usefulness/performance
  Implementation was successful in that the above mechanisms were carried out despite their inadequacies.

3.4 Scholarly/learning/teaching

Mobilisation
The project raised awareness amongst lecturers and students of the potential of electronic teaching materials as an alternative to traditional course packs, text books and library provision.

Cultural change


• SCOPE has led to exploitation of the possibilities of electronic teaching materials, thereby increasing demand for such services.
• SCOPE requires lecturers to produce reading lists much earlier than they are used to, in order that copyright clearance can be obtained in time for delivery to the course. This can be a significant barrier to participation.
• The concept of on-demand provision is perceived as a beneficial service by both lecturers and students.

  Cost-effective/value-added
  

• Departments can save money through the decreased use of handouts and departmental photocopying.
• Electronic printouts are far better quality than photocopies.
• Students only need to print out the texts they wish to use.

  Sustainability
  

• Costs need to remain competitive, compared with alternative services.
• Equipment needs to be in place and maintained.
• Reliable delivery is essential to retain lecturers' and students' enthusiasm.

  Demand/usefulness/performance
  

• The requirement for IT skills in students becomes the norm. Evaluation reveals that such skills are not always present and that lecturers see SCOPE as a means of filling the gap in a meaningful way. This has implications for introductory student training.
  See Appendix XII
• Access to materials is guaranteed for all students, with the removal of reliance on limited library materials.
  • 'we were able to focus on the specific materials we wanted, [because all students had access to the material] instead of having to compromise.' (lecturer)
  • 'opportunity to study the same data... as part of their preparation [for workshops].' (lecturer)
  • 'you know it will be there' (student)- convenience of having access to basic reading was a recurring theme in all courses, particularly those using course packs.
  • 'This could be a useful way to access materials, however I feel that there is no difference from printing this off and being given copies of them by the relevant departments' (student)
  • 'convenience of having printed copy at home was important for advance preparation' (student)
• Teaching can be more focused.
  • 'Focused students on the required reading' (lecturer)
• On-demand material can be used as introductory reading matter, and a jumping-off point for wider reading.
  • 'introducing students to basic conceptual and theoretical issues [to be discussed during workshops]' (lecturer)
  • '...you know you should read those [SCOPE materials] and then get additional reading.' (student)
• Material is available throughout teaching and revision periods
  • material was selected to be 'relevant to, but not sufficient for, examination preparation' (lecturer)
  • 'the SCOPE on-line system had been exceptionally useful for examination preparation' (student)
• Regarding uptake of resources, it appears that students' assessment of value is based largely on lecturer endorsement in their first year. Anecdotal evidence suggests that in later years, peer recommendation plays a role.
  • 'you've got the whole rumour thing going around. I didn't use it because my friend said that it's just a nightmare ...' (student)
  • 'I heard stories coming back about it so I never bothered going up.' (student)

  Future scenarios
  

• Demand is high for off-campus access to materials.
• There is a demand for servicing of distance learning courses.
  See Appendix XVIII
• There is demand for selective printing of materials, i.e. the option to print a page or range of pages from a text. The SCOPE system limited printing to the whole of any one text.
  See Appendix XVII

  Difficulties
  It was noted that in some cases there was a reluctance to use the online service:

  Students
  

  had worries about access to computers

  Solution: improved campus-wide computer provision.

  were hampered by a lack of IT skills

  Solution: introductory IT training as standard throughout institutions.

  preferred the tangibility and completeness of a course pack

  Solution: fast, cheap printing to allow students to compile their own 'pack' by printing out preferred materials in batches.

  resisted printing costs if charged

  Solution: requirement for cultural change unless costs can be absorbed by institutions.

  Lecturers
  

  had worries about student access to computers

  Solution: improved campus-wide computer provision.

  feared that students would attach more importance to electronic material simply because it was electronic, at risk of ignoring printed material of equal importance. (This was investigated and refuted during evaluation)

  Solution: ensure wide dissemination of evaluation results when promoting service.

  were concerned about the price and size of course packs, where inclusion of all the desired material could push the size/cost of the pack beyond the purchasing power of students.

  Solution: work towards increased online provision. Although this would spread the cost to students and circumvent the problem of course pack size, if copyright costs are not recovered from students the problem is simply shifted to the library or academic department.

3.4.2 SPOONFEEDING

This is one reason for lecturers being reluctant to use the service, either online or as course packs, although of course different lecturers have different ideas. One lecturer is happy to specify to students which materials to read, but considers including his own handouts in the packs to be spoonfeeding. Another is happy to provide basic handouts as packs.
Anecdotal evidence suggests that different subject areas may have different definitions of spoonfeeding. For example, students on factual courses such as engineering may receive more in the way of handouts than students on courses such as English literature, where the students' own assessment and evaluation is a priority.

'Unlike a core textbook, the packs do not encourage "spoonfeeding" because they encourage students to read a wider range of material by different authors. They can enable students to focus on different perspectives within an area, rather than overconcentrate on only one interpretation by a single author.' (lecturer)

SCOPE asks: 'Would you like to have included any extra material of your own in the pack?' (Lecture notes etc.)
Lecturer replies: 'No, I wouldn't, for I call that spoon-feeding. My experience is if one does that, the students read only that, instead of reading Plato, Bostock etc.'

Students, naturally enough, have a different perception and in questionnaire responses a number of them asked for lecture notes and handouts to be included in course packs.

The content of the resource bank as built up during the project's lifetime, by response to lecturers' requests, reflects lecturers' perceptions of how the material is to be used, and their attitudes to spoonfeeding. This in turn has a bearing on usage patterns. If the resource bank holds material which is essential to the course and students are directed to read it, then usage will be high. If material is equivalent to that in the library short loan collection - i.e. useful, but not all 'essential' - then usage will be lower. If material is general background reading then usage is likely to be very low as students are not familiar enough with the medium yet to look there for background reading, unless specifically told to do so.

Lecturers see the resource bank as a good way for students to gain easy access to out-of-print materials and obscure references which are otherwise difficult to locate. However, clearance of electronic rights on older material can be difficult, due to the non-existence of the concept at the time of publication.

Access to materials over and above those requested by lecturers, such as the MBA texts which were added to the resource bank and offered to all CMIs, has not been fully exploited during the short period SCOPE was able to offer it. However, such materials are felt to be a valid addition to the service in the long term, in that they add to critical mass whilst placing a lower production workload on the service.
See Appendix XV

See Appendix XII

3.5 Sustainability

The conclusions of the Impact Study1 have been reinforced during SCOPE's final year. Necessary conditions for OD/ER are:

Resource, system and support requirements

• Streamlined processes with sufficient capacity to make material available in time for academic deadlines.
• A system developed by a well-supported and multi-skilled team, including representatives of the people who will actually be using it.
• A resource of sufficient quality and size to warrant t
• The effort involved in using it and in learning to use it.
• A resource that contains materials which are in high demand.
• A resource that offers at least as much value as the hard-copy version in terms of usability.

  Circumstances

• A situation where demand for resources is not being satisfied by traditional methods of provision.
• A situation where OD/ER is seen as part of a general 'appropriate technology' approach to resource provision and is integrated into teaching and learning accordingly.
• A group of end-users who are adequately equipped to use IT and who are appreciative of the skills development aspect of using electronic information resources.
• An environment that provides access to sufficient numbers of computers (equipped with appropriate software) and printers and adequate IT support.
• Acceptance by academics, students and librarians within each HEI as to who pays for what.

  It has been particularly difficult given the scale of alternative provision within each institution to discern the effect on traditional library provision - especially since there has been an understandable reluctance to withdraw parallel print material in which libraries have already invested. It is also very difficult to cost IT requirements for large-scale provision: investment will however be required not just for OD/ER but for a wide range of university activities. The modelling exercise in the Impact Study1 gives cost comparisons which suggest that in the long term OD/ER could indeed be viable.

  Key considerations are:
  

• the cost of retrodigitisation (can it be reduced or shared?)
• the length of time before textbooks in original electronic form become widely available (there has been major progress in the last 2-3 years with electronic journals)
• pricing policies for electronic text, which are still being worked out by rightsholders
• clarification within HEIs over policy on payment for student texts

3.6 Future scenarios

The number of enquiries received over the last year from the UK, Scandinavia and Australia in particular, demonstrates widespread interest in OD/ER. While many university libraries wish to move to electronic reserve systems, there is also significant interest in course packs. For instance, in one older university, a survey to determine service priorities revealed a shift in attitude with interest in course packs from academics in every subject area.

There have been several significant developments within the last year which augur well for the future of OD/ER:

• the reports and recommendations of the JISC/PA working parties2,3,4 on fair dealing, clearance mechanisms and a model licence
• the announcement that the CLA is to start dealing in electronic permissions in the second half of 1998
• the award for the establishment of HERON, the national on-demand service

  All these should help to remove remaining blockages in copyright clearance, although there are still uncertainties as to how quickly rightsholders will opt in to new electronic licensing arrangements. Delays will destroy the momentum built up by all the OD/ER projects. In this final phase of the eLib programme, publishers and other rightsholders should see the JISC funding as a rare window of opportunity: it will provide experience of large scale services not hitherto feasible and pave the way for the online textbook.

  There are early hints (not yet confirmed) that:

• pdf normal (using Adobe Capture software) may become the preferred format both of rightsholders and HEIs. This would obviate proofreading and lead to significantly reduced costs.
• rightsholders may relax conditions for online delivery, providing that they are assured that HEIs will take seriously their responsibilities to foster a culture of compliance with copyright law and licence terms. This could make it very much easier for libraries to establish and maintain electronic reserve services.
• the textbook and library substitution models recommended in the JISC/PA charging mechanism report4 may be widely adopted. The critical factor here is where the line is drawn between the categories. A best-selling 'essential' textbook at one university will be 'optional' or 'for background reading' at another. An alternative approach which might more closely reflect real demand and which might merit further investigation is course pack and electronic reserve models.

  ---

  [Next page] [Contents]