Evaluation Documents SCOPE home page

SPACER SPACER SPACER

SCOPE Project Stage 6 User Satisfaction Survey Report July 1998

1. Introduction

This report summarises the evaluation results from students, lecturers and library representatives collected during Stage 6, Autumn 1997. Further details are available in the individual reports listed in the Annex.

Two lists were dropped by the lecturer after consultation with SCOPE, before clearance procedures started. The Theology pack was dropped because the lecturer felt that ‘I cannot in all honesty commend this to the students when there are also other things to buy, like an introductory book, and there will be other xeroxing they need to do of things not in the pack.’ The Administrative Studies pack was dropped because too much material would need to be excluded to make the pack affordable, so students would still have needed to buy a textbook: ‘However it looks as if we are going to lose too much [material] from our original idea so I have decided not to go ahead with our proposal. ...the students will now be asked to purchase a textbook.’

Of the courses for which material was cleared, four did not use the SCOPE resources provided. This was due to Institutions 1 and 6 failing to get the relevant hardware/ software running in time for the course (see 4.3 Delivery for details); The library at Institution 3 was undergoing building work which precluded any project involvement; and one lecturer at Institution 4 did not promote the SCOPE pack sufficiently strongly to Electrical Engineering students, so they did not buy it.

Institutions were required, as a condition of participation in the project, to evaluate the effect of SCOPE on students and staff. This was made clear in the Letter of Understanding which went to Chief Librarians, and in the individual Service Agreements which were signed by library representatives for each course.

Evaluation was done by a combination of questionnaires, focus groups, and interviews.

Two generic questionnaires were designed by SCOPE for evaluating packs and for online delivery. These were then modified by Library Representatives according to local needs. Most results were quantitative, with a few qualitative questions. The questionnaires were usually distributed by lecturers during classes and handed back at the end, except questionnaires for Institution 8, where Sociology(d) questionnaires were sent out via Royal mail with prepaid return envelopes, and Sociology(f) questionnaires were sent via Royal Mail with course evaluation. Evaluation was disappointing in the second semester due to either failure by library reps or by the lecturers to distribute questionnaires on time, or due to lack of usage. For example, students on Sociology(f/g) were sent questionnaires by Royal mail, which elicited a 4% return rate; Electrical Engineering students at were not given the questionnaire by the lecturer; and students at Institution 1 did not use the service. On the advice of Institution 8’s University representatives, Economics students were not asked to evaluate the pack, and the Sociology(b/c) packs were not evaluated in depth. In the case of Economics, the pack was well received the previous year and the only comments were requests for changes in layout which were carried out for this year. Sociology(b/c) packs were exactly the same as the previous year, therefore it was felt that no new data would be produced by evaluating the packs. All questionnaire analysis was done by SCOPE using PinPoint 3 for Windows (except for questionnaires from Institution 4, which were analysed by the institution using Excel), this allowed comparability of results.

In addition to the questionnaires, focus groups were also used at three institutions. These provided qualitative information on areas which were identified by the Library Representative. One institution used an external facilitator, whilst the others used lecturers from the course. Some groups were self selecting, consisting of students who turned up to an advertised group, and others were requisitioned by running focus groups during timetabled tutorials. Questionnaires were distributed to Information and Society students by the lecturer, but the lecturer was unwilling to use teaching time for focus groups, so none were held. Philosophy students did not have focus groups due to the library representative being off sick.

Library Representatives attended a focus group facilitated by 101 Dimensions Ltd. This aimed to get general information about how they felt the SCOPE project had worked, and also how it had affected their workload and the library as a whole.

Most academic staff wrote reports about their experience of using SCOPE. These reports were supplemented with telephone calls and emails to expand or clarify some points.

The methodology, where implemented, generally worked well, although some modifications to the questionnaires should have taken place. For example, the prices of the packs sometimes fell at the boundary of two categories when students were asked what price they would be willing to pay. In addition several questions which were inappropriate to Institution 5’s local situation were included on their questionnaire, leading to meaningless and confused results.

4.1 Preparation

Most lists were submitted with a good level of bibliographic detail. The most commonly lacking aspects were details of illustrations; page ranges of associated notes; the start and end points for extracts which broke mid-page and were not part of a clearly defined range; and requests for journal articles often lacked details of publishers and ISSNs. Problems also occurred when material was requested from a compilation, and the book was not checked for details of who owned the original rights: a series of extracts from two items for The Women’s Studies and English courses turned out to be licensed from their original sources, therefore details for these extracts needed to be re-submitted. A further issue which became apparent through online delivery was the length of extract. After printing trials, it was decided that items should not be longer than 30 pages. Therefore time was spent by the SCOPE team identifying (and clearing with publishers) relevant break points in the text.

Most lists underwent some revision before the final materials were produced, even if only the removal of items due to unco-operative publishers. Where no substitution took place, lecturers relied on traditional library services to provide the required material.

• The Philosophy list underwent extensive substitution to obtain clearances.
• The Engineering(a) list had an item refused by SCOPE because it comprised too large a percentage of the original book.
• The Sociology(a and d) lists also needed prioritising to cut down the amount of material and to ensure that the most useful material was provided.

(See 4.2 Composition for more detail.)

All lecturers questioned felt that the time spent preparing SCOPE materials was worthwhile. Library representatives also viewed time spent on the project as an investment.

There have been some problems in getting hard copy of materials to the technical unit for scanning. This arises during progressive mounting of online materials, because library representatives are understandably unwilling to remove recommended texts from the library once teaching has started. In addition, at Institution 5 the lecturer was heavily involved and provided books from other university libraries, which he visited for this purpose rather than requesting the Library Representative obtain them on inter-library loan. This caused him considerable inconvenience and illustrates the problems which unclear lines of communication between lecturers, library representatives, and SCOPE staff can cause.

 

4.2 Composition

The packs produced ranged from entirely institutional/author copyright through to entirely commercial copyright, with one pack containing a significant amount of public domain material.

Philosophy: 10 items: 8 book extracts, 2 journal articles; rights held by three publishers, seven public domain items. Initial list included details of alternative sources for extracts: the final choice was determined by the speed at which publishers responded and their price. The public domain material was substituted for material by a publisher who would not give permission.

Sociology(a): 9 items: all book extracts; rights held by: one author, one editor, one agent, four publishers. Initial list of twelve items requested, six items dropped due to four unco-operative publishers. Additional six extracts supplied; list prioritised to eleven extracts, one dropped due to high pictorial content, one dropped to keep price to what the lecturer felt was an acceptable level.

Business Applications Software: material comprised of lecturer-written materials; rights held by authors.

Engineering(pack): material consisted of lecturer-written material and extracts from one out of print book; rights held by University and publisher.

Economics: material consisted of lecturer-written material; rights held by author.

Sociology(b/c) pack 1: 11 items: 8 book chapters, 3 journal articles; rights held by eleven publishers. Remaindered packs from previous years.

Sociology(b/c) pack 2: 13 items: 10 book chapters, 3 journal articles; rights held by nine publishers. Remaindered packs from previous years.

Sociology(b/c) pack 3: 14 items: 12 book chapters, 2 journal articles; rights held by nine publishers and three authors. Remaindered packs from previous years.

Sociology(b/c) online: 11 items: 8 book chapters, 3 journal articles; rights held by eight publishers. Initial list of thirteen extracts requested, two dropped due to two unco-operative publishers, no substitutions made.

Sociology(d): 24 items: 18 book extracts, 3 journal articles, 3 official publication extracts. Rights held by 14 publishers, one author, one executor, three public domain items. Initial list of 12 items requested, 18 extra requested as additional reading, six items dropped due to three unco-operative publishers, no substitution made.

Engineering(a): an entire out of print volume. Rights held by publisher. Initial request for extracts from two books, one dropped due to unco-operative publisher.

Biology: 8 items: 1 journal extract, 7 newspaper extracts. Rights held by four publishers. Initial list of 21 items requested, thirteen items dropped due to three unco-operative publishers, no substitution made.

Communication and Information Science: 7 items: six book extracts, one journal article. Rights held by 6 publishers. Initial list of ten items requested, three items dropped due to two unco-operative publishers, no substitution made.

English: 8 items: 4 book extracts, 4 journal articles. Rights held by 4 publishers. Initial list of 24 items requested, thirteen dropped due to 2 unco-operative publishers, 1 publisher having copyright fees which were too expensive, 5 untraceable rightsholders, no substitutions made.

Women’s Studies: 16 items: all book extracts. Rights held by 2 publishers, 6 authors (6 negotiated through literary agents) two public domain extracts. Initial list of 32 book extracts, 1 dropped due to 1 unco-operative publisher, 1 dropped due to high cost of permission, 1 dropped due to untraceable rightsholders, 13 unresolved due to 6 non responsive publishers and 2 agents, no substitutions made.

Electronic Information and Society: 16 items: 5 book extracts, 11 journal articles. Rights held by 8 publishers and 2 authors. Initial list of 20 items, 4 dropped due to 3 unco-operative publishers, no substitutions made.

Sociology(f/g): 45 items: 42 book extracts, 3 journal articles. Rights held by 19 publishers and 3 authors. Initial list of 67 items, 20 dropped due to 15 unco-operative publishers, 2 dropped due to untraceable rightsholders, no substitutions made.

Sociology (e): 10 items from Sociology(d), 6 book extracts, 3 journal articles, 1 official publication. Rights held by 7 publishers, one public domain.

The format of the materials (printed pack or electronic) was usually dictated by the availability of computing equipment, or the lecturer’s preference. However, some first year students indicated that they found that being expected to use the Resource Bank during their first semester was overwhelming (in terms of the amount of new skills they were already learning) and therefore expressed a preference for packs. Thus the expected profile of students is also a factor for consideration when deciding on the format of materials. (This will also affect the necessary training for students.) Many students and at least one lecturer expressed preference for packs because of their tangibility: ‘printed course packs are much more attractive and tangible.’ and also the convenience of having all material bound together in one place.

Substitution of materials occurred on the packs whose content was essential reading for all tutorials during the course. If the first choice material was not available, then the lecturers selected an alternative. On other courses, where material was dropped from the list, lecturers simply relied on traditional library provision. The most common reason for dropping items from reading lists was publishers refusing to participate in the project. Cost was also a factor: where packs were produced, the final decision on inclusion was made by the lectruer. Where material was provided online, the decision was left, as much as possible, to the party expected to bear the cost of the copyright fee - either the library, the department, or the student. Where it was borne by the library or the department, expensive items were dropped. Where the cost was borne by students, material was included to give them the final choice (unless an up-front payment was requested by the rightsholder). The final factor in dropping items from reading lists was the timescales involved in clearance: if a rights-holder was taking to long to grant permission, alternative materials were used. This factor was mainly evident in course pack selection.

 

4.3 Delivery

Delivery of the packs was relatively painless, but to a very tight schedule which did cause some concern as to whether packs would be available at the start of the courses, particularly at Institution 7 where a James Thin bookshop was involved for the first time. The Sociology Department at Institution 2 did not find selling the packs to be particularly disruptive, but would not be able to do this on a large scale (i.e. for other modules).

Online delivery in the first semester threw up more problems than expected. At Institution 8, the resource bank was available in an un-staffed computer laboratory (also used for teaching), and later in the semester through a library computer. The machine in the library was only available during the last two weeks of teaching, but was heavily used when it became available. Sociology(b/c and e) material was available for the start of the semester; Sociology(d) material was mounted on the server throughout the duration of the course, prior to it being required. There were complaints from the students that material for Sociology(d) was not available in time to be useful. This contrasted with the lecturer’s view that material had been made available in a timely manner. Students also complained about the limited number of terminals from which SCOPE could be accessed, and the fact that they could not access SCOPE from terminals outside the university (i.e. from home and work). The main source of complaints was printing problems, such as failure to print the whole document and the fact that they could not attempt printing again without getting staff to reset the database. In addition, there were instances of students’ paper being used by other documents queued on the networked printer.

In the second semester, printing at Institution 8 progressed relatively smoothly, with the main delays being caused by mounting student names.

Students at Institution 5 also had access to SCOPE from one PC lab with a networked printer for those PCs, and could view documents from the library. The lecturer was unaware of the facilities in the library, so did not promote it to the students. Printing in the lab was initially subject to delays caused by a faulty printer. This was replaced, but the print server was very slow: ‘delays of five minutes for the first page then a delay of three minutes after every third page, so a twenty page article takes nearly half an hour to print out - in a busy general- purpose lab.’ This situation caused immense frustration amongst the students, and was very detrimental to uptake of the service. The department had had two solutions suggested by the SCOPE Technical Unit, but chose not to implement them due to time and financial constraints.

The only courses to have extensive demonstration and timetabled support were Sociology(d) and Electronic Information and Society, where use of SCOPE was covered in practical tutorials. There was no library induction on the subject, or demonstration to Sociology(b/c), leading to students requesting more support/demonstration of the system from staff, particularly at the start of the semester. Sociology (e/f) students were given a powerpoint demonstration of the SCOPE service at the start of their course.

Institution 1 and Institution 6 both failed to make use of the resources provided. At Institution 1, installing the software and use of image files for the engineering course caused local problems which were not solved before the course for which material was provided had finished. The material for the English course was considered peripheral and does not appear to have been strongly promoted by the lecturer. The Women’s Studies material would have been used, had more been available at the start of the semester: the lecturer did not feel that the amount of material available at the start of the year justified the time it would take to teach students to use the system, therefore although they were told about SCOPE, students received no instruction until the Liaison Officer visited the University in May. Both the English and Women’s Studies courses were requested from SCOPE at very short notice and the library representative was warned that it was unlikely that much of the material would be available in time for the start of the courses.

Institution 6 had severe technical problems caused by lack of on-site support, late delivery of software and general installation. By the time the equipment was up and running, students had already started their assessment for the course and did not make use of the SCOPE material. Students were made aware of the service by the lecturer, but no demonstrations took place.

 

4.4 Sales/Usage

4.4.1. Amount of usage

The size of the print run was decided in conjunction with the lecturer. All packs were sold through John Smith and Son bookshops, unless otherwise stated.

Philosophy: 40 packs, sold out, reprinted 6, sold out (via James Thin).

Sociology(a): 50 printed, sold out, 48 copies reprinted, 2 remaindered (via the Department).

Business Applications Software: Initial 420 copies sold out, 67 reprinted, 14 remaining.

Engineering(pack): Initial 20 copies printed, 60 reprinted, 4 remaining.

Economics: 40 packs supplied, 15 remaining.

Sociology(b): 165 copies supplied, 36 remaining.; 4 printouts by 4 students - 2% of those registered on the database. Some sales of second-hand packs were also apparent.

Sociology(c): 3 printouts were registered by 3 students - 9% of those registered on the database.

Sociology(d): 166 printouts were registered during the first semester, 10 during the second, by 34 students - 68% of those registered on the database.

Electronic Information and Society: 399 printouts were registered from 114 students - 66% of those registered on the database.

Sociology(f): 48 printouts were registered by 12 students - 18% of those registered on the database.

Sociology(g): 2 printouts were registered by 1 student - 4% of those registered on the database.

It should be noted that evaluation results indicate that students are not always asking for the database to be reset when printouts fail, so the number of printouts registered is higher than the number of items received by students. However, the figures do give an indication of demand.

Sociology(b/c) packs again failed to repeat the high level of sales which occurred in the first year. It was suggested by the manager at James Thin that students would be less likely to buy books if the grapevine indicated that previous users had not been convinced of their value. This could be a partial explanation for the decreasing sales of the Sociology(b/c) packs. However, it is uncertain how effective this method of communication would be for first year students during their first semester. The remaindered packs from Economics and Engineering will be saleable to future courses. Sociology(b/c) will have a different organiser during 1998/99, but the material in the packs is still be considered relevant.

The ethos behind the provision of SCOPE materials to Electronic Information and Society, and Sociology(d) was very similar. Both lists contained reading essential to the course, which was not held by the library, and students were expected to use the resource bank as part of their course. The materials for both courses had a similar uptake, although Electronic Information and Society students printed proportionally more copies.

Material held for Sociology(f/g) was used more than Sociology(b/c) had been; this can be attributed to several factors - there was far more material available online for Sociology(f/g) (one complaint from Sociology(b) students was that there was not enough material available online to make it worth using), and also it was the second semester of delivery so the worst of the delivery problems had been sorted out, and students had had more time to develop their IT skills. Usage was suppressed by the following factors: the material was also readily available in the library; students were not told that using the resource bank was a vital part of the course; and some students who had previously been on Sociology(b) may well have been reluctant to use the resource bank after the previous semester’s problems.

4.4.2. Usage patterns

Most sales of course packs occurred during the first two months of the semester (i.e. September and October), with a small amount of sales later on in the year.

Printouts for Sociology(d) were made throughout the course as material became available, with an increase towards the end of semester when exams were pending and the library printer came online. However, it is unclear whether this trend marks an actual increase in demand, or that the library computer and printer allowed existing demand to be fulfilled, or that there was more material available (or a combination of these factors). Printouts for Electronic Information and Society were steady throughout the semester, showing a slight rise before exams. This steadier rate may be an indication of the fact that materials were available from the start of the course, and were not being continuously mounted.

Anecdotal evidence suggests that some students, having located the information they required using the resource bank, photocopied the pages they needed rather than printing off the whole article. Some students used the facility to search the text on screen, and some took notes from the screen, especially when printing problems occurred.

Previous experience of using the internet does not seem to have affected students’ use of the SCOPE resource bank.

Of those students questioned, 18-27% had shared their packs. Very few of these stated that they had not purchased a SCOPE pack, so it can be assumed that most students are splitting the cost of the pack as well as the use. Anecdotal evidence suggests that sharing of packs is more likely where students have shared accommodation, and where the material is background reading. However, statistical evidence is inconclusive on both these hypotheses.

4.5 Cost

The cost of the packs ranged from £3.50 to £15, depending on size and copyright content. Satisfaction with value for money appears to be directly linked with the integration of the materials in the pack with the course. Philosophy rated highest with 81% of students satisfied with value for money, and all items being required reading for tutorials; Sociology(a) was next with a 66% satisfaction and 60% of the material being essential reading for tutorials. Students at Institution 3 had a lower satisfaction rating at 53% - probably explained by the lack of direct reference to the Medical Engineering course in the packs (see 4.7 Relevance), and a perception that lecturer-written materials in the Business Applications Software pack were overpriced: ‘..fair enough making a profit but [it] does not have to be so vast.’ Only 26% of students indicated satisfaction with value for money of Sociology(b/c) packs; this is probably due to the fact that none of the material contained was essential to the course, and it was not directly integrated into teaching.

Format also affected perceptions of value - the Philosophy pack included PDF image files which looked like photocopies, so students did not attach very high value to them. A Sociology(a) student pointed out that ‘[Packs] can never be cheap enough’. This reflects lecturers’ views that students will always complain about the price, no matter how cheap it is. This is a point of conflict with the rightsholders’ perspective that (issues of competitive pricing aside) there is no reason why packs of customised material should be cheaper than an equivalent text book. It can be argued that the saving to users comes from not having to buy the original sources of extracts, not from comparing the cost to equivalent photocopy charges.

All lecturers approved the prices of the packs. Restrictions included keeping the cost to less than a photocopied equivalent at Institution 7, and that packs should not cost more than £15 at Institution 2. These objectives were fulfilled by using public materials, and by the lecturer prioritising the reading list and eventually dropping items.

Online printing at Institution 8 did not incur any costs to the students, although they had to provide their own paper. This is an institution-wide policy. Some anecdotal evidence suggests that students would find it more convenient to pay for use rather than supply paper. One also stated that even the cost of buying paper was too much: "I just didn't want to pay out for anything, not even paper. I was just totally skint. There were other courses which I had to buy books for." This is a slightly surprising statement since the cost of paper is less than photocopying. Institution 6 planned to pass all costs on to students, who would pay for the printouts when they collected them from the library desk. Institution 1’s library was going to cover the cost of copyright. Electronic Information and Society students were to have been charged for printing using an emos card attached to the printer (cards to be bought at the department to offset copyright fees). However, this printer did not work, and the emos card reader was not attached to the replacement printer, so printing was free throughout the course.

The financial cost to libraries who requested, but did not use the service was minimal, except where a flat fee was charged in advance by the rights holder. Costs will have been increased by students failing to get the database reset after unsuccessful printouts.

In general, all costs directly incurred in the production of course materials were recovered by SCOPE. However, overheads such as salaries and equipment were not.

4.6 Binding/Covers

Generally speaking, users were not very satisfied with the covers of the packs. They were felt to be too flimsy and did not stand up to the amount of use that the packs got. It may be that the acetate covers are more of a hindrance than a help, since once they are bent, they do not fold back as easily as the card does. However, there is no feedback to either support or refute this. There were also complaints from students about the sturdiness of the binding.

4.7 Relevance

As with the cost, perceptions of the relevance of the pack are directly linked to the integration of the materials and the course. The majority of students on all courses were satisfied with the usefulness of the pack. However, the fact that students consider the material to be useful is no guarantee that they will actually use it.

• Philosophy: over 90% students were happy with usefulness. 69% of students used more than 75% of the material in the pack. The main complaints were about some incorrect bibliographic references (which were corrected for the reprint), and also that the public domain translation of Plato was out-dated. (This translation was approved by the lecturer in order to keep the price of the pack below equivalent photocopying costs.)
• Sociology(a): No students were dissatisfied with usefulness, and only one was uncertain. 63% of students used more than 75% of the pack. The main complaints about content were that they were not in the order in which they were taught, and also that some of the extracts were very long. Students did not perceive the pack as being of use for future study, although lecturers thought that it would be.
• Institution 4 (mainly Business Applications Software students): 85% of students were satisfied with usefulness, although only 27% used more than 75% of the pack. This lack of usage is probably due to the fact that evaluation took place whilst the course was being taught. Relevance was a major complaint by the Medical Engineering students, who thought that the pack should be more directed towards their course, using medical terms and examples, rather than general engineering theory: ‘Surely we are not just gears and crankshafts?’ The lecturer’s opinion was that engineering principles are the same, irrespective of their application, and did not see a need to tailor the contents further than had already been done.
• Sociology(a): 64% of students were satisfied with usefulness of pack although only 5% used more than 75% of the materials. Although this is low, overall usage is up on last year’s results (1996 - 18% used more than 50% of the pack; 1997 - 34% used more than 50% of the pack). As with last year there was call for material to be collated by subject, rather than to give overall perspectives.
• Sociology(d): 73% of students were satisfied with the usefulness of the material, although none used more than 75% of the material. This is probably due to the time it took to get the material mounted and to solve printing problems.
• Electronic Information and Society: 85% of students were satisfied with the usefulness of the material, although only 5% used more than 75% of the materials. This is undoubtedly affected by the printing problems which the course experienced.

4.8 Use of material

Most of the material which SCOPE produced for this semester was essential course reading. Philosophy, Sociology(a), Sociology(d) and Electronic Information and Society all required students to read specific extracts in advance for discussion in tutorials, as well as for lectures and assessment. Courses at Institution 4 and the Economics course used the material to support lectures and assessment. Sociology(b/c and f/g) were the only courses for which use of SCOPE materials was not considered by the lecturer to be essential for the course.

All material was designed to be of use for preparing for assessments, be it continuous assessment (e.g. essays and tutorials) or revision for end of semester exams.

4.9 Effect on alternative provision

4.9.1 Library usage:

Most courses for which SCOPE had to clear commercial copyright material had been run in previous years, and were supported by established collections within the library. The only exceptions were Sociology(d) and Electronic Information and Society. Both these courses were new, and did not have full library provision. Not all material for Sociology(d) was available in the library; The lecturer for Electronic Information and Society provided a back-up short loan collection containing alternative material to that provided by SCOPE.

The only pack which had any significant effect on library usage was Philosophy. Comparison done with the main library collection showed a decrease in loans of the journal articles included in the pack. However, there was an increase in loans of Plato dialogues, which may reflect the dissatisfaction with the translation used in the pack, and also the decrease in sales of the recommended book (see 4.9.2 below). Other library provision for Philosophy included a short loan collection and a small departmental library, but the effect on these collections was not assessed.

Electronic Information and Society was a new module, therefore there were no previous usage figures for direct comparison. However, the back-up short loan collection was not used as heavily as the lecturer would have expected, which can probably be attributed to the presence of SCOPE materials.

Institution 8 did not provide any library usage data for Sociology(b/c). Sociology(d) was a new course so there are no previous figures to compare issues against. There is some evidence that issues for Sociology(f/g) were affected by SCOPE usage: the most heavily used extracts showed a decrease in loans, whilst extracts which were not used showed maintained, or higher loans. However, this is not definite proof since the sample of books examined was too small to be conclusive, and it is not known if the books were used by students on other courses.

Sociology(a) was a new course, so no previous figures were available. The lecturer stated that library provision was an important complement to the pack and provided additional reading.

The situation remained static at Institution 4: too few books were issued to allow comparison for the engineering courses; and there was no equivalent book for the Business Applications Software course.

4.9.2 Book Buying Habits:

Philosophy: sales of the recommended Plato textbook halved, despite an increase in students numbers. It was expected by the bookshop that 50% of students would buy books, but in fact only 15% made purchases. However, this was not reflected in students’ responses to their questionnaires: 45% indicated that they had bought new books. It is likely that the SCOPE played some part in the reduction of sales, and was exacerbated by students making more use of the library (see 4.9.1 above), and buying books from sources other than James Thin.

Sociology(a): 25% of students indicated that they had bought new books. The course was new and there is no campus bookshop at Institution 2, so comparative figures are not available.

Sociology(b/c) students were given suggestions of books they could buy if they wanted, but were not told that they had to buy any. The students’ questionnaires indicate that those who bought SCOPE readers were statistically more likely to buy other books as well. 30% of students had bought new books.

4.9.3 Departmental Spending:

The Sociology(a) pack was sold through the department office; although this did not cause the staff any problems, it would have cost the department in terms of administrative staff time.

Institution 5 incurred increased departmental spending since they covered the copyright fees.

Institution 4 was the only institution for which SCOPE materials decreased department spending. This was in terms of less staff time and departmental resources spent on photocopying.

4.9.4 Teaching practice:

Responses from staff have been generally positive. SCOPE materials give students guaranteed access to key reading materials, which is important in allowing staff to plan their teaching sessions, and students to prepare for them. Staff can focus teaching and discussions more precisely, and teaching can progress faster because less time is required to introduce new subjects. It also gives students access to a wide range of ‘hard to get hold of’ material.

On the downside, if material has not been mounted in time, teaching has to be changed to accommodate this. It would also be useful to lecturers and tutors if they could access the material in the same format as the students (i.e. if they could have access to the resource bank), so everyone could work from the same source. One student complaint from Institution 4 is that the presence of the engineering pack ‘Makes lectures less interesting because they talk to the pack.’

4.9.5 Student reading

Results are split between those who have used the packs as a starting point for studying, and those who have tried to use SCOPE materials as their sole reading. All packs (except for Business Applications Software and Economics) were designed specifically to provide reading which would be relevant, but insufficient as the sole source of information for assessments. The packs also avoided making the student dependent on the views of one particular text or author perspective. Students were informed of this by the lecturers, and most appear to have used the packs as they were intended. ‘It’s not saying that you should only read this, it is just the pieces that are the most essential. So you know you read those and then get additional reading.’

4.10 Electronic delivery

Perceptions by non-users

Most students welcomed the idea of providing materials online - they felt it would be cheaper and would save paper, but were put off by the problems of accessing machines. There was also evidence of computer literacy problems and technophobia, but access to machines was the main concern. Those who used packs liked them for their tangibility and convenience: once purchased, all the key reading was available.

Perceptions of users:

Many of those who used the online resources for Sociology(d) and Electronic Information and Society stated that they would have preferred a pack. This may be partially caused by the lack of alternative methods of access, such as traditional library provision; unavailability of material at the start of the course; and printing problems. This is illustrated by comments about the possibilities of the system, rather than the reality of it.

‘In theory it should have been an invaluable asset but in practice was nothing more than an aggravation - I would rather have had a book’
‘I think that it is a great idea, but you should be able to get access to SCOPE through a computer at home’
‘The principle is good and it should have worked...’


Barriers to use:

Not being able to access the resource bank from computers off campus caused complaints, as did having to print off entire articles.
Printing problems meant that, where possible, many students gave up attempting to use the system and relied on traditional library provision and the photocopier. Other students were put off by hearing about problems and did not even attempt to use the system.
The levels of security required by rightsholders (passwords, one-copy printing, etc.) acted as a barrier to students, in that it made the system appear less user-friendly. It also delayed access, since students had to wait for participating institutions to pass their details to SCOPE before they could be authorised to use the system. This system highlighted the problems that higher education institutions can have with timely identification of students, and with providing this data in a usable format.

5.1 Preparation:

Library representatives are usually too busy with their own work to promote SCOPE to all interested parties. Therefore this has limited the uptake of SCOPE services. Ideally, the Liaison Officer should work more directly with the lecturers involved, or SCOPE duties should be shared by more library staff.

Lists continue to be passed to the SCOPE office with incomplete details, which delays clearance. This is very frustrating for the SCOPE team since finding the required information often requires sight of the hard copy, and can sometimes only be provided by the lecturer.

Most lists required some prioritisation of requested materials. The substitutions which took place were caused by publishers refusing to grant permission, the need to speed up the clearance process, and by high copyright fees.

Lecturers seem happy with the amount of effort required to produce SCOPE materials, especially when it directly benefits their teaching methods. However, obtaining the lists early enough to get clearance and scanning done before the course starts continues to be a problem.

Library Representatives dislike providing hard copy materials to the technical unit once the course is running, because this prevents student access to materials. The only solutions are to use high quality photocopies or to have materials cleared and scanned before teaching starts.

5.2 Composition/

All new materials produced consisted of commercially published extracts, a small minority of which was public domain. Production of institution and author-owned material was maintained.

5.3 Delivery

All packs were delivered on time, despite the tightness of the schedules arising from late delivery of reading lists. There was some concern from lecturers and booksellers about whether they would arrive in time for the start of teaching, and it would be preferable to try to have packs ready sooner, although this is obviously dependent on getting reading lists earlier and copyright clearance going to schedule. This problem is less serious for online materials than for packs, since online materials can be mounted as permissions are granted (thus preventing bottlenecks, rather than having to wait for all items to be cleared before they can be made available.

Online delivery was thought to be a good idea in principle, but was hampered by initial technical problems, which meant that printing was unreliable and time consuming. There were also complaints about being unable to print selected pages. Printing problems were the most common cause of complaint. PDF files composed of images are not suitable for delivery over the internet to networked printers, due to their large size. It is highly recommended that students receive some sort of SCOPE training. The minimum requirement is a demonstration using the powerpoint presentation designed for this purpose.

5.4 Sales:

Print runs, with the exception of Sociology(b/c), Economics, and the Engineering pack, were appropriate to the numbers of students and had a very high percentage uptake. The remaining packs can probably be sold next year.

Use of the resource bank was hampered by technical problems, and also by late mounting of materials. These technical problems are expected to be less significant in the future as institutions gain more experience of this method of delivery.

5.5 Cost:

There was a wider spread of course pack prices this year, but the cheapest and the most expensive both had a high uptake. It appears that price is, within reason, far less important than the lecturer’s recommendation and the integration of the material into the course. No conclusions can be drawn from the cost of online provision to students, since no students were directly charged for the materials.

5.6 Binding/Covers:

There continue to be complaints about the covers of the packs, but it is unknown whether students would be willing to pay the extra money to use stronger cardboard. Similarly, there were complaints about the durability of the spiral binding.

5.7 Relevance:

Satisfaction with the usefulness of most material was very high, although this did not guarantee usage. Incorporation of material into the course (i.e. it is directly relevant) has more effect on usage, and increases perceptions of its value.

5.8 Use:

Material generally enabled lecturers and students to focus teaching and learning on specific, key readings, and also to prepare students for assessment. In addition, the material helped introduce students to new subjects and increased their confidence because they knew that they had access to essential materials.

5.9 Effect on Alternative Provision:

It is impossible to draw any firm conclusions about the effect that SCOPE materials have on other provision. The results from Sociology(b) course indicate that students who buy SCOPE packs also buy other books. However, at Institution 7, the sales of the recommended book from the campus bookshop halved. It may be that where essential reading is integrated into the course, sales of other books can be affected, but where SCOPE provides general reading, there is no obviously detrimental effect on commercial sales.

Library issues do appear to drop when SCOPE resources are used. The effect is most obvious where students are directed towards specific items rather than general reading. Although the effect is not pronounced enough for definite conclusions to be drawn, Library staff feel that use of SCOPE does reduce pressure on library stock.

5.10 Electronic provision

SCOPE’s first experience of providing online information has been a success, despite the early difficulties which arose with printing. Students and staff support the idea in principle, and are enthusiastic about the possibilities once initial printing problems are solved.

 

• Lack of flexibility in clearance and production schedules has raised concerns amongst all parties. This can only be resolved by reading lists being presented to SCOPE at an earlier date than currently occurs, or by clearance procedures becoming faster.
• Uptake of SCOPE services by students was generally very good, especially where lecturers promoted the service and materials were integrated into the course.
• Demand for library services as a whole cannot be significantly altered until more courses use SCOPE.
• Electronic delivery of materials experienced some teething problems, but is viewed with optimism by students and lecturers.
• Access to computers is perceived as more of a barrier to use than technophobia or lack of computer literacy.
• The most successful use of SCOPE materials occurred when the materials provided were directly relevant to teaching sessions. These materials were far better received and used than those which were general background reading.
• The main benefit of SCOPE materials to teaching and learning is that it allows improved preparation and focus by students.

Stirling University, 4711: Social Differentiation
Report on end of semester student questionnaire, 4711_97.doc (Helen Pickering)*
Lecturer’s Report, 4711ab97.doc (Alison Bowes, via email)*

Stirling University, 47JF: Child Welfare Policy and Practice
Report on end of semester Student questionnaire, 47JF_97.doc (Helen Pickering)*
Lecturer’s Report, 47JF_ac.doc (Chris Turner, via email)*

Stirling University
Student Focus Groups report (Neal Hazel, Stirling Uni.)*

Stirling University, 41MG: Banking, Finance and Money
Lecturer’s report (David King, via email)

Stirling University, 4712: Social Problems
Report on end of semester student questionnaire, 4712_98.doc (Helen Pickering)*

Glasgow Caledonian University, Document Production Using Microsoft Word 6 & Manufacturing and Materials Technology
Report of students questionnaire (Mary Kelly, GCU)
Report of student Focus Group (Keith Hunter for GCU)

University of St Andrews, LM2003: Plato and Mind
Report on student questionnaire, lm200397.doc (Helen Pickering)*
Lecturer’s report, lm2003ac.doc (Stephen Read via email)
Lecturer’s report, (Stephen Read)*
Bookseller’s report and library report, lm2003lr.doc (Cate Newton via email)

University of Abertay Dundee, Sociology 3
Report on student questionnaires, abty_97.doc (Helen Pickering)*
Evaluation of SCOPE packs (Lecturer’s report, Norman Gabriel)*
Focus Group comments (Alison Gordon)
Distribution report, abtydist.doc (Alison Gordon, via email

University of Aberdeen, EG1055 Energy Conversion
Librarian’s report (Angela Carr, via email)

Queen Margaret College, Communication and Information Science
Librarian’s report (Vicki Cormie, via email)
Lecturer’s report (Virginia Cano, via email)

Napier University, Electronic Information and Society
Report on student questionnaire, napier98.doc (Helen Pickering)*
Lecturer’s report, nap_ac.doc (Alistair Duff via email)
Lecturer’s report, nap_ac2.doc (Helen Pickering)

Library Representatives
Report on Focus Group (John Daley for SCOPE)

* Available on this site