Evaluation Documents SCOPE home page

SPACER SPACER SPACER

SCOPE Project Report on end-of-semester student questionnaires: Electronic Information and Society (Alistair Duff) Napier University Spring 1998

Electronic Information and Society is a second year module, running throughout the year. 16 documents were accessible via by the SCOPE resource bank from the start of the second semester. Students accessed the resource bank from a PC lab at Craighouse Campus, where the course was taught, and could also use computers in the library. Printing was free and paper was supplied. Some of the material was also made available in the library, but a significant proportion was only available through SCOPE or via ILL. Students were told by the lecturer that the resource bank was a compulsory part of the module.

A draft questionnaire was designed by SCOPE and sent to the Library Representative for local modifications. This was not entirely successful, since several questions were included which are not directly relevant to students at Napier:

Q.1b - no students live on campus at Napier’s Craighouse campus.

Q.3c&3d - the service was only accessible from computers in one laboratory, and not at all from the library.

In addition, circumstances changed rendering Question 11.g. (Value for money of SCOPE) obsolete: students were to have been charged for printing, but due to a problem with the relevant printer, the service was free.

The questionnaire was generally filled in adequately, with some confusion arising from the above questions, and also several students indicating that they had not used SCOPE, but still evaluating it; or indicating that they had used SCOPE, but not including the resource bank in their list of reading sources (Question 13.)

173 students were authorised to use the resource bank. The lecturer distributed questionnaires to students during May 1998, and 116 were returned to SCOPE for analysis during June - a sample of 68%.

The student profile was fairly homogenous, with 93% being full-time, second year students living off campus. (Tables 1.1, 1.2, 1.3) 90% of students had used the internet prior to starting the course, with the largest group having used it more than once a week. (Table 1.4) This is probably due to the fact that they were almost all in their second year of study. Previous usage did not seem to have influenced the ease of obtaining a computer password (Tables 1.5 and 1.6) although there may have been some confusion over whether this was referring to their network password or their SCOPE password.

Table 1.1 Q.1a. Are you a full-time or part-time student?

Table 1.3 Q.1c. What year are you in at Napier University?

Table 1.2 Q.1b. Do you live on campus?

Table 1.4 Q.1d. Did you experience any problems obtaining your password for computer access?

Table 1.5 Q.1e. Please tick the box which reflects how often you used the internet prior to starting this course.

Table 1.6 Password access against internet use

Of those students who responded to Question 2, 94% had used the resource bank (Table 2.1). Of the 12 who did not answer the question, subsequent responses indicate that 11 of them had used the resource bank, maintaining total usage at a probable rate of 94%. The main use of the resource bank was for printing documents - 91% of students indicated that they had printed from the system, it is likely that unsuccessful attempts have not been included. 89% had also used the resource bank to view the documents. Fewer students had searched the text on screen - 64%. (Table 2.2)

The 94% usage rate derived from the questionnaire responses (Table 2.1) indicates that the sample is biased towards those who used the resource bank: data from resource bank usage indicates that 139, or 80% students viewed items in the resource bank. Database analysis also show that 114, or 66% of students on the course printed documents.

Table 2.1 Q.2. Did you use the resource bank?

Table 2.2 Q.3. When you use the resource bank, how often do you:

69% of students indicated that they would access the resource bank from outside the university (Table 2.3) and 82% would prefer the option to print parts, rather than the whole of the document (Table 2.4).

Table 2.3 Q.4. If the Resource Bank was accessible from computers outside of the university, would you use this facility?

Table 2.4 Q.6. Would you prefer to be able to print selected pages from the documents, rather than the whole document?

82% of students indicated that the length of the document affected their decision to print it out (Table 2.5). There is some quantitative evidence to support this - four documents which were all more than 20 pages long had below average print rates, and of these four, the greater the length, the fewer printouts occurred. However, only 16 documents were provided by the resource bank, which is not a large enough sample to prove a conclusive link, since perceived importance of the document and access to alternative sources will also have played a part. Library accessibility of documents does not seem to have affected print decisions: only 7 of the sixteen items were available in the library, and one of these was the joint most heavily used piece in the resource bank and the least used document was not available in the library.

Table 2.5 Q.5. Did the length of the documents affect your decision to print them?

Although 11% of students never had problems printing (Table 2.7), 32% stated that they always had problems. Of these 32%, disappointingly few stated that they got the database reset to allow another printing attempt: 26% never bothered with resetting (Table 2.8). This may reflect the inconvenience of this procedure since one student stated that they did not know that this procedure existed, and another saying that they had not bothered due to unhelpful staff (Table 2.9, Appendix 1: Problems). Only two successful resets were carried out at Napier. According to the database, 41 students accessed a total of 80 documents which they had previously attempted (successfully or otherwise) to print. Their motives for doing so are unknown: they may have simply been reviewing text on-screen if they did not have the paper copy with them, but it is probable that some, if not all, account for failed prints. The worst case scenario is that of the 399 printouts, 54, or 14%, failed.

Table 2.6 Q.7. If a document failed to print the first time, how often did you ask for the database to be reset, allowing printing at a later date ?

Table 2.7 Q.8. How often do you have problems trying to print documents?

Table 2.8 Database resetting against printing problems

Of the 116 students, 74% commented on problems they had encountered. Printing problems were by far the most common complaint, followed by access to the computers and the internet. (Table 9a) Many students highlighted more than one problem. (see Appendix 1 for full list of problems) Print times of half an hour were not uncommon, and one student waited 3 hours for a document to print. Other printing problems included partial printing of the documents, and abuse from other computer users who wanted to print non-SCOPE materials. Only two students noted that printing problems were the responsibility of the university rather than SCOPE. The Department has been presented with three possible solutions to speed up printing (upgrading the print server; use of Apple MAcintosh computers which do not use the print server, and upgrading to Windows 95), and it is hoped that at least one will be implemented in time for the next academic year. Cost was also mentioned as a problem, despite the fact that the service was free to students.

Table 2.9 Q.9. Please outline any problems you have encountered when using the SCOPE Resource Bank.

Only 24% students indicated that they had used more than 50% of the resource bank (Table 2.10). According to the database (Table 2.11), each student printed an average of 3.5 printouts, although 60% of students printed 3 or less items, and only 8% students requested printouts for more than 50% of the resource bank (Table 2.11).

It is less clear how much on-screen use was made of the data. On average, students retrieved 7.5 documents each, which is only 47% of the documents available. However, as has already been discussed, 24% of students requested the same document more than once, so the actual usage is likely to be lower than this. The lecturer expected students to look at all the SCOPE readings online, but to print out and master a minimum of 3 or 4.

Table 2.10 Q.10. What percentage of the material from your reading list in the resource bank did you access?

Table 2.11 Printouts per student (derived from database)

Satisfaction levels with the service(Table 2.12) were over 50% for content and layout of the materials. 85% were satisfied with usefulness, 76% with layout, and 66% with the range of material. As reflected by the comments, (Table 9 and Appendix 1) the technical aspects caused most dissatisfaction. Only 7% were satisfied with print time, 33% with access to the computers, and 40% with access to the resource bank. Satisfaction levels with value for money were of little relevance since the only cost to students was time.

Table 2.12 Q.11. For this question, please tick the box which reflects your level of satisfaction with each aspect of the SCOPE Resource Bank

74% of students indicated that they would rather have had the materials available as a pack, although several elaborated that they would prefer free handouts in class, or several small packs.

Table 2.13 Q.12. Would you rather have had the materials in the Resource bank sold as a pre-printed pack?

The most common method by which students obtained reading for this course was from the library, with 84% using at least one of the library’s collections. The SCOPE resource bank was used by 83% of students (Table 2.14), although this may be higher, since five students who indicated that they had used SCOPE did not list it as a reading source. The comparable levels of SCOPE and library usage are not surprising given that only 7 of the sixteen items in the resource bank are available in the library (as discussed above). Books were purchased by 33 students (29%), and half those who bought second-hand books also purchased new ones. Other sources of reading included the internet, newspapers and journals. There is some evidence of sharing of SCOPE resources: of the 6 students who stated that they did not use SCOPE (Table 2.1), two indicated that they had used SCOPE materials. There was no other reading suggested to students, other than SCOPE resources and the short loan collection, although students had to search some web sites.

Table 2.14 Q.13. How are you obtaining reading materials for this course?

46 students indicated how much they had spent money on reading materials (Table 2.15). The most common spending level was less than £10, with 75% spending less than £30.

Table 2.15 Q.14. How much money have you spent on text books for this course ?

There was little variation in the student profile of this class, so no comparisons were possible between different types of students.

80% of all students accessed the resource bank, and 66% printed (or attempted to print) materials from the resource bank.

Whilst students were generally happy with the contents of the resource bank, the technical aspects of accessing it, and especially printing caused severe dissatisfaction. There were calls for more flexibility in terms of accessing and printing the materials, and for the materials to be made available earlier on in the course. The major concern is the printing facility which needs to be greatly improved.

Issues which were not covered in the questionnaire, but students raised, included access to technical support, and having to submit separate authorisation forms for each item requested. In addition, one comment indicates that the Technical Unit was not informed of students who registered late.

SCOPE and library usage were at approximately the same level. SCOPE usage was affected by a number of factors including length of documents, accessibility to the documents via other sources, and perceived importance/relevance.

The high percentage who indicated that they would prefer packs

Appendix 1 Comments