|
|
SPACER SPACER SPACER |
SCOPE Project Report on student focus groups:
|
Evaluative Report on the Autumn 1997 SCOPE Electronic Reader:
- Student Focus Groups
Report Contents: Page
A Introduction 1
B Purposes and principles 2
1 Flexibility of working practices 2
2 Concern over restrictions to reading 2
C Content and relevancy 3
D Patterns of use 4
E Relationship between Electronic and Printed Readers 5
F Problems encountered 5
1 General perceptions 5
2 Access to computer terminals 6
3 Logging on and accessing the Reader 7
4 Procedural printing problems 8
5 Technical printing problems 8
6 Technical assistance and training 9
G Concluding concerns 10
A Introduction
Within this report, emphasis has been placed on evaluating the SCOPE Electronic Reader rather than the printed packs. The intention of the report is to allow the critical voice of the students directly involved with SCOPE to assess the benefits and difficulties perceived and encountered with this new format.
The focus groups included students from each of the Sociology and Social Policy units participating in the SCOPE project, whether or not each student had successfully employed SCOPE material. The evaluative exercise included 55 students in total; 10 students from 4711, 10 students from 47JF and 35 students from 4791. Two focus groups were held involving students from 4711, one group from 47JF and one group of students completing 4791. Sessions lasted between 30 and 50 minutes and were held in the last week of the taught element of each course.
B Purposes and principles
1 Flexibility of working practices:
In general, the idea of the SCOPE Electronic Reader was welcomed by focus group participants. Perceived purposes and advantages related closely to those cited for the printed packs in previous years. Primarily, students focused on the flexibility of study practices which SCOPE would allow, meeting the various demands of the heterogeneous student population registering for courses.
Students studying for both the introductory and advanced Sociology unit highlighted the flexibility and freedom to choose the time to study which SCOPE would provide. It was noted that the necessary life-patterns and external commitments of those within the current student population meant that the time available for study would not always coincide with the library opening hours. SCOPE permitted access to reading resources at other times:
It's good because you haven't got the RBR for 24 hours a day. (47JF)
It was recognised that the electronic reader could operate on a day-to-day basis independently of library staff. In addition, students wouldn't be restricted to travelling to, and working within, the confines of one building:
In principle it has many, many, many advantages. Because getting it off the computer is the easiest way to get reading. It also doesn't involve actual travel to the library, involve library staff. You can do it at any time when the library is closed. It gives you a lot more freedom. (4711)
Participants referred to the confidence and freedom presented by the knowledge that SCOPE articles would be available when required, and not unobtainable due to a lack of resources or 'seasonal' high demand around, for instance, essay deadlines. This was contrasted with the practical problems of securing more traditional library materials:
You know it will be there. Somebody hasn't taken it out and you are waiting for it to come back. And people haven't torn out the various odd page, preventing vandalism. (4711)
As noted in previous years with the printed packs, these advantages were felt to be particularly useful to Evening Degree students, other part-time students and those living away from the University campus. It was felt that the limited opportunity those students had to visit the library meant that particular benefit could be drawn from the immediate availability of electronically stored material.
2 Concern over restrictions to reading:
Again reflecting earlier comments relating to the printed Readers, a certain amount of concern was expressed that the SCOPE Electronic Reader may actually inhibit use of the library and threaten the acquisition of valuable research skills or an appreciation of the breadth of available reading:
It might discourage you from looking on the shelves to see what else might be there. (4791)
As such, some students indicated that one of the reasons that they didn't access the SCOPE resources was so that they concentrated their research efforts on developing their 'library skills'. This was seen as particularly important for the introductory units:
One of the reasons why I didn't do it was that I wanted to get familiar with how to use the library. Just so that I could feel my way a bit. (4711)
Rather more worrying was the indication by a number of students that they embraced this implicit inhibition of library research because the Reader was perceived as a prescription by the lecturer of all the articles which were seen as strictly relevant or important reading. The Reader was essentially seen as a core textbook which would provide the reading necessary for the course. Participants even indicated that they inferred the pieces chosen for the Electronic Reader as a direct indicator of exam topics and questions. Students implied that the content of the Reader determined the content of their own reading:
It does limit your reading to a certain extent I suppose, but with this early part of Sociology I don't think that's a bad thing. I assumed that because the lecturers chose those specific pieces as being the relevant ones, then it gave some indication as to what were the important articles within the reading list. (4711)
More widely expressed was the more moderate perception that the Reader could be seen as giving some indication of relative importance of reading, like the list of suggested reading supplied by lecturers, but that this list should be seen only as a starting point for further study:
It's not saying that you should only read this, it is just the pieces that are the most essential. So you know you read those and then get additional reading. (4711)
C Content and relevancy
A few negative comments were received from students regarding the limited extent of material available through the Electronic Readers. It was felt that the small number of articles resulted in too few pieces relating to each area of the courses. In particular, some students were unhappy with the content of the Reader because they were searching for specific pieces of reading. The students were aware of particular articles featured on the course reading list or in the Reserve Book Room photocopy packages, and expected SCOPE to be an alternative source for articles of such perceived importance. They were clear that only a much wider 'pick 'n' mix' selection in the Reader, reflecting the RBR photocopy package selection would facilitate this need:
I was looking for things that were available in the reserve book room photocopies. There were not the things on that I was looking for at that time. I was looking for one and that wasn't on it, and it was out in the RBR as well. (4711)
I went in thinking, 'I want this piece or this piece or this piece', and there wasn't any of them in the computer.... there was one of them in the reader, the book. (4711)
Converse to concerns over the prescriptive nature of the Reader, some students felt that only a small number of those articles which did relate to each topic could be seen as strictly relevant to their needs:
There wasn't enough information on there. There were only a few pieces of information that could be used ... There just wasn't as much to it, just a couple of pieces that I'd have gone to the library to have got out. (4711)
The printed packs were felt to have material more relevant to the lectures, perhaps because the choice for material for the packs was made before the Electronic Reader. As such, it was suggested that the Reader may be made up of 'reserve material' which weren't as important to the structure of the courses:
The SCOPE packs seemed to contain the pieces which were most often mentioned in the lectures, whereas the reader on the computer just seemed to be random pieces from the reading list and not so much as the mentioned ones. (4711)
D Patterns of use
Students who had accessed the SCOPE electronic resources indicated that the articles featured were being used in a wide variety of capacities. These uses did, however, seem to be influenced by the nature of the course being studied and the status of the student. The day students seemed to utilise the Reader for more specific elements of their study schedule, whereas Evening Degree students indicated that they found it more useful for general background reading throughout the course.
Both Advanced Unit and Introductory Unit students indicated that they used, or would use the Reader for obtaining material for their essays and exams. In addition, first semester students noted the usefulness of the Reader to preparation for Workshops on a particular topic.
Students taking Introductory units were prepared to use the articles as reference material to note from the screen, particularly when searching for useful quotations for essays or Workshops. However, Advanced students indicated that such a practice would not give them the time nor practical opportunity for appropriate critical study when dealing with complex and lengthy articles:
I took notes from the bit that I saw. It has been useful ....The amount of notes I had to take from it were quite small. There was one particular statement on one particular page that I was looking for. (4711)
There was too much of it on the screen. You couldn't take notes. And it would lock it up for people wanting to use it. And it's not the same, highlighting bits you need when it's on the screen. (47JF)
In addition, Evening Degree students stated that the short time available to them at the University prohibited spending lengthy periods taking notes at the computer.
Again the main restriction on the patterns of use appeared to be the students' perceptions that there were not enough articles relevant to their needs:
I would have used it for everything, but it didn't have the right reading. (4711)
E Relationship between Electronic and Printed Readers
Overall, the indication from participants was that the existence of the Electronic Reader had little effect on deciding whether or not to buy a printed pack. The students saw them as separate entities with different contents and serving separate purposes:
It didn't really seem to be the same pieces of information. (4711)
Students indicated that the printed packs were still important because of the geographical restrictions to accessing the Electronic Reader each time they needed an article on a different topic. The Electronic Reader articles felt to have to be printed off when they were needed, not in their entirety at the start of the course. In contrast, the printed pack was permanently close to hand as a source of reference, particularly for those living off-campus:
I would have bought the packs even if I'd have accessed the computer, because it was easier to read them than keep on coming down here to access SCOPE on the Internet. (4791)
There was some indication that students were more inclined than in previous years to obtain copies of the printed packs through wider means than a bookshop purchase. Students indicated that they had borrowed packs from those completing the courses in previous years, or occasionally from other current students. There was also the suggestion that packs were now available for purchase second-hand, advertised on University notice-boards.
F Problems encountered
1 General perceptions
The overall feeling indicated by participants was that practical and technical problems with the Electronic Reader had marred and eclipsed its purpose and usefulness. The problems encountered resulted in the Reader not being able to fulfil its promise or potential during this semester, leaving a rather bitter taste for many students:
It has been a nightmare from start to finish. This is the first day that I have been able to get anything off it. It has not been as helpful as it ought to have been. (47JF)
Particularly in relation to the Introductory Units, the Reader was seen as presenting unnecessary problems for new students, best avoided once difficulties were encountered:
I think people were using it early on in the course, run into some problems and stopped using it. (4711)
Once students on the courses began to come up against such problems, other members of the course were less inclined to even investigate the Reader. It appears that word of technical problems spread fast, soiling the reputation of SCOPE amongst students and prohibiting extensive use:
You've got the whole rumour thing going around. I didn't use it because my friend said that it's just a nightmare trying to use it. He couldn't print out and it would just freeze. (4711)
I never went up at all because I heard stories coming back about it so I never bothered going up. In the end I just didn't bother. The feedback coming back was, 'don't bother'. (4791)
There did not, however, appear to be a problem with general student awareness of the SCOPE Reader per se. Participants indicated that there was enough publicity in lectures and course material to ensure that students would know about the Reader:
It was in the pack and it was pointed out at the beginning of semester. I think most people knew that it was there. (4711)
There was some evidence that students may have been deterred from using even the Electronic Reader because of financial considerations. Problems with late payment of Maintenance Grants meant that students were operating a strict priority schedule on financial demands which excluded the use of SCOPE resources:
I just didn't want to pay out for anything, not even paper. I was just totally skint. There were other courses which I had to buy books for. (4711)
2 Access to computer terminals
As predicted by focus groups in previous years, there was a degree of complaint from students that they were unable to gain access to computer terminals in order to view or download SCOPE material. A number of participants commented that the limited number of terminals on which SCOPE could be accessed meant that they were often being used, quite often by students completing other work.
We've been back a few times when there hasn't been a spare computer to take the stuff off. (47JF)
Again, there is evidence to suggest that this led to some students abandoning attempts to utilise the SCOPE Reader:
I thought that it's more of a waste of time going back and forwards trying to get on the damn thing when you could be trying something else. (4711)
Participants referred to organised lab classes as one specific obstacle to terminal access:
Certain times of day there are classes or all the computers are busy. (4711)
In relation to this problem, and in connection with a wider call for maximum flexibility, participants in every focus group voiced their support for the development of remote access to SCOPE resources. It was felt that the project could never reach its full potential until students were given the freedom to download material away from fixed locations on the University campus:
You are still restricted with SCOPE. You can't get it on the Internet at home and that is what I was hoping to do. So you have got to use the lab on a Monday evening. (4791)
It defeats the purpose of easy, remote access. It would be a lot easier. (4791)
I found that it really defeated the whole purpose. It was as easy to go to the library as it was to access SCOPE. In fact easier. (4791)
As the above quotations suggest, remote access is viewed as particularly important to part-time students and those living away from campus. Echoing the reasons given in previous years for the usefulness of the printed packs to these groups, participants argued strongly for the development of remote access to be given priority in the project:
I think that it is a great idea, but you should also be able to get access to SCOPE through a computer at home, for people who find it harder to get in or have other commitments. Time saving. You can use the SCOPE, and then when you have a lecture, you can come in and use the library as well. (4711)
Some students were not convinced that problems involving copyright were fixed. Various suggestions for ensuring security were made:
What I really wanted to do was access the SCOPE reader from my own computer at home. It would be quite easy to sign a little piece of paper saying that we weren't going to reproduce anything that was on the SCOPE reader. Make a legal undertaking if necessary. (4711)
A straw poll conducted during such a discussion with the Evening Degree focus group indicated that about half of the 35 students have more convenient and regular access to the Internet from outside the campus. Evidence from the participants also suggests that students were unaware during this course that remote access was not possible. Many had actually tried to access the resources, and either realised the situation or blamed the failure on technical problems:
I tried to get into it from home and I was quite disappointed. (4791)
3 Logging on and accessing the Reader
Participants indicated that there was a degree of problems trying to access the SCOPE resources:
I found it difficult to get in, and I'm not frightened of computers. (4711)
Problems seemed to surround the interface for the logging-on procedure rather than general problems with the browser or operating system. Students highlighted experiencing difficulties successfully completing the registration form with their personal details, arguing that the time spent on trying to get into SCOPE could have been better spent in the library. It was felt by a significant number that the access problems diverted their attention away from studying the substantial academic subject:
I gave up. I spent an hour and a half trying to get into that. I was so disgusted that it was an hour and a half that I could have been in the library and read a book. (4711)
I found that I would waste so much time trying to figure out the computer that I would totally lose track of what I was doing with my Sociology (4711)
4 Procedural printing problems
A number of practical problems surround the process of printing were outlined during the focus group sessions. These generally related to the procedures for loading the student's own paper into the printer. A few participants commented that they didn't realise that they had to take their own paper along. As such they felt the frustration of achieving access to both a terminal and the Reader, finding the relevant article, but being unprepared to produce the final goods. In addition, there was some confusion surrounding the inclusion of the statement indicating the cost of the print to the library, and evidence that this confusion deterred some students from printing:
I had no paper. So the thing was up on the screen in front of me and I couldn't print it. (4711)
I didn't realise that you've got to take your own paper along. And it came up that I had to pay £2.65, even though I had my own paper, so you've got to pay for that as well ... The way I read it was that I had to pay. (4711)
It should be noted that there was a request for the printed pages to have smaller margins in order to help reduce the number of pages per article. It was felt that articles for the Advanced Unit were using a lot of paper, and blocking print queues. (47JF)
Concern was expressed that some students had experienced situations where their paper had been ready in the printer, but used by somebody else at another terminal:
I heard that somebody's paper was used by somebody else. Obviously it wasn't intention, it was inadvertently done. (4711)
You put your paper in and you are waiting for it to print off. When you hear the printer going you think, 'good' but when you go up it is for somebody doing business studies. And they've used your paper. (47JF)
It should also be noted that there was a request that the interface be reviewed to eliminate the need to repeat the process of filling in personal details after every print (47JF). In addition, one student indicated that they had been deterred from printing because of a security message from the browser:
It kept on asking if I really want to send this because everyone will know your work, 'Are you sure you want to access because it's not secure'. (4711)
5 Technical printing problems
Problems experienced whilst trying to print articles from the SCOPE reader accounted for by far the biggest number of concerns and complaints from participants. Students reported frustration, feelings of isolation, anguish, distress and a loss of confidence with computers as resulting from such problems:
It is just so frustrating. You can spend hours up there and get nowhere. I keep trying the same thing over and over thinking, 'it's me, it's me'. But it's not us, it's the system, it's not working. (47JF)
When you've got other things to do with your time, you've got other reading and preparations to do for tutorials. You say, 'right, I've got these two hours to spare, I'll go and get these documents off'. When you go up you come out of there exasperated, frustrated and you feel like going home and crying. (47JF)
Mirroring suggestions relating to time spent trying to locate free terminals and access the Reader resources, students from all courses complained that they had spent lengthy periods of them trying to print articles:
When I started this course it was completely new to me. I was a little bit frightened but I thought that I'd give it a go. But I spent hours and hours and hours up there not being able to print anything. Or it gives you just the first sheet, and when you go to print it again because it didn't come out, it won't let you. (47JF)
In addition, it was felt an 'unacceptable' length of time taken for the print job to be completed intensified the practical problem of other students using their paper. It became impossible to estimate when students should load their paper. Participants reported having to wait for several minutes if they were alone in the lab, or more than three quarters of an hour if the lab was busy:
I went on it yesterday, and I wasted nearly two hours trying to get off an article. When I asked to print it, you put your paper in and you would be waiting for something to happen, and nothing happened. There is so much of a delay that you are getting frustrated because you are running out of time. (47JF)
I printed off three documents yesterday, but I haven't got them. I put it in for print and it told me that I've got it, so I couldn't wait any longer than about 45 minutes. When I was exiting the program it warned me that I lose the material. (47JF)
6 Technical assistance and training
Participants suggested that some of the procedural problems might have been avoided if there had been a certain amount of training provided on how to access the SCOPE resources, either during a separate information session or as part of a Sociology lecture:
Perhaps if [the lecturer] could have shown us in the lecture time. (47JF)
Alternatively, it was argued that having a staff member around a certain lab for a short period of time at the beginning of the course would help ease both procedural and technical difficulties:
A staff member, maybe just for the first week when everybody is going in. (4711)
The man in the library was really, really helpful. He sat with us for like an hour. Because he couldn't get it to work. (47JF)
Overall, it was felt that additional assistance would help ease students into using SCOPE, and would support the confidence of those students who may be worried about computers or using the Internet for the first time:
It is intrinsically simple, but if people are a little bit worried, then a little introduction set in front of a computer screen would probably make it much more widely used. (4711)
There was, however, less evidence than in previous years that a substantial proportion of students would avoid a computer accessed SCOPE Reader because they were frightened of using computers. Although there were isolated suggestions that there was enough information technology to learn in the first semester at the University, there was a general and quiet acceptance of the Internet interface by even those claiming to be 'techno-phobic'.
I was nervous at first, going into the web. But after a couple of tries you get used to it and it's OK. (47JF)
It should be noted that participants were concerned that this help should not come as part of the wider Infoskills programme. It was felt that this programme is stretched too far at present and that justice would not be done to the needs of SCOPE users:
You don't learn anything because they don't stop for a minute. They just keep going and going and if you get lost that's you. (4711)
G Concluding concerns
Overall, the fairly consensual view of participants was that the SCOPE Electronic Reader would be of real benefit to the current student population. Students supported the aims and principles of the project and identified advantages for the future, particularly with any development of remote access. The participants felt that they were forced to look towards the future because of the significant problems encountered during this semester:
The principle is good and it should have worked. Had it been working from the beginning I'd have been quite happy with it. (47JF)
Indeed, participants were quite forceful with their criticisms of the technical failings of SCOPE during this semester. Particularly worrying were the widespread claims by the focus group taken from the Advanced Unit that the problems encountered have had an adverse effect on their academic progress, and actually resulted in lower grades.:
It's really going to affect my mark. I'm sure that it's really going to affect everybody else's. (47JF)
Participants indicated that, particularly earlier in the semester, they didn't have access to reading material seen as essential for completing their essays satisfactorily as a direct result of problems with SCOPE:
It is available in the library, but for these core pieces of reading there are only one or two books, so everybody has always got them out. (47JF)
There were examples of children's systems on SCOPE which I could have done with for my essay. In fact, that was the main criticism of my essay. But I couldn't get them. We needed that information but we couldn't get it off ... I've got a lower essay mark than ever before, and it's not that I wasn't prepared to work, just that there wasn't the material to work from. (47JF)
The participants suggested that problems were intensified because this was the first time that the course had been run, resulting in only limited copies of key texts being available in the library. It was felt that, when selecting courses to participate in the SCOPE project, attention should be given to ensuring that enough alternative resources are available for all core tasks within the programme in case of technical or procedural failure:
There should have been contingencies for SCOPE not working like
a pack, or more books in the library. (47JF)
Neal Hazel - 19 December 1997