Abstract

This paper reports an experiment in flexible delivery techniques using 'budget' multimedia resources with a group of first year university students. The subject matter concerned introductory computing knowledge and skills which all students must study. The results of the study confirm previous studies indicating no significant difference in learning as a consequence of the delivery medium. Qualitative feedback indicated generally positive attitudes and this coupled with obvious advantages in terms of flexible delivery, encourages us to provide further learning modules.

Keywords

Multimedia, self-paced, flexible delivery

1 Introduction

This paper describes the implementation and testing of a multimedia learning module in an introductory computing subject at the University of Western Sydney, Macarthur. The subject is a first year level course of study concerning the basic elements of computer systems which is taken by all students in the Faculty of Business & Technology. The subject fulfills, in part, the need for all students to satisfy university policy concerning Computer Competencies and typically has 500 - 800 students enrolled. A commonly held view is that such material could and should be offered in a self-paced mode of delivery so that students can learn material as and when needed. Additional motivations include relief of the burden on teaching budgets, and to 'free up' time in introductory subjects.

Students entering university have a wide variety of skills, nowhere more evident than in the use and understanding of computers. There is clearly a strong need for self-paced flexible learning systems to cater for this variation. In addition, large tutorial/practical classes necessitate employment of additional staff to enable effective learning. Moreover a significant proportion of our students have claimed that the textbooks are boring - they are unable to learn effectively from them. The university draws most of its students from the Western areas of Sydney where university entrance scores are generally in the lower parts of the spectrum. Such students are less accustomed to intensive 'book learning'. Issues such as these motivated us to explore means of enabling more efficient, interesting and cost effective learning. Studies of Computer Assisted Learning reported elsewhere have indicated both positive and negative outcomes. Kulik and Kulik (1991) reported positive effects in student attitudes while Boyle etal (1994) demonstrated a reduction in failure rates. Langham and Johnson however give evidence of too much visual detail impairing student learning by overloading learner's working memory. In 1995 we set out to conduct an experiment to gather more evidence in relation to these suggestions.

2 Materials & Methods

The introductory computing subject deals with a number of topics, only some of which would lend themselves to a multimedia delivery. We chose Input/Output Devices in order to maximise the opportunity to use video and audio clips to stimulate student interest. This topic has no particular hierarchy or sequence of concepts and thus lends itself to a hypertext-based exploration of the learning space. Following selection of the topic, the concept space was mapped out and classified in terms of its potential use of text, graphics, animation, still images, video and audio. As the learning materials were delivered on IBM PC 486 computers with 8MB RAM the system was developed on a similar computer with 1.2GB of disk space. Video was shot using a Video 7 camera and captured using a Video Blaster card and Microsoft Video. The actual learning material was compiled using Microsoft Viewer. The final set of materials occupied 70MB of disk space, this included 20 short video clips (one quarter screen size) and 30 bitmap images.

Production of self-paced learning materials is labour-intensive and hence expensive. Only very large audiences would enable cost savings for sophisticated multimedia productions. In this experiment we set out to generate adequate as against 'state of the art' materials because of resource limitations. The reality at UWS is that no-one will produce superb, commercial level productions in any cost effective manner. Thus, one aspect of the study was to see if students would accept, and be able to learn from the standard of production which could be achieved with a modest effort in a relatively short time frame. The module in the present study required approximately 50 hours of preparation time using resources costing less than $2000. Figure 1 shows an illustration of a typical screen.

Figure 1: A typical screen from the learning materials, showing a video window, text, control buttons and hot words (hyperlinks).

Students were randomly selected and asked if they would participate in the study. Those who agreed became part of the test group, the rest constituted the control group. We used random selection instead of simply calling for volunteers, in order to avoid biases resulting from selecting students already favourably predisposed to computer-based work. Students were not compelled however, so some bias conceivably may still exist. The test group studied the topic in the computer laboratory using the multimedia system, whilst the control group covered the same material in 'chalk and talk' fashion in the lecture room. All groups were then tested on the topic using a standard written test. In addition the multimedia group was given a questionnaire to measure more qualitative aspects. The test group comprised 25 students whilst the control group numbered approximately 185.

3 Results

3.1 Quantitative

The students studied 6 modules, the sixth of which was delivered via the multimedia package. At the end of each module students took a written test. If they scored poorly, they were allowed to attempt another test later.

Module	1	2	3	4	5	6 M
% students with 2nd attempts Test group Control group	28 21	16 18.3	32 23	20 18.5	48 22.3	16 10.3

Table 1: The percentage of students that required a second attempt to pass each of the six modules. The multimedia module was module 6.

Module	1	2	3	4	5	6 M
% students failing Test group Control group	0 11.1	4 13.4	16 16.8	20 17	20 28	12 11.4

Table 2: The percentage of students failing each of the six modules. The multimedia module was module 6.

The results summarised above in Tables 1 and 2 confirm studies carried out elsewhere which showed no statistically significant difference in quantitative outcomes between groups using the multimedia materials and those which did not. In terms of flexible delivery, however, this is encouraging, as the self-paced multimedia appears to be just as effective as more intensive teacher-centred classes.

3.2 Qualitative

Qualitative testing revealed generally positive feedback as shown below:

3.2.1 Attitudes/Reactions

• 96% indicated that they felt they had learned something worthwhile from the package.
• 76% indicating that they would also like to study other topics using this approach.
• 20% of the students indicated that the experience had positively changed their view of what could be accomplished with computer mediated instruction.
• typical comments were:

'quicker and easier to use than a textbook'

'more interesting'

'use at your own pace'

'good for those who miss lectures'.

3.2.2 Comparison with Traditional Methods

• 52% indicated that this method of instruction was preferred to the lecture method with 8% indicating a preference for lectures.
• 48% indicated that this method was preferred to tutorials however they did confirm the notion that human interaction is useful too and that all tutorials should not be totally replaced with computer mediated instruction.

3.2.3 Interface Issues

• 100% of students found the package easy to use in terms of navigation, and comprehension
• The features cited by students as most interesting were: video in conjunction with text, spoken information, animation and ability to move easily from topic to topic.
• Features cited as negative included, lack of music, too much text and quality of video.

3.2.4 Suggested Improvements

Students suggested the following improvements to the package:

• Follow the same layout as the textbook;
• Include a quiz at the end;
• Use only material which is to be tested.

Clearly the students were very assessment-oriented in their consideration of the improvements!

4 Summary

Quantitative and qualitative feedback obtained from the experiment was encouraging. Students appear to find the delivery medium a useful one for learning and staff can see obvious advantages for increased flexibility in revision and for students whose study gets out of sequence. It seems unlikely that students will experience better learning via a computer-mediated package than with a good human teacher. However, quantitative results suggest that it is not significantly impaired. This coupled with obvious advantages in flexibility for students working outside scheduled class hours encourages us to explore its further use. The limited budget did not appear to pose overwhelming problems, only 16% of students cited the quality of the materials (images/diagrams) as a problem.

5 References

Boyle, T, Gray, J, Wendl, B and Davies, M (1994). Taking the plunge with CLEM: The design and evaluation of a large-scale CAL system. In Kibby, M R and Hartley, J R  (Eds.), CAL into the Mainstream, Computers and Education, Vol. 22, No. 1 / 2, Pergamon, pp. 19-26.

Kulik, C C  and Kulik, J A  (1991). Effectiveness of computer-based instruction: An updated analysis, Computers in Human Behaviour, Vol. 7.

Langham and Johnson (1984). In Barker, J and Tucker, R  (Eds.) (1992). The Interactive Multimedia Revolution, Kogan Page, London.

Acknowledgements

The authors wish to thank the Faculty of Business & Technology for providing the grant in support of this experiment and the Computer Centre for providing facilities for computer-based instruction.

Copyright

Mark Johnston, Graeme Salter, Kevin Donegan, Paul Davies © 1996. The authors assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 96 conference papers, and for documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.