This paper reports an experiment in flexible delivery techniques using 'budget' multimedia resources with a group of first year university students. The subject matter concerned introductory computing knowledge and skills which all students must study. The results of the study confirm previous studies indicating no significant difference in learning as a consequence of the delivery medium. Qualitative feedback indicated generally positive attitudes and this coupled with obvious advantages in terms of flexible delivery, encourages us to provide further learning modules.
Multimedia, self-paced, flexible delivery
This paper describes the implementation and testing
of a multimedia learning module in an introductory computing subject
at the University of Western Sydney, Macarthur. The subject is
a first year level course of study concerning the basic elements
of computer systems which is taken by all students in the Faculty
of Business & Technology. The subject fulfills, in part, the
need for all students to satisfy university policy concerning
Computer Competencies and typically has 500 - 800 students enrolled.
A commonly held view is that such material could and should be
offered in a self-paced mode of delivery so that students can
learn material as and when needed. Additional motivations include
relief of the burden on teaching budgets, and to 'free up' time
in introductory subjects.
Students entering university have a wide variety of skills, nowhere more evident than in the use and understanding of computers. There is clearly a strong need for self-paced flexible learning systems to cater for this variation. In addition, large tutorial/practical classes necessitate employment of additional staff to enable effective learning. Moreover a significant proportion of our students have claimed that the textbooks are boring - they are unable to learn effectively from them. The university draws most of its students from the Western areas of Sydney where university entrance scores are generally in the lower parts of the spectrum. Such students are less accustomed to intensive 'book learning'. Issues such as these motivated us to explore means of enabling more efficient, interesting and cost effective learning. Studies of Computer Assisted Learning reported elsewhere have indicated both positive and negative outcomes. Kulik and Kulik (1991) reported positive effects in student attitudes while Boyle etal (1994) demonstrated a reduction in failure rates. Langham and Johnson however give evidence of too much visual detail impairing student learning by overloading learner's working memory. In 1995 we set out to conduct an experiment to gather more evidence in relation to these suggestions.
2 Materials & Methods
The introductory computing subject deals with a number
of topics, only some of which would lend themselves to a multimedia
delivery. We chose Input/Output Devices in order to maximise the
opportunity to use video and audio clips to stimulate student
interest. This topic has no particular hierarchy or sequence of
concepts and thus lends itself to a hypertext-based exploration
of the learning space. Following selection of the topic, the concept
space was mapped out and classified in terms of its potential
use of text, graphics, animation, still images, video and audio.
As the learning materials were delivered on IBM PC 486 computers
with 8MB RAM the system was developed on a similar computer with
1.2GB of disk space. Video was shot using a Video 7 camera and
captured using a Video Blaster card and Microsoft Video. The actual
learning material was compiled using Microsoft Viewer. The final
set of materials occupied 70MB of disk space, this included 20
short video clips (one quarter screen size) and 30 bitmap images.
Production of self-paced learning materials is labour-intensive
and hence expensive. Only very large audiences would enable cost
savings for sophisticated multimedia productions. In this experiment
we set out to generate adequate as against 'state of the art'
materials because of resource limitations. The reality at UWS
is that no-one will produce superb, commercial level productions
in any cost effective manner. Thus, one aspect of the study was
to see if students would accept, and be able to learn from the
standard of production which could be achieved with a modest effort
in a relatively short time frame. The module in the present study
required approximately 50 hours of preparation time using resources
costing less than $2000. Figure 1 shows an illustration of a typical
Figure 1: A typical screen from the learning materials,
showing a video window, text, control buttons and hot words (hyperlinks).
Students were randomly selected and asked if they would participate in the study. Those who agreed became part of the test group, the rest constituted the control group. We used random selection instead of simply calling for volunteers, in order to avoid biases resulting from selecting students already favourably predisposed to computer-based work. Students were not compelled however, so some bias conceivably may still exist. The test group studied the topic in the computer laboratory using the multimedia system, whilst the control group covered the same material in 'chalk and talk' fashion in the lecture room. All groups were then tested on the topic using a standard written test. In addition the multimedia group was given a questionnaire to measure more qualitative aspects. The test group comprised 25 students whilst the control group numbered approximately 185.
The students studied 6 modules, the sixth of which
was delivered via the multimedia package. At the end of each module
students took a written test. If they scored poorly, they were
allowed to attempt another test later.
with 2nd attempts
Table 1: The percentage of students that required a second attempt to pass each of the six modules. The multimedia module was module 6.
|% students failing
Table 2: The percentage of students failing each of the six modules.
The multimedia module was module 6.
The results summarised above in Tables 1 and 2 confirm studies
carried out elsewhere which showed no statistically significant
difference in quantitative outcomes between groups using the multimedia
materials and those which did not. In terms of flexible delivery,
however, this is encouraging, as the self-paced multimedia appears
to be just as effective as more intensive teacher-centred classes.
Qualitative testing revealed generally positive feedback as shown
'quicker and easier to use than a textbook'
'use at your own pace'
'good for those who miss lectures'.
3.2.2 Comparison with Traditional Methods
3.2.3 Interface Issues
3.2.4 Suggested Improvements
Students suggested the following improvements to the package:
Clearly the students were very assessment-oriented in their consideration of the improvements!
Quantitative and qualitative feedback obtained from the experiment was encouraging. Students appear to find the delivery medium a useful one for learning and staff can see obvious advantages for increased flexibility in revision and for students whose study gets out of sequence. It seems unlikely that students will experience better learning via a computer-mediated package than with a good human teacher. However, quantitative results suggest that it is not significantly impaired. This coupled with obvious advantages in flexibility for students working outside scheduled class hours encourages us to explore its further use. The limited budget did not appear to pose overwhelming problems, only 16% of students cited the quality of the materials (images/diagrams) as a problem.
Boyle, T, Gray, J, Wendl, B and Davies, M (1994). Taking the plunge
with CLEM: The design and evaluation of a large-scale CAL system.
In Kibby, M R and Hartley, J R (Eds.), CAL into
the Mainstream, Computers and Education, Vol. 22, No. 1 / 2, Pergamon,
Kulik, C C and Kulik, J A (1991). Effectiveness
of computer-based instruction: An updated analysis, Computers
in Human Behaviour, Vol. 7.
Langham and Johnson (1984). In Barker, J and Tucker, R (Eds.) (1992). The Interactive Multimedia Revolution, Kogan Page, London.
The authors wish to thank the Faculty of Business & Technology for providing the grant in support of this experiment and the Computer Centre for providing facilities for computer-based instruction.
Mark Johnston, Graeme Salter, Kevin Donegan, Paul Davies ©
1996. The authors assign to ASCILITE and educational and non-profit
institutions a non-exclusive licence to use this document for
personal use and in courses of instruction provided that the article
is used in full and this copyright statement is reproduced. The
authors also grant a non-exclusive licence to ASCILITE to publish
this document in full on the World Wide Web and on CD-ROM and
in printed form with the ASCILITE 96 conference papers, and for
documents to be published on mirrors on the World Wide Web. Any
other usage is prohibited without the express permission of the