Abstract

Whether due to amalgamation, funding cuts or export education strategies, many universities now face a problem of delivering subjects, programs and courses across more than one campus. There are significant costs to students, staff and faculty of this teaching and administrative duplication. Information and communication technologies offer potential to achieve economies of scale but the effects on other things like student learning are not so clear. The objective of this paper is to report the outcomes of a video conferencing trial in a mass lecture undergraduate context. The effects on staff, students and support issues highlighted through a formative and summative evaluation are reported. Surveys, focus groups, interviews and analysis of video record and reflective diaries were the data for the evaluations. These results will be useful for both decision makers considering a video conferencing solution to duplication as well as staff potentially involved in using the technology in a mass lecture context.

Keywords

Information technology, Communication technology, Business education, Video conferencing, Teleconferencing, Evaluation

1. Introduction

The modern university faces many of the same pressures as a business - namely, it must do more and better with less. Video conferencing is one potential solution academic managers are considering for reducing duplication. Laurillard's [5] observation that research on the effectiveness of information technology generally is severely lacking, applies particularly to video conferencing (Mitchell et al [8; p.73]). Frand and Ng [3; p.20] survey of learning technologies in US business education report a fair degree of users consider the technology ineffective. However, they conclude that it is too early to tell its real effectiveness. While research on video conferenced meetings, training (eg. Lundin and Donker [6]) and small group teaching (eg. Martin and Jamieson [7], Goddard [4], Treagust et al [11]) exist, research relating to large groups in a tertiary context in particular is extremely sparse if not negative (eg. Goddard [4; p. 211]).

The objective of this paper is to report the outcomes of a video conferencing trial involving mass lectures across a multi-campus university. Reporting these outcomes should aid decision makers considering a video conferencing solution to duplication. Potential users in a large group context can optimise student learning by considering the costs and benefits on staff and students highlighted in this paper.

The structure of this paper is as follows. Section 2 briefly elaborates the context leading up to the trial. The following section describes the mass lecture video conferencing trial. Section 4 discusses the results of the formative and summative evaluations. Effects on students, staff and academic managers are considered. Section 5 contains concluding remarks and suggests future directions.

2. Context

Problems of duplication in the tertiary teaching context arose in the late 1980's in Australia from government amalgamation policies. For others, duplication has become evident as universities seek to establish new campuses. In response to government funding cuts, universities have sought to expand into export education, increasing their revenue base with off-shore campuses. This scenario applies to the University of Technology Sydney (UTS), a university supporting in excess of 23,000 students. The trial itself took place in a business faculty struggling under considerable financial constraints. Reasons for the latter include the relatively lower funding per business student (as compared to a science or medical student) and the relatively higher cost of teaching part time students (as compared to full time students).

A further complication was the faculty requirement that subjects taught and assessed on both campuses were meant to be identical in all respects. Given the well-documented power of assessment to motivate student learning (eg. Ramsden [9]), it is no surprise that this requirement caused student disquiet. Students had previously expressed concerns about the equity of delivery between campuses.

Other problems included the need for staff to travel the 18 kilometres to deliver subjects offered at the other campus. Apart from the explicit travel cost, staff incurred lost time in travel plus the inconvenience of being without their resources. Students also travel to the other campus when there is insufficient interest to justify duplicate subject offering. The final trigger for the video conferencing trial came in 1993. An external faculty review recommended video conferencing to resolve duplication and other pressures arising from the two-campus situation.

A video conference trial between the two Sydney metropolitan campuses was initiated in the second half of 1995. The object of the trial was mass lectures in a compulsory subject of the undergraduate business degree. No attempts were made to video conference tutorials or student consultations. The purpose of the trial was to evaluate the extent to which video conferencing mass lectures would

• be cost-effective by reducing duplication between the two metropolitan, domestic campuses. Improvements would then be expected to flow to servicing off-shore campuses and increasing the capacity to provide export education.
• improve perceived equity concerns faced by students on different campuses enrolled in the same subject. Students had previously expressed a desire for equal access to a lecturer or relevant information for assessment.
• improve the quality of learning for students. This was expected to be achieved for a number of reasons. As staff effectively halve their teaching load (in hours), they would have more time to produce better teaching material. Second, the prospect of looking a fool seemed in a television-type context added further motivation to preparation of both teaching materials and teaching strategy. Finally, the extra features of the video conferencing equipment (eg. document sharing and close-up interactions) were expected to make it more interesting for students.

3. Video conference trial

To achieve these objectives, a project team was established in July 1995. The team comprised computing, instructional technology, external consultants and academic staff. There was a five week lead time before the first lecture for the semester was scheduled. During that time specifications were established, equipment purchased and installed, lecture theatres modified appropriately and cables installed. Staff also had a one day training session where a typical, small-group video conferencing session. Strategies for using the equipment in a mass lecture situation were brainstormed. Following the training session, efforts were made to discover previous research in the mass lecture context. While a literature search proved unsuccessful, phone discussions with several inter-state academics involved with small lecture situations provided valuable hints for adapting appropriate teaching strategies.

There were 250 students were enrolled at one campus and 80 students at the other. The video conferenced mass lecture took place on a Thursday night between 5.30 pm and 7 pm for 13 weeks. The preceding lecture hour was booked for equipment setup and testing. To circumvent concerns about potential inequities highlighted in the literature (eg. Treagust et al [11]), the 'live' or local campus each week was alternated between the two campuses. The remote campus received the video image of the lecturer and/or teaching materials.

Video conferencing is typically used for small group meetings (with two to six people at either end as shown in figure 1). As such, the standard equipment entails one image from the remote view and the other screen showing the live image being sent. A coder-decoder system, or 'codec', enables images and audio to be sent between conferencing parties. Generally speaking, the broader the bandwidth of the link, the more natural the conference session. Utilising a leased VTel 227M Quick Frame System and transmitting via a microwave link between the campuses, a very satisfactory image and audio reception was expected. While the link could transmit up to the maximum bandwidth of two megabytes per second, close to television quality (at 30 frames per second) was achieved with 1534 kilobytes per second. This overcame the standard problem of non-synchronisation of image and audio experienced with 128 kbps (or even 384 kbps).

A mass lecture context however, is quite different to a meeting. Even without video conferencing equipment, motivations for attending are different and interaction is more difficult. Preconceived perceptions are that lectures are more a one-way interaction of presenter enlightening or entertaining the audience. The content is typically much more structured than in a meeting of peers with a common goal to achieve. Video conferencing complicates the whole situation. Not only is it extremely difficult for remote students in a mass lecture theatre to be seen, without a portable microphone it is almost impossible to hear a student's question. A number of pre-emptive strategies were therefore undertaken to improve the learning context.

To improve student concentration in the mass lecture context, it was felt that there should be more opportunities to see the lecturer. To enable a larger audience to see, the image was also projected onto a wide screen using a data projector. Because of the large lecture theatre context, neither the standard camera position (on top of the video conferencing system) nor one at the rear of the theatre could be used. A camera operator was therefore employed to provide a range of shots for the remote end. Most were close-up images of the lecturer. During student-student interactions, the camera operator would focus on some students near him who were embroiled in the learning task.

Another problem to be overcome was that the codec could only allow one image to be sent at a time. The image of the lecturer could be sent or the image of the content points of the lecture. The solution was to use a laptop at the live end to link lecture slides through the University network to a second data projector at the remote end. This brought the number of devices under lecturer control to three, namely the lectern panel, laptop and video conferencing tablet. In case of breakdown, a backup lecturer was always available at the remote campus. This person also acted as facilitator by transporting a portable microphone during staff-student interactions. Guest lecturers also made presentations over the link. The specific teaching context of the mass lecture trial is depicted in Figure 2.

4. Effectiveness of video conferenced mass lectures

Given that mass lectures themselves are not in themselves conducive to student learning (Ramsden [1993]), the evaluation set out to gauge effectiveness of the whole teaching and learning environment rather than focus on the video conferencing equipment per se. A variety of qualitative and quantitative techniques were used in the formative and summative evaluations.

4.1 Formative evaluation

Formative feedback was solicited through a staff interview (in week 2), a student survey (n=239 in week 4), and three student focus groups (n=25 in week 7 and 8). Unsolicited feedback from peers and students during the semester was also used for feedback and corrective action. Technical and academic staff regularly reviewed the video record, session reports and reflective journals.

In line with a desire to focus on the big learning picture rather than just video conferencing per se, students were asked to rate various learning resources. Students rated nine resources on a five-point Likert scale ranging from 1 (= Quite useless) to 5 (= Really useful).

Few students (ie. 14%) perceived the video link as affecting their ability to learn in a negative way. Comparison with other resources indicates that the video link was neither their highest or lowest learning resource. Overall subject satisfaction rating of 7.2 (out of 10) compares well to 7.5 from previous years where no video link had been used and far higher than the co-requisite subject undertaken by the same students. Analysis of the free-hand responses to the best thing about the subject indicated some, though not surprising, effects of video conferencing. About 16% of respondents expressed a sense of excitement from being involved in an innovative technology. Less than a handful felt that the equipment was getting in their way of learning such that it became the aspect in most need of improving. Both results suggest a low profile for video conferencing. Since learning is the prime objective of the subject, this outcome was welcomed by academic staff.

Of the 25 students that participated in the three focus groups, students made far more positive comments (ie. 76 in total) than negative ones (ie. 59). These are summarised in Figure 3.

Students perceived various benefits including:

• greater equity in assessment and learning. Assessment is paramount in motivating students (Ramsden [9]). Many commented that unlike previous studies, no student group was advantaged because they got the 'better lecturer' or the 'lecturer who is running the subject and writing the exam'. The video link meant that students on both campuses could have the same lecturer at the same time.
• motivational aspects by being involved in an innovative multiple media approach to mass lectures. Care however should be taken in attributing this result to video conferencing per se. This subject had received such comments from students in previous semesters when data projection and audio visual technologies, but not video conferencing, had been used.

being exposed to an emerging business technology. Students saw this as beneficial for their future employment prospects much as PC skills were once a competitive advantage because they were not so commonly held.

• an increased sense of cross campus interaction. This was surprising given that there was little direct communication from students in one lecture theatre to the other. Examples included students volunteering answers via the portable microphone or the document camera, and chats with friends on the other campus before the lecture. It seems that simple interactions like seeing and waving to their cross campus colleagues during the lecture were seen as valuable. Competitions between campus groups during the lecture also fostered interest, concentration as well as this sense of cross campus interaction.
• better access to information flowing from a more effective presentation. Feedback from the focus groups noted significant learning benefits from the document camera. For example, students were now able to see three dimensional objects in use (eg. financial calculator). Once again, that technology can be, and has been, used as a stand-alone presentation aid.

Students also expressed various problems with the video conferencing. These included:

• a reduction in their learning time. Time for setup and setdown of the system ate into students' perception of their time to cover the material. While few of these technical difficulties involved complete system breakdowns, it simply took longer to do most things (eg. computer images ran slower). With three control devices, multiple actions were required to change from one type of teaching aid to another.
• Attempts to have staff-student interactions, a critical component for student learning, also took longer than normal. This was particularly the case when the students were at the remote campus since audio from the students attending there was limited by the speed of the facilitator to get the microphone into their hands. This is more of a problem with mass lecture video conferencing than video conferencing per se. The current state of audio technology is not able to adequately distinguish between genuine questions and students simply chatting in the large lecture theatre.
• increased potential for disruptive behaviour. Student chat at the remote end could not be heard by the lecturer. Unless the cameras were pointed to the right place in tight focus, it was difficult for the lecturer to notice and control disruptive students. Many remote students treated the lecture like a television session. Students were more likely to chat or walking in and out at the remote end. Students reported that these disruptions could dramatically affect their concentration and learning. The presence of a facilitator was not sufficient to reduce the noise even when attempts were made to do so.
• a perception that the money could be better spent. It was obvious by the investment in the equipment and the number of support staff involved that the experiment was costly. Suggestions included diverting the money to making tutorial class sizes smaller or longer.
• reduced access to the lecturer for students at the remote end. Many students prefer to discuss things personally. Following the end of a lecture is considered a good time to do so. Students at the remote end experienced reduced access in this trial for several reasons. The first reason related to the need to vacate for the incoming lecture. Inevitably, setdown of the equipment was a rush because, as explained above, learning time was at a premium and there was more equipment to clear. On top of that, students could not have their discussion on the way to the next lecture because the lecturer was not physically there. Confidential issues in particular were discouraged. Only a handful felt sufficiently strongly about having a face-to-face session that they alternated their attendance to coincide with that of the lecturer.
• reduced staff-student interactions. Mass lectures by definition discourage extensive interactions with individual students. Students already feel somewhat self-conscious interacting with staff during a lecture either because they felt they were wasting their colleagues' valuable listening time or because they might say the wrong answer. While various strategies work to encourage interaction in a normal mass lecture, even bright, confident students who normally give correct answers avoided these opportunities in the video conferencing sessions. The focus group highlighted that the self-consciousness was due to the camera spotlight aspect of the video conferencing. In addition, it was simply harder for students to initiate an interaction because they could not be as easily seen or in fact or heard.
• timetable related problems. Students were already sensitive to a reduction in face-to-face lecture time from the technical hitches and delays during interactions and so this idiosyncratic organisational problem caused a disproportionate amount of aggravation.

Survey (n=246)

Av (SD)

1. The video link means students on both campuses get equal treatment 4.0 (1.0)

1. The video images were clear and comprehensible when lecturer is remote 3.8 (0.9)
2. The audio was clear and comprehensible when the lecturer was 'remote' 3.8 (0.9)
3. I felt more self conscious in lectures because of the video link 2.8 (1.3)
4. I felt I couldn't interrupt the lecturer to ask questions when he's remote 3.2 (1.1)
5. I feel the students on the 'live' campus have an advantage 3.2 (1.2)

NB: Scale is 1=Disagree strongly 2=Disagree 3=Neutral 4=Agree 5=Agree strongly

Several strategies to minimise the effect of these unfavourable aspects of video conferencing were implemented. For example, students volunteering answers were not projected in tight focus. Because of reduced interruptability, more opportunities for asking questions were structured into the lecture action plan, including a conscious attempt to alternate them between campuses. The final evaluation below records the effectiveness of these strategies.

4.2 Summative evaluation

Summative feedback at the conclusion of the video conference trial included a student survey (n=246), three interviews with selected students and formal peer review by two colleagues. In addition, transcripts of focus groups and interviews were analysed, as were the reflective journals kept by academic and technical staff. The effects on students, staff and support issues are each discussed in turn below.

4.2.1 Student perceptions

The feedback from students attending the video conferenced mass lectures indicates a number of perceived costs and benefits.

Table 2 confirms the focus group finding that shows that equity in assessment and learning is critical. Only 9% of students disagreed with statement 1.1. The equity aspect was aided by the quality of audio and images (in statements 1.2 and 1.3). In each case only 10% of respondents disagreed or strongly disagreed. Statements 1.4 and 1.5 indicate that substantial progress had been made on several fronts by the end of semester. Only 28% of students surveyed still felt too self-conscious because of the video link. 39% still felt it was more difficult to interrupt when the lecturer was live at the other campus. However despite preventative efforts, nearly half (ie. 48%) felt that remote students were still disadvantaged (in statement 1.6). This confirms previous research by Treagust [11]. The only consolation to this fact is that the disadvantage was equitably shared since the remote end was alternated each week.

Free hand responses yielded similar results to the previous survey. Some 9% (or 10 from 115 responses) attributed the video link as the best thing about the subject and not one student indicated that the video link was the aspect needing most improvement. Some 44% (or 39 out of 88 responses) attributed the equity of access to expert and material as the best thing about the video link. Ranking on other positive aspects mirrored the focus group responses in Figure 3. The aspect of the video link most needing improvement was better remote images (24%) followed by better remote audio (20%). While some 17% indicated that it could not be improved, the remaining reasons included disruptive remote students, intrusiveness of camera and lost learning time.

4.2.2 Staff perceptions

There are a number of benefits to staff involved in video conferenced teaching. These include:

• reduced duplication teaching time, travel time and inconvenience of being away from their resources. Reduced duplication means staff can be more productive in their preparation of their learning materials and research.
• incentives to be better prepared to meet students learning needs. Thinking about the needs of students at the remote campus raised a number of issues that needed to be addressed. For example, how were remote students going to see what was happening? How could we read the non-verbal clues from student? How could interaction activities be achieved? How could the document camera be best utilised? What materials and plan needed to be in place if the link failed? Would students expect it to be more professional because it was like television?
• access to powerful teaching aids (eg. document camera), which has been noted elsewhere (eg. Martin and Jamieson [7]).
• access to a powerful crowd-control aid. Focussing the camera tightly on a disruptive student is a powerful way of disciplining them without having to say one word, thus improving the learning environment.
• access to a video record of one's teaching which is useful to remedy deficiencies.
• access to relevant expert guests who otherwise could not get to the lecture location.

The disadvantages to staff of video conferencing mass lectures included:

• a greater need to prepare materials and plan for effectively using them. At least in the first instance this is quite time consuming.
• a restriction on their lecturing style. Staff must think through the ramifications of what remote students are seeing. Things like on what one focuses (ie. at the camera not the monitor), how much one moves or interacts with (more self-conscious) students or even the clothes one wears become important. These things have already been noted in the literature (eg. Schiller and Mitchell [10]). Staff must therefore prepare for and practise with the technology, adapting their teaching style to optimise student learning.
• a greater reliance on other people to make a lecture 'work'. Unlike an overhead projector that is simple to use, video conferencing equipment currently can not work in a mass lecture context without computing and/or instructional technology support people. A greater risk of error is likely because academic and technical staff have more devices and links to maintain. If the equipment fails in a mass lecture situation, more students are affected. Time wasted is critical and rescheduling is impossible. Students expressing dissatisfaction can dent one's confidence. Technical support services must be both responsive and efficient or staff will not risk using the equipment. Improvements must be undertaken so the equipment complexities become transparent.
• additional distractions to promoting a positive learning environment. Crowd control of a normal mass lecture is hard at the best of times. Combining this in cross campus situation with a restricted sense of what is happening at the noisier remote campus complicates one's delivery of learning material. A further complication is the need to remember which buttons to push (on laptop, lectern panel and video conferencing tablet) to achieve different things. The technical aspects need to be made 'idiot proof'. Not only are these distracting, but they take time to manage. In particular, focussing the camera is time-consuming. Fortunately there are systems being purpose-developed for mass lectures that are far more transparent to use for lecturers.
• a massive increase in stress caused by the above. Clearly some of this would abate with more experience.

4.2.3 Support issues

There is a range of effects academic support managers need to consider in regard to video conferenced mass lectures. There are potential gains in:

• productivity by a reduction in duplication, inconvenience and travel expenses.
• access to a greater potential pool of students (eg. distance education).
• student satisfaction arising from better-prepared lecturers and equity in access.
• economies of scale in using the video conferencing equipment for other purposes, such as small group research or administration meetings, many of which the technology is better suited.

The costs of video conferencing mass lectures for academic support managers include:

• substantial up-front resource implications (eg. buying/leasing the equipment and adequate cabling, upgrading rooms, providing adequate security against theft or sabotage).
• substantial and increasing operating costs. Support services must be efficient and responsive to users needs or the equipment will simply gather dust.
• supporting equipment that is constantly being superseded.
• mechanisms to deal with vocal student dissatisfaction when things go wrong.
• a need to incorporate the side effects in strategic planning (eg. setup and setdown, training, timetabling).
• greater accountability on a daily basis from users as usage increases.

The extra costs of video conferencing these mass lectures amounted conservatively to $53 per minute! With overseas transmission currently several multiples of that figure, exporting video conferenced mass lectures also appears not cost-effective.

5. Concluding remarks

This paper reports the outcomes of a video conferencing trial involving mass lectures across a multi-campus university. Prior expectations were that mass lecture video conferencing would more cost-effective and improve equity in access concerns expressed by students previously. It was also expected that the quality of learning for students would improve because staff would have incentives to be better prepared and the technology would enable better presentations and interactions.

Only one of these expectations was realised to any extent. The primary benefit students perceived was the equal treatment and access to experts and information they could receive because the mass lecture was video conferenced. While the document camera in particular was a valuable teaching aid, it can be used independently of the video conferencing equipment. Student satisfaction with lectures and the subject were unchanged on previous years.

Echoing Goddard [4; p. 209], students and staff felt the lecturing, learning activities and interactions were not improved but went more slowly. Other disadvantages were the time lost through technical difficulties and the greater likelihood for distractions at the remote campus. Students at the remote campus felt disadvantaged despite preventative strategies implemented.

The equipment could become more cost-effective than experienced in this trial if it were integrated into the network system so video conferencing could be transmitted to many rooms adequately fitted. However, its usage in a mass lecture context is less attractive given the additional pressures and stresses on academic staff in delivering material, and given what we know about how students learn. A cheaper option, given the constraints on interaction in a mass lecture context anyway, may be to simply video-tape the lecture. The tape could then be replayed at multiple times and other locations and the video conferencing equipment could be used effectively then for questions at the end. The tasks for which the medium was really first designed, namely administrative meetings, small group teaching and research seminars, appear to offer greater and significant productivity improvements. Hence a permanent video conferencing system appears inevitable when these uses can be included in the cost-benefit equation.

In terms of future directions for video conferenced mass lectures, further research is needed to clarify when and how it can work better. The messages from this experiment relate to technical, staff and organisational factors. They comprise:

1. Ensuring any system is simplistic to use and comes with upgrade and maintenance guarantees.

2. Using video conferencing in a mass lecture context only if lecturers have some motive for participating, are confident with their content, competent in crowd control, are adequately trained and are supported by reliable and efficient technical services.

3. Resolving any organisational encumbrances to its success (eg. timetabling, equipment booking) and initiating an awareness and incentives program to maximise the chances of success.

4. Implementing an ongoing process of evaluation.

Acknowledgements

The author gratefully acknowledges the contribution of Reg Collins, Jo McKenzie and Graham Partington. All errors remain the responsibility of the author.

References

[1] DAUNT, C., YOUNG, J., [1994] Videoconferencing users' manual - for Lecturers,

Teachers & Trainers, Central Queensland University, Rockhampton

[2] DOYLE, K., CROCKER, M.A., BERNASCONI, R.J., ROBERTSON, S.C., [1993] "Three University Study of Student Costs Using Activity Based Costing Methodology", Evaluations & Investigations Program undertaken by Ernst & Young and funded by the Department of Employment Education and Training, June

[3] FRAND, J.L., NG, H.A., [1994] Eleventh Annual UCLA Business School Survey: Use of Learning Technologies in Business Education, University of California

[4] GODDARD, J., [1995] "Perspectives on video conferencing", ASCILITE'95 - Learning with Technology, Science Multimedia Teaching Unit, Melbourne University, pp. 205-213

[5] LAURILLARD, D., [1993] Rethinking University Teaching: A Framework for Effective Use of Educational Technology, London: Routledge

[6] LUNDIN, R., DONKER, A. [1992] Report of the Evaluation of the Queensland Government Trial of Compressed Videoconferencing between Brisbane and Townsville, Queensland Government Media and Information Service

[7] MARTIN, E., JAMIESON, P., [1995] Teaching cross-campus with video conferencing: how researchers and teachers make sense of this mew electronic classroom, HERDSA'95 - Higher Education: Blending tradition and technology, pp. 508-513

[8] MITCHELL, J., ATKINSON R., BATES, A., KENWORTHY, B., KING,B., KNIGHT,A., KRZEMIONKA, Z., LATCHEM, C., SCHILLER, J. [1993] Video conferencing in Higher Education in Australia, Department of Employment, Education and Training, AGPS

[9] RAMSDEN, P. [1993] Learning to Teach in Higher Education, Routledge

[10] SCHILLER, J., MITCHELL, J. [1993] "Interacting at a distance: Staff and student perceptions of teaching and learning via video conferencing", Australian Journal of Educational Technology, 9(1), pp. 41-58

[11] TREAGUST, D.F., WALDRIP, B.G., HORLEY, J.R., [1993] "Effectiveness of ISDN video-conferencing: a case study of two campuses and to different courses", Distance Education, 14(2), pp.315-330