Explanation and Guidance in a Learning Environment: Recording and Using Sam Multimedia Demos
Jan Les1, Geoff Cumming1, Neil Thomason2, Sue Finch1
1 - School of Psychological Science, La Trobe University
2 - Department of History and Philosophy of Science, University of Melbourne
Learners need guidance and explanation to make best use of a learning environment. Beyond the tutor and the printed worksheet, how can this be provided? In the context of StatPlay, our set of multimedia simulations and tools for statistics learners, we have developed Sam, a multimedia demonstration facility. Sam, as in 'Play it again Sam', allows the recording, storage and playback of any sequence of interactions with StatPlay, together with spoken commentary. Sam recordings, or 'demos', can be used to guide learner activities, provide teacher explanations, present examples, and pose problems. Students can prepare a demo to show their understanding. Brief multimedia demos with accompanying voice offer a particularly natural, appealing and effective way to make guidance and explanations available to learners. Sets of demos can be given a personality and a perspective, or might relate to a particular discipline-introductory statistics for biology rather than psychology, for example. We introduce StatPlay and describe Sam and initial trials in the classroom, then discuss the design of database and presentation facilities to make large sets of demos easily usable by students and teachers working with StatPlay. The techniques of Sam, which are the subject of a provisional patent application, can be applied to any software application to provide friendly multimedia guidance for users.
multimedia demos, explanation, dialogue, learning environment, statistics learning, recording
Supporting the Learner in the Learning Environment
Our starting point was the simple observation in the computer lab that it is natural and appealing to grab the mouse and make a short interaction with the learning environment while giving a spoken explanation. Students as well as teachers do this constantly. Would it be useful to give teacher and student a tool to capture such brief demos for later use by others? Our answer is Sam, as in 'Play it again...', which records and replays multimedia demos, with spoken commentary. Sam runs in the StatPlay learning environment for introductory statistics, but the Sam techniques could be used to build a demo facility in any application.
From a theoretical perspective, the problem is to provide guidance and explanation to a learner working in a learning environment. One approach to this problem is the Intelligent Tutoring System (ITS), which attempts to use artificial intelligence techniques to provide comments tailored for the individual learner. This approach raises problems of educational philosophy, and has not been very successful in practice. Even within the research field of Artificial Intelligence In Education (AIED), researchers are turning away from the ITS and are exploring a range of less onerous strategies to support the learner. One idea (Mayes & Neilson, 1996) is to make available to the learner a collection of excerpts from previous learner-teacher dialogues. Other proposed systems are based on stored expert answers to sets of questions (Ackerman & Malone, 1990; Graesser, Langston, & Lang, 1992).
Sam can store fragments of learner-teacher dialogues, or expert answers to questions, and so can be used to study these approaches to augmenting a learning environment. However, the demo is such a general concept that Sam can be used in many other ways as well. In this paper we first give a brief description of StatPlay and the current prototype Sam, then discuss some of the educational possibilities Sam offers. Key issues are the design of a database facility to organise for teacher and student a large number of Sam demos, and provision of branching and decision-making facilities to allow Sam demos to be linked flexibly into curriculum sequences.
StatPlay, a Learning Environment for Introductory Statistics
StatPlay comprises demonstrations and interactive simulations to help learners overcome some fundamental misconceptions and acquire correct understanding of key concepts in statistics. The cognitive science rationale for StatPlay was described by Thomason, Cumming and Zangari (1995). By analogy with naive physics, we use the term 'naive statistics' for everyday beliefs about probability and statistics. An important goal in statistics education is to overcome the misconceptions of naive statistics. There is a useful analogy with the work of White (1993) and others who have shown how learning activities based on computer simulations can help learners overcome misconceptions in naive science. Partly influenced by that research, we have designed StatPlay to present vivid representations of the target statistical conceptions. We use a variety of types of learning activities, from guided to exploratory, and including games.
An important idea is that understanding requires the ability to express the target concept in several different ways, to translate readily between these different formulations, and to see how they relate. A pervasive goal in designing StatPlay and learning activities is to help students work with multiple, dynamically linked representations in order to build such understanding. Further descriptions of StatPlay were given in Cumming, Thomason, Howard, Les and Zangari (1995), Cumming and Thomason (1995), and Cumming, Thomason and Les (1997).
The Windows 3.1 version of StatPlay
StatPlay is being developed in Visual C++ under Windows. The Windows 3.1 version of StatPlay has been used in increasing numbers of classrooms over four years and in 1997 was used by more than 3,000 students across more than 12 departments at The University of Melbourne. As a single example from the Windows 3.1 version, Figure 1 shows the screen display illustrating a surprising aspect of the Central Limit Theorem, which is readily discovered by exploration in the Sampling Playground, or microworld, of StatPlay. Even with a highly skewed distribution, here drawn freehand, and a sample size as small as n = 3, the sampling distribution of the mean shows surprisingly small skew: progression towards the Normal Distribution occurs surprisingly early as n increases.
The Windows 95 version of StatPlay
We want a clean interface and excellent usability, as crucial features required if StatPlay is to be enduring and successful. Recent interface thinking has reached beyond cognitive psychology and human factors to become more diverse, with influences from theatre, various social sciences and post-modernism. The goal is to make the interface disappear: the user should be thinking about the task and the target concepts, ideally with no awareness of the tools. In moving from Windows 3.1 to Windows 95 we decided to redesign our screens from scratch and avoid off-the-shelf Microsoft components as being too visually complex. For our work on StatPlay and Sam, useful design ideas have included:
Figure 1. The Sampling Playground of StatPlay. The upper panel shows a highly skewed population distribution, drawn freehand. The lower panel compares this shape with that of the sampling distribution of the mean, for samples of size n = 3. The sampling distribution is surprisingly close to the symmetric shape of the Normal Distribution.
These diverse and interesting writings do not give immediate prescriptions for the designer, but encourage us to think broadly about the whole statistical and learning purpose of StatPlay and Sam. They encourage us to talk to and observe our students with open minds, the better to build tools that help them work and learn effectively, and feel positive as they do so.
Figure 2 shows a prototype screen in the Distributions Playground of Windows 95 StatPlay. Figure 1 shows the familiar off-the-shelf Microsoft windows, buttons and other components. Figure 2, by contrast, shows the palettes, icons and other things that we have designed, following the principles mentioned above.
Sam, the Multimedia Demo Facility
Sam allows the user to record and store as a demo any sequence of interactions with StatPlay, with a synchronised spoken commentary (Les, Thomason, & Cumming, 1997). Menu selection allows playback of any Sam demo.
To record, Sam stores the initial state of the whole StatPlay application-including the current screen-then stores microphone input of speech in real time, and mouse and keyboard inputs. To replay, Sam restores the initial state of the application, plays back the speech, and submits the stored sequence of mouse and keyboard signals to StatPlay at the correct times. The user sees and hears the same sequence of StatPlay behaviour, with synchronised spoken commentary, as was earlier recorded. Demos can be nested.
The construction of Sam
Figure 2: Prototype screen from the Windows 95 version of StatPlay, currently in development, showing two distributions and the effect size between the two means.
The key idea of Sam is that, at playback, submission of a suitable stream of inputs to StatPlay can reproduce the exact sequence of interactions that occurred during recording. To achieve this, the state of StatPlay at the start of playback must be reset to match exactly the state at the start of recording. Sam must be able to store and reset the state of StatPlay, and therefore Sam is application-specific. The great advantage of Sam is that only small amounts of information need be stored, and so demo files are small, little larger than the compressed digitised speech.
By contrast, Lotus ScreenCam allows the recording of demos with any software application, but is based on recording sequences of whole screen images. It therefore gives extremely large files even when sophisticated compression techniques are used.
Sam is implemented in Visual C++ 5.0. The fact that the Windows interface is message driven allows Sam when recording to use Windows hook functions to intercept mouse and keyboard input before this passes to the Windows kernel and then the StatPlay application. A copy of the input is time-stamped and stored in the demo file. At playback, the recorded initial state of the application is set up-the user sees the screen as needed for the start of the demo-then mouse and keyboard are disabled and Sam passes stored mouse and keyboard information at the proper time to StatPlay via the kernel. Digitised speech is played back synchronously.
Educational possibilities for Sam
A wide range of educational possibilities suggest themselves, including:
Sam in educational action
We have used the Sam prototype with students in three psychology and environmental science statistics subjects. It has proved quick and easy for teachers to record demos of very acceptable quality. Students have worked in pairs or small groups, with printed worksheets to guide their activities, including suggestion of which Sam demos would be useful. Demos were presented via loudspeaker or multiple headphones. In every case there was an immediate positive response, with many teachers and students enthusiastic about the appeal, ease of use, and educational value of Sam demos. Demos were also seen as making it easy to become familiar with the interface and the range of features offered by StatPlay.
Almost the only questioning comment was from one or two tutors who noted that sometimes their students were less available for talk with the tutor because they were absorbed with the spoken message from StatPlay!
From Demo to Learning Sequence to Curriculum
Our Sam prototype supports only a simple menu list of demo titles, which is unwieldy beyond one or two dozen items. The next version should support many more demos, but we need to keep strict limits on complexity and development time. There are two central challenges:
A database of demos
Some demos serve as part of the Help system, and should be linked to the icons, statistical symbols or other display elements they explain. Other special purpose demos provide explanations of statistical terms, and are accessed via a glossary list.
Most demos are intended to contribute to statistics learning activities. Information can be stored automatically with each demo, including author, date of creation, StatPlay playground, and duration. The author can add descriptors, including demo title, subject context and topic, and a brief free text summary of the demo. A relational database is then the natural structure for demo storage, so that a user can scan and select a demo for replay via any of the types of descriptors, or via a text search of demo summaries.
When a user is working in StatPlay and calls up Sam, presentation of the demo database should be as an easily-traversed tree. The presentation should be sensitive to the most recently used demo, and to the current location in StatPlay. In other words it should be easy to select demos close to the one most recently used, and most relevant to the current context in StatPlay.
When scanning to select a demo for replay, the user should see the descriptive information and a small preview picture showing the screen at the start of the demo: in many cases this picture would give a fast and natural indication of demo content.
Beyond the single demo: The learning sequence
We are convinced that it is best to keep demos short, usually no more than a minute, and to have students spend most time carrying out their own activities. Therefore we need to be able to specify sequences of related demos. A teacher should, for example, be able to arrange for a student to play one demo, do some StatPlay work, play the next demo, maybe elect to step sideways to a demo giving further explanation, do some more StatPlay exploration, decide on the answer to a question, be directed to a demo appropriate to the answer given, and so on. We need to offer the teacher easy ways to set up links, and choice points so that a curriculum structure can be specified, drawing on a set of related demos.
There are many exciting possibilities, including the development of demos into agents with some limited ability to converse with the user. For now we need to select for construction a very limited number of facilities so that the Sam interface is simple and the software development task is manageable. Our current plan is to provide the creator of a demo with the following facilities:
A learner working through a sequence would at any time be able to see the current position in the sequence, to step back or forward, or to leave the sequence.
The Potential of the Multimedia Demo
The naturalness and appeal of our demos within StatPlay and the immediate positive response of users to Sam convinces us that the multimedia demo has enormous potential. The key technical issue in providing Sam facilities in other applications is the need to store the full state of the application at the start of a demo. There are software library facilities now becoming available that make it easy for an application to be designed so that it can 'store its own state'. It is therefore feasible to think of Sam being incorporated into a wide range of software. In most cases Sam demos could form the most user-friendly part of the Help system, and could also be arranged to give tutorial sequences to assist new users.
When the application has, like StatPlay, education as its primary purpose, demos can play an even more central role by offering a wide range of support to the learner. Many pedagogic and implementation issues need study before the full educational potential of the demo is realised.
StatPlay has been supported by the Committee for the Advancement of University Teaching, the Committee for University Teaching and Staff Development, the Australian Research Council, the Apple University Development Fund, and The University of Melbourne.
Ackerman, M. S., & Malone, T. W. (1990). Answer garden: A tool for growing organisational memory. In Proceedings of the ACM Conference on Office Information Systems, pp. 31-39.
Carroll, J. (1997). Human-computer interaction: Psychology as a science of design. International Journal of Human-Computer Studies, 46, 501-522.
Cumming, G., & Thomason, N. (1995). Learning environments for conceptual change: The case of statistics. In Greer, J. (Ed.) Artificial intelligence in education, 1995 (pp. 389-396). Proceedings of AIED95, the 7th World Conference on Artificial Intelligence in Education, Washington, August. Charlottesville, VA: AACE
Cumming, G., Thomason, N., Howard, A., Les, J., & Zangari, M. (1995). The StatPlay software for statistical understanding: Confidence intervals and hypothesis testing. In J. M. Pearce & A. Ellis (Eds.), Learning with technology: ASCILITE 95 Conference Proceedings (pp. 104-112). Parkville, Vic.: Science Multimedia Teaching Unit, The University of Melbourne.
Cumming, G., Thomason, N., & Les, J. (1997). Concepts and images: StatPlay and learning statistics. In C. McNaught (Ed.), Teaching with technology at La Trobe. Bundoora, Vic.: Academic Development Unit, La Trobe University.
Graesser, A. C., Langston, M. C., & Lang, K. L. (1992). Designing educational software around questioning. Journal of Artificial Intelligence in Education, 3, 235-241.
Landauer, T. (1995). The trouble with computers: Usefulness, usability, and productivity. Cambridge, MA: MIT Press.
Laurel, B. (1991). Computers as theatre. Reading, MA: Addison-Wesley.
Les, J., Thomason, N., & Cumming, G. (1997). Play it again SAM: StatPlay and a recording and playback facility to support learning. In B. du Boulay, & R. Mizoguchi (Eds.), Artificial intelligence in education: Knowledge and media in learning systems (pp. 466-473). Amsterdam: IOS Press.
Mayes, J. T., & Neilson, I. (1996). Learning from other people's dialogues: Questions about computer-based answers. In B. Collis & G. Davies (Eds.) Innovating learning with innovative technology. Amsterdam: North-Holland.
Oren, T., Salomon, G., Kreitman, K., & Don, A. (1990). Guides: Characterising the interface. In B. Laurel (Ed.), The art of human-computer interface design (pp. 367-381). Reading, MA: Addison-Wesley.
Thomason, N., Cumming, G., & Zangari, M. (1995). Understanding central concepts of statistics and experimental design in the social sciences. In K. Beattie, C. McNaught, & S. Wills (Eds.), Interactive multimedia in university education: Designing for change in teaching & learning. (pp. 59-81, and subject of commentary pp. 99-102) Amsterdam: North-Holland.
Tufte, E. (1983). The visual display of quantitative information. Cheshire, CT: Graphics.
Tufte, E. (1990). Envisioning information. Cheshire, CT: Graphics.
Tufte, E. (1997). Visual explanations. Cheshire, CT: Graphics.
Turkle, S. (1995). Life on the screen. New York: Simon and Schuster.
White, B. Y. (1993). ThinkerTools: Causal models conceptual change and science education. Cognition & Instruction, 10, 1-100.
(c) Jan Les, Geoff Cumming, Neil Thomason, Sue Finch
The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.
This page maintained by Rod Kevill. (Last updated: Friday, 21 November 1997)
NOTE: The page was created by an automated process from the emailed paper and may vary slightly in formatting and layout from the author's original.