ASCILITE 2004: Clayton - investigating online learning environments

[ ASCILITE ] [ 2004 Proceedings Contents ]

Investigating online learning environments

John Clayton
Centre for Learning Technologies
Waikato Institute of Technology

Increasingly the perceived benefits of information stored in a digital format are being exploited. The sophistication and ease of supporting web browsers, the creation of internet search engines and the advancing computer skills of students' mean, educational institutions at all levels are using the Internet and Intranets to supplement classroom instruction, to give learners the ability to connect to information (instructional and other resources) and to deliver learning experiences. This paper will focus on the concepts and procedures used in the development of an online learning environment perceptual measure, the Online Learning Environment Survey (OLLES). It is envisaged this instrument will inform educationalists of the effectiveness of tactics and strategies they are employing in e-environments.

Online learning

By reviewing the term 'online learning' we could provide a simple definition, 'the use by learners and tutors of connected (online) computers to participate in educational activities (learning)'. While this definition is technically correct, it fails to explain the full range and use of connected computers in the classroom. Historically the term appears to have evolved as new technological tools have been developed. For example Zhu & McKnight (2001), described online instruction as any formal educational process in which the instruction occurs when the learner and the instructor are not in the same place and technology is used to provide a communication link between the instructor and students. Chang & Fisher (2001) described a web based learning environment as consisting of digitally formatted content resources and communication devices to allow interaction. Chin & Ng Kon (2003), used the term 'e-learning' to identify eight dimensions (institutional, pedagogical, technological, interface design, evaluation, management, resource support, and ethical) that constructed an e-learning framework.

However, the range of definitions of 'online learning' is not only a reflection of technological advances, it is also a reflection of the variety of ways educationalists, at all levels, use connected computers in learning. For example, in one situation a group of 14 year old students, following a pre-prepared unit in a supervised computer laboratory, may use the information storage capacity of the WWW to gather additional resources in preparing a presentation on the Antarctica. A second group of 16 year olds, studying the same topic in a classroom with a dedicated computer work station situated by the teacher's desk, could use the communicative functions of the Internet to establish mail lists with Antarctic staff to follow studies being undertaken on weather patterns. A third group of 12 year olds, consisting of small pockets of learners in isolated locations using home based connected workstations, may use an educational courseware package, incorporating information storage and communicative functions, to participate in a complete distance unit studying animal life in the Antarctic. Each of the groups described have used connected computers in different ways to achieve different objectives. The technical competencies required, the learning support needed and the physical location of the students in each case appears to be different and distinct. Initially it appears to be impossible to investigate each scenario using a common instrument, there does not appear to be any 'commonality'. On closer examination we find this is not the case.

Interactions in online learning

In each of the scenarios described in the previous section there is an assumption students have a functional knowledge of computer operations. There is the assumption that students will, be able to know if the computer is turned on or turned off, be able to use a keyboard and computer mouse, be able to view information presented on a visual display unit, be able to select and/or use appropriate software applications. A student - computer relationship, common to all scenarios, can be identified and therefore investigated. This can be further expanded by focusing on our understanding of the process of learning and the relationships created in this process. (Haynes, 2002) has outlined four relationships associated with online learning. These are student - interface interaction, student - tutor relationships, student - student relationships, and student - content relationships. The importance of creating time for and encouraging self reflection of the learning process is well documented (Taylor & Maor, 2000). It would appear to be crucial to investigate if, when and how reflective activity takes place. Therefore, for the purpose of this paper, there appears to be 5 broad categories of online learning activity that can be identified, described and therefore investigated. These are outlined below;

Student - Interface Interaction (What are the features of the interface created that enhance / inhibit student learning and navigation?)
Student - Student Relationships (How, why and when students communicate with each other and what is the nature of this communication?)
Student - Tutor Relationships (How, why and when students communicate with their tutor and what is the nature of this communication?)
Student - Media Interaction (How is the student engaged with digitally stored information and how do they relate to the information presented?)
Student Reflection Activities. (How are students encouraged to reflect on their learning, are they satisfied with the environment and how do they relate to the environment created?)

Learning environments

The investigation in and of learning environments has its roots nourished by the Lewinian formula, B=f(P,E). This formula identifies that behaviour (B) is considered to be a function of (f) the person (P) and the environment (E). It recognises that both the environment and it's interaction with personal characteristics of the individual are potent determinants of human behaviour (Fraser, 1998). Early learning environment surveys and inventories exploring the broad picture of learning environment activities and relationships have been expanded. For example cconstructivist views of learning, the shift from teacher centered instruction to learner centered construction of knowledge, have influenced the development of a number of instruments (Taylor & Maor, 2000; Walker, 2003). Instruments have been developed to investigate computer simulations (Maor, 1999) and computer supported learning environments (Newhouse, 2001). Through ongoing research, instruments developed have been proved to be flexible (Ommundsen, 2001), reliable and cost effective (Fraser & Wubbels, 1995). The above description of learning environment research demonstrates the feasibility of developing perceptual measures capable of successfully analysing the range of learning environments created when using connected computers and the World Wide Web in teaching and learning.

Developing the online learning environment survey (OLLES)

A number of instruments have been developed to explore the use of computers in education (Clayton, 2003) and the interactions that occur in computer mediated environments (Chang & Fisher, 2001; Newhouse, 2001). Using these previous studies as a guide an initial web based instrument consisting of 8 scales and 81 items was developed. This survey was reviewed by current online tutors and learning environment researchers. This review led to the refinement of the instrument and while the 8 scales remained, the number of items was reduced to 61. After this review the URL location was distributed to a number of online tutors, known by the author, within New Zealand. These scales, and examples of associated items, are described in Table 1.

Table 1: OLLES scales and items

Scale Description Sample Item

Computer Competence Extent to which the student feels comfortable and enjoys using computers in the online environment. I have no problems using a range of computer technologies.

Material Environment Extent to which the computer hardware and software are adequate and user friendly. The instructions provided to use the tools within the site are clear and precise.

Student Collaboration Extent to which students work together, know, help, support and are friendly to each other. I communicate regularly with other students in this course.

Tutor Support The extent to which the tutor guides students in their learning and provides sensitive, ongoing and encouraging support. The feedback I receive from my tutor helps me identify the things I do not understand.

Active Learning The extent to which the computer activities support students in their learning and provide ongoing and relevant feedback. The feedback I receive from activities / quizzes is meaningful.

Order and Organisation Extent to which class activities are well organised and assist student comprehension. The learning objectives are clearly stated for each topic.

Information Design and Appeal Extent to which class materials are clear, stimulating and visually pleasing to the student. The material presented is visually appealing.

Reflective Thinking Extent to which reflective activities are encouraged and how students enjoyed learning and participating in this environment. I am satisfied with my experience of using the Internet and learning online.

Discussion of initial results

To ensure learning environment instruments are economical, valid and reliable draft versions are often reviewed and tested for reliability and consistency (Dorman et al., 1994). While learning environment researcher's can use a number of common statistical procedures, one of the most common statistical procedures used in the initial stages of instrument development is the Cronbach alpha reliability coefficient (Chang & Fisher, 2001). A review of the internal consistency of the instrument is shown in Table 2.

Table 2: OLLES scale reliability
n = 104

Scale	No Items	Cronbach's Alpha Coefficient
Computer Competence	8	0.86
Material Environment	7	0.79
Student Collaboration	8	0.84
Tutor Support	8	0.85
Active Learning	7	0.90
Order and Organisation	9	0.90
Information Design and Appeal	7	0.89
Reflective Thinking	7	0.88

The alpha for the scales, Order and Organisation and Active Learning (both above 0.9) could be considered to be excellent. The alpha for the scales Information Design and Appeal, Reflective Thinking, Tutor Support, Student Collaboration and Computer Competence (all above 0.8) could be considered to be good. The remaining scale, Material Environment (alpha above 0.7) could be considered acceptable. While high internal reliability does not necessarily mean there is an assurance of high quality, the results obtained are encouraging for further development. Further analysis of the scales and items will be undertaken and the refined version of the instrument will distributed to a wider audience.

Summary

This paper has described a new instrument which assesses student perceptions of the online learning environment. The study of the 61 item pilot instrument with 104 students indicates the instrument can be further developed with confidence. The review presented here is the first part of a more extensive study. An extensive analysis involving the further refinement of the instrument and further analysis of data collected will be reported in a doctorate thesis. It is hoped the availability of this instrument will allow researchers and developers to evaluate the use of online learning in educational settings. The author believes the development of a perceptual measure that explores the online learning environments would be a valued tool.

References

Chang, V. & Fisher, D. (2001, December 2-6). The validation and application of a new learning environment instrument to evaluate online learning in higher education. Paper presented at the Australian Association for Research in Education, Fremantle. http://www.aare.edu.au/01pap/cha01098.htm

Chin, K. L. & Ng Kon, P. (2003). Key factors for a fully online e-learning mode: a Delphi study. In G.Crisp, D.Thiele, I.Scholten, S.Barker & J.Baron (Eds.), Interact, Integrate, Impact: Proceedings of the 20th ASCILITE Conference. (pp.589-593): Adelaide, 7-10 December 2003. http://www.ascilite.org.au/conferences/adelaide03/docs/pdf/589.pdf

Clayton, J. (2003). Assessing and researching the online learning environment. In M. S. Khine & D. Fisher (Eds.), Technology-rich learning environments: A future perspective. (pp. 127-137). Singapore: World Scientific.

Dorman, J., Fraser, B. & McRobbie, C. J. (1994). Rhetoric and Reality: A study of classroom environments in catholic and government secondary schools. In D. Fisher (Ed.), The Study of Learning Environments (Vol. 8, pp. 124-141). Perth: Curtin University of Technology.

Fraser, B. (1998). Science learning environments: Assessment, effects and determinants. In B. Fraser & K. G. Tobin (Eds.), International handbook of science education (pp. 527-564). Dordrecht: Kluwer Academic Publishers.

Fraser, B. & Wubbels, T. (1995). Classroom learning environments. In B. Fraser & H. J. Walberg (Eds.), Improving science education. (pp. 117-143). Chicago: The University of Chicago Press.

Haynes, D. (2002). The social dimensions of online learning: Perceptions, theories and practical responses. Paper presented at the Distance Education Association of New Zealand, Wellington, 10-12 April.

Maor, D. (1999). Teacher and student reflections on interactions in an Internet based unit. In K. Martin, N. Stanley & N. Davison (Eds.), Teaching in the Disciplines/ Learning in Context, 257-261. Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999. Perth: UWA. http://lsn.curtin.edu.au/tlf/tlf1999/maor.html

Newhouse, P. (2001). Development and use of an instrument for computer-supported learning environments. Learning Environments Research: An International Journal, 2(2), 115-138.

Ommundsen, Y. (2001). Students' implicit theories of ability in physical education classes: The influence of motivational aspects of the learning environment. Learning Environments Research: An International Journal, 4(2), 139-158.

Taylor, P., & Maor, D. (2000). Assessing the efficacy of online teaching with the Constructivist On-Line Learning Environment Survey. In A. Herrmann & M. M. Kulski (Eds.), Flexible Futures in Tertiary Teaching. Proceedings of the 9th Annual Teaching Learning Forum, 2-4 February 2000. Perth: Curtin University of Technology. http://lsn.curtin.edu.au/tlf/tlf2000/taylor.html

Walker, S. (2003). Development and validation of an instrument for assessing distance education learning environments in higher education: The Distance Education Learning Environments Survey (DELES). Unpublished Doctor of Science Education Thesis, Curtin University of Technology, Perth.

Zhu, E., & McKnight, R. (2001). Principles of online design. Florida Gulf Coast University, Office of Instructional Technology [viewed 15 Mar 2003, verified 22 Nov 2004] http://www.fgcu.edu/onlinedesign/

Author: John Clayton can be contacted at ctjfc@wintec.ac.nz (Hamilton, New Zealand)

Please cite as: Clayton, J. (2004). Investigating online learning environments. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 197-200). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/clayton.html

© 2004 John Clayton
The author assigns to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the author.

[ ASCILITE ] [ 2004 Proceedings Contents ]
This URL: http://www.ascilite.org.au/conferences/perth04/procs/clayton.html
HTML created 22 Nov 2004. Last revision: 22 Nov 2004.

Scale	Description	Sample Item
Computer Competence	Extent to which the student feels comfortable and enjoys using computers in the online environment.	I have no problems using a range of computer technologies.
Material Environment	Extent to which the computer hardware and software are adequate and user friendly.	The instructions provided to use the tools within the site are clear and precise.
Student Collaboration	Extent to which students work together, know, help, support and are friendly to each other.	I communicate regularly with other students in this course.
Tutor Support	The extent to which the tutor guides students in their learning and provides sensitive, ongoing and encouraging support.	The feedback I receive from my tutor helps me identify the things I do not understand.
Active Learning	The extent to which the computer activities support students in their learning and provide ongoing and relevant feedback.	The feedback I receive from activities / quizzes is meaningful.
Order and Organisation	Extent to which class activities are well organised and assist student comprehension.	The learning objectives are clearly stated for each topic.
Information Design and Appeal	Extent to which class materials are clear, stimulating and visually pleasing to the student.	The material presented is visually appealing.
Reflective Thinking	Extent to which reflective activities are encouraged and how students enjoyed learning and participating in this environment.	I am satisfied with my experience of using the Internet and learning online.