Conference logo
[ ASCILITE ] [ 2004 Proceedings Contents ]

Modern methods for traditional tasks: Developing an electronic version of student fieldwork assessment at The University of Queensland

Wendy Chalmers and Paul Smith
Teaching and Educational Development Institute
The University of Queensland

Katie Ward
Division of Occupational Therapy
The University of Queensland
Fieldwork placements are an integral part of many professional tertiary programmes. At The University of Queensland, Occupational Therapy students undertake block fieldwork affiliations off campus at a wide range of sites as part of their studies. Students' fieldwork performance has traditionally been assessed using a hard copy format of the Student Placement Evaluation Form (SPEF), which is posted to the university on completion by the clinical supervisor.

This project aimed to develop an electronic version of the UQ Occupational Therapy Student Placement Evaluation Form (SPEF), to allow the assessment to be completed and returned in an on line format. Practitioners had become very comfortable with using the existing print based form so in order to encourage and assist users to extend beyond their comfort zones, numerous steps were taken to ease the learning process including incorporating the existing page layout, consistent colour coding, considerable user instruction, testing and software enhancement cycles.

Additionally, the e-version of the SPEF aimed to provide a range of benefits such as on screen assistance in the form of instructions, roll overs and feedback to supervisors, increased accuracy, faster completion, cost savings to the School, up to date design, improved security and confidential and anonymous storage of fieldwork results for potential future research.


Background

Occupational Therapy Students take part in fieldwork as part of their professional education to develop a range of skills considered essential to being a competent Occupational Therapy practitioner. Through fieldwork, students develop both practical knowledge and interpersonal skills (Hays cited in Simhoni, 2002), and professional skills and attributes.

Occupational Therapy students are required by the World Federation of Occupational Therapists to complete a minimum of 1,000 hours of supervised clinical practice before graduation (WFOT, 2002). This clinical practice may take place in a range of settings including hospitals, community centres, private practices and even businesses and industrial companies (James & Prigg, 2004).

Models of supervision

The practising Occupational Therapist plays a pivotal role as clinical educator/supervisor to OT students on placement (Swinehart & Meyers, 1992; Mackenzie et al, 2001a). A range of models of supervision may be utilised in different placements.

Students and supervisors may be in a one to one relationship (one OT to one student); shared or joint supervision (two OTs or more to one student) (Steele-Smith et al, 2001); collaborative supervision (one OT to two or more students) (Huddleston 1999a); or group supervision (multiple supervisors to multiple students) ( Fleming et al 1996, Farrow et al 2000). Depending on the placement, supervisors may also be at different locations (Fisher & Savin - Baden, 2002).

With the growth in Occupational Therapy programs across Australia in recent years, there is currently a significant crisis in obtaining sufficient fieldwork placements, and exploration of new and innovative models of fieldwork placements will be required in the future (Huddleston,1999b ; Thomas, 2003; James & Prigg, 2004).

The SPEF

The original Student Placement Evaluation Form (SPEF) was developed at The University of Queensland in the mid 1990s, by a joint working party of UQ academics, primarily from the Occupational Therapy Division, and OT practitioners, with the aim of producing a more comprehensive and objective fieldwork assessment tool, based on criterion referenced assessment (Allison & Turpin, 2004).

Since its introduction at UQ in 1998, the SPEF has now been licensed to eight of the ten Occupational Therapy schools in Australia for use in their fieldwork programmes (Ward, 2004).

The assessment process

The SPEF consists of eight learning objectives. The eight Objectives are - 1. Professional Practice; 2. Self Management Skills; 3. Communication Skills; 4. Documentation; 5. Assessment/ Information Gathering; 6. Intervention; 7. Evaluation; and 8. Group Skills (the latter being optional according to workplace).

Each learning objective lists a number of criteria for assessment. Students are graded on a 1 to 5 scale for each required criteria. All nominated core criteria must be passed and each learning objective has a set number of items required to be passed overall.

Students are formally assessed using the SPEF on two occasions during a placement - at the halfway point for feedback and planning, and then again at the final assessment in the last week of the placement.

Streams

The SPEF has three assessment streams that reflect the variety of Occupational Therapy practices and settings in which students may be placed - Direct Client Care, Case Management, and Project Management or Consultancy. A student may be in a placement reflecting one of these streams only, or a mix of streams. For example, a student may be engaged in direct client care as well as undertaking a project or some form of consultancy.

The SPEF contains a summary page for feedback to students at both halfway and final assessments, and comments sections for each Learning Objective in the assessment (Division of Occupational Therapy UQ, 1998).

Development of the e-SPEF

Occupational Therapists using the paper based SPEF have been generally comfortable with the format. However, some feedback had been received in relation to the setup of streaming codes and tallying of learning objective results, which had indicated some confusion amongst practitioners (particularly new users) when filling in the paper form.

The Teaching and Educational Development Institute (TEDI) foresaw that advantages of an e-version of the form could include providing practitioners with on screen assistance, having instructions appear when needed, simplifying the selection of streaming choices and improving completion time and accuracy through adding totals automatically. Given the variety of computer skills amongst clinical therapists, it was integral that the proposed e-version of the form employed user friendly and consistent features to encourage them to move outside their comfort zone. Working through an electronic medium, the school will also benefit from decreasing costs associated with printing and posting the existing printed version of the SPEF.

It was also important that the research potential of the SPEF be able to be utilised more fully, both at a campus and national level (as the SPEF is now in use in the majority of OT Schools in Australia). The brief to TEDI then included a need for capability of the SPEF results to be electronically collated, confidentially transferred and anonymously stored. This will provide benefits in reducing the time in entry data to analyse in future research, as opposed to labour intensive use of paper based records.

Format

The solution that TEDI proposed was to purchase and use 'Informed Designer' and 'Informed Filler' software (Shana Corp, 2002). Using this software, which is compatible with both MACs and PCs, the students' scores information from each class and year will be held in a 'scores' Access database and the data from these 'scores' databases (with the students' personal details removed) will be stored in a master Access database.

The solution has been designed so that it can be adopted by other disciplines that may identify a need for evaluating students in industry placement (for example education, medical, mining students etc.).

The design of the new e-SPEF was a major consideration in the project. The paper format of the SPEF was not altered significantly in its transition to an e-version with all assessment content (i.e. learning objectives and criteria) being unaltered. The actual instructions to the Clinical Supervisors were significantly remodelled to match the new e-format usage. Every effort was made to retain the original look of the SPEF to ease the transition for users.

Methods of use

Using the 'Informed' software, the print based form has been modified and converted into an 'Informed' template document that the supervisors can either download from the Internet or be sent via post on CD ROM.

Supervisors install the 'Informed Filler' software on their computers and then use this software to open and fill in the evaluation questions. The software gives the user feedback to ensure that the form is filled in correctly. When the form has been completed, the users can send their information to the facilitator as an email attachment - this is simply done by clicking on the 'send' button included in the form. The facilitator then checks the form and adds it to the central database.

Structure

The scores database was created to contain the halfway and final scores of a single year's worth of students. At a set time at the end of the intern program for a year, the database is archived and replaced with a new blank 'scores' database. TEDI has developed a method for importing the data into the SPSS for analysis. The base SPSS is able to interface with Access databases.

Keeping separate versions of the database means that, not only can the system be used with other organisations but additionally the processing and archiving of assessment results over a fieldwork cohort is much easier.

Master database

In order to view trends in student results over time, certain data from each year from all schools will also be held in a central master database. This would not include data which could be used to identify a particular student, to ensure privacy.

Security

Given that the results of the student placements have an impact on students' final results, security of the system was of paramount importance. This was one of the reasons the database which holds the student results was not exposed to the internet.

A form of digital signature was also used in the e-SPEF to ensure that the identity of the supervisor was authentic, and to prevent the form from being altered once it was completed.

Future plans for e-SPEF

The e-SPEF developmental trials aim to be completed by the end of 2004, with release of the finished product for UQ placements, planned in 2005. The feedback to date from clinicians involved in the pilot trials has been positive although the trials have been limited to date. More extensive trials are currently planned.

It is anticipated that the e-SPEF will yield positive gains for supervisors in terms of its time, efficiency and utility, particularly when multiple supervisors are involved. Time usage is a critical aspect in modern day practice (Whiteford & Wright- Sinclair,1997).

The portability of the e-version has particular promise for use in a geographically disperse placement base, including overseas. The electronic approach developed by TEDI to deliver the e-SPEF may also be of value to other disciplines in their fieldwork assessment processes. The authors however are not aware at this point as to the utilisation of electronic assessment and lodgement strategies similar to the e-SPEF in other disciplines.

Use of the same assessment tool is seen as valuable by clinical educators where they may host placements for students from several different Occupational Therapy programs (Mackenzie 2001b). Other licensed universities using the original hard copy version have expressed interest in obtaining the online version and this will be followed up.

It is envisaged that there will be significant scope in using the e-SPEF as a research tool regarding fieldwork across Australia, particularly as many other OT schools in Australia are also utilising it. The need to continue to further evaluate and understand the learning experience in the fieldwork process for Occupational Therapists may be seen as a priority for the profession. (Ferraro Coates & Crist, 2004).

The hard copy will remain in use for the foreseeable future for those therapists who prefer this medium or where access to computers/the web is not possible.

Occupational Therapy education in the 21st century will become increasingly linked to technology in all aspects of educational coursework (Royeen, 2001; Gallew, 2004). The development of a user friendly electronic version of the familiar and accepted hard copy Occupational Therapy Student Placement Evaluation Form is part of this educational transition.

References

Allison, H. & Turpin, M. (2004). Development of the student placement evaluation form: A tool for assessing student fieldwork performance. Australian Occupational Therapy Journal, 51(3), 125-132.

Farrow, S. Gaiptman, B. & Rudman, D. (2000). Exploration of a group model in fieldwork education. The Canadian Journal of Occupational Therapy, 67, 239-251.

Ferraro Coates, G. & Crist, P. (2004). Brief or new: Professional development of fieldwork students: Occupational adaptation, clinical reasoning and client-centredness. Occupational Therapy and Health Care, 18, 39-47.

Fisher, A. & Savin- Baden, M. (2002). Modernising fieldwork, Part 2: Realising the new agenda. British Journal of Occupational Therapy, 65, 275-282.

Fleming, J., Christensen, J., Franz, D. & Letourneau, L.(1996). A fieldwork model for non-traditional community practice. Occupational Therapy in Health Care, 10, 15-35.

Gallew, H. (2004). Brief or new: The benefits of on line learning in occupational therapy. Occupational Therapy and Health Care, 18, 117-125.

Huddleston, R. (1999). Clinical placements for the professions allied to medicine, Part 2: Placement shortages? Two models that can solve the problem. British Journal of Occupational Therapy, 62, 295-298.

James, C. & Prigg, A. (2004). A self directed fieldwork program to provide alternative occupational health placements for undergraduate occupational therapy students. Australian Occupational Therapy Journal, 51(2), 60-68.

Mackenzie, L., Zakrzewski, L., Walker, C. & Mc Cluskey, A. (2001). Meeting the educational needs of fieldwork supervisors: A collaborative workshop developed by New South Wales occupational therapy fieldwork coordinators. Australian Occupational Therapy Journal, 48(1), 1-10.

Thomas, Y. (2003). Traditional and Innovative Models of Fieldwork Supervision. Paper presented at OT Australia Qld Fieldwork Symposium, Brisbane. [verified 10 Oct 2004] http://www.otqld.org.au/fieldwork_symposium.htm#thomas

Royeen, C.(2001). Reflections on education and learning in occupational therapy for the 21st century. Occupational Therapy in Health Care, 15, 209-213.

Shana Informed Designer (2002). Version 4.02. Alberta, Canada.

Simhoni, O. & Andersen, L. (2002). Fieldwork: A road to employment. Occupational Therapy in Health Care, 16(1), 37-43.

Steele-Smith, S. & Armstrong, M. (2001). 'I would take more students but...': Student supervision strategies. British Journal of Occupational Therapy, 64(11), 549-551.

Swinehart, S. & Meyer, S. (1993). Level 1 fieldwork: Creating a positive experience. The American Journal of Occupational Therapy, 47, 68-73.

University of Queensland, Division of Occupational Therapy (1998). Student Placement Evaluation & Handbook. Brisbane: Author.

Ward, K. (2004). Unpublished Report from University of Queensland to ANZOTFA (Australian and New Zealand Fieldwork Academics) Annual Meeting.

WFOT (World Federation of Occupational Therapists) (2002). Revised Minimum Standards for the Education of Occupational Therapists. Stockholm.

Whiteford, G. and Wright-St Clair, V. (1997). Costing and purchasing clinical education: The New Zealand experience. Australian Occupational Therapy Journal, 44(1), 41-48.

Authors: Wendy Chalmers, BEd (Hons), Project Manager, Teaching and Educational Development Institute (TEDI)
Paul Smith, BMm, Software Designer, Teaching and Educational Development Institute (TEDI) The University of Queensland
Katie Ward, B.Occ.Thy, Grad.Dip.Teach. (Special Needs), Associate Lecturer (Fieldwork), Division of Occupational Therapy, School of Health & Rehabilitation Sciences

Please cite as: Chalmers, W., Smith, P. & Ward, K. (2004). Modern methods for traditional tasks: Developing an electronic version of student fieldwork assessment at The University of Queensland. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 170-174). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/chalmers.html

© 2004 Wendy Chalmers, Paul Smith & Katie Ward
The authors assign to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the authors.


[ ASCILITE ] [ 2004 Proceedings Contents ]
This URL: http://www.ascilite.org.au/conferences/perth04/procs/chalmers.html
HTML created 21 Nov 2004. Last revision: 21 Nov 2004.