Descriptive Account

Using Online Microassessments to Drive Student Learning

Alasdair G Thin

School of Life Sciences, Heriot-Watt University, Edinburgh, EH14 4AS, UK

Date received: 31/03/2006      Date accepted: 04/05/2006

It is not what is taught that has the most influence on students’ study behaviour, but rather what is assessed. Computer-assisted assessment offers the possibility of widening the scope of the material that is assessed, without placing excessive burdens on either staff or students. This article describes a computer-assisted assessment scheme comprising frequent, short, focused online assessments — termed “Microassessment”. It was designed and implemented with the primary aim of increasing the time students spent engaging with the learning activities of a module which was the second in a series of three introductory human anatomy and physiology modules. The impact of the introduction of the microassessments was evaluated by comparing results in the written end of module exam with the exam performance in the other two modules, where the continuous assessment tasks involved submitting completed workbooks at the end of each of the modules. The introduction of the microassessments resulted in a demonstrable improvement in the end of module exam performance and provides evidence in support of this specific pattern of continuous assessment as having a significant impact on student learning.

Introduction

Educational research has found that traditional teaching methods, based around lectures to large numbers of students do not promote active engagement in the learning process, such that students are passively learning at a relatively superficial level (Horgan, 1999). Early work by Snyder revealed that what influenced student learning most was not teaching, but rather assessment (Snyder, 1971). As access to HE has expanded, resource constraints have generally led to a reduction in module coursework and the associated feedback to students (Gibbs and Simpson, 2004). The solution, according to Gibbs and Simpson, is to design assessment regimes that engage students with the learning tasks without at the same time generating large amounts of marking.

Just as the emergence of the World Wide Web heralded new models of commerce, it is now promising new ways of teaching and learning. Indeed there is an “almost generalised belief” that communication and information technology (C&IT) developments can make HE more “accessible, affordable and effective” (Maier et al, 1998). However, simply making material available online does not guarantee student engagement. Faced with the changing realities of undergraduate life, students have become increasingly strategic with their use of time due to a number of competing demands such as part-time work (MacFarlane, 1992). The perceived relevance of online material is an important factor in student engagement (Overfield and Bryan-Lluka, 2003). In a study of the effectiveness of a variety of computer-based learning packages, it was found that a self-assessment quiz was little used in the first 14 weeks of teaching and just over half the class used it in the three weeks leading up to the end of module exam (Gunn and Pitt, 2003).

According to the concept of “Time on Task” (Chickering and Gamson, 1987), there is a direct positive correlation between the time spent studying and student performance in assessments. This article describes the introduction of frequent, short, focussed online assessments, with the intention that such “Microassessments” should lead to students spending more engaging with the learning activities in a module. In order to ensure that the students engaged with the assessments each week, these were counted towards their continuous assessment mark for the module.

The primary impetus for the introduction of the microassessments was poor end of module exam performance and a consequent high failure rate (26%) in the preceding year. It is a common perception among academic staff within the author’s institution that students do not spend enough time studying as they proceed through the module but rather leave it all to the week prior to the exam. This view is supported by both student responses to questionnaires and informal feedback elicited by the author. It is likely that this lack of engagement is related to the current institutional term structure which consists of eight weeks of scheduled of classes, followed by a study week prior to the exam week. On reflection, two further issues were apparent. First, the students manifest a general lack of preparedness for laboratory practical sessions, where the first 10-15 minutes of each session were wasted as students attempted to work out what they were supposed to be doing. Second, there was a general lack of effort regarding tutorial questions, with few students coming prepared to discuss the questions.

A way to increase the students’ engagement with the scheduled learning activities was required, in the hope that this would translate into improved end of module exam performance. However, it would have been impractical to take the laboratory practical workbooks in each week to mark.  Similarly, whilst answers to tutorial questions could be handed in, they could not realistically be marked and returned in an appropriately short space of time. It was therefore clear that an innovative approach was needed to increase the students’ amount of “time on task”.

Three forms of microassessments were devised, each with a different aim relating to the topic of study in a given week. Specifically, these were to improve preparedness for the laboratory practical sessions, to better consolidate the learning activities contained therein, and to increase engagement with the tutorial questions. The impact of the microassessments was evaluated by comparing the end of module exam performance after their introduction with scores from the previous year.

Methods

The microassessments were introduced into the module “A32HH2 Systemic Human Anatomy and Physiology” which the author leads and which forms part of a series of three introductory human anatomy and physiology modules in the second year of a four year undergraduate sport and exercise science degree programme (Table 1). The module involves study of the form and function of the various organ systems within the body. The module content is structured in the form of eight weekly topics, each with a problem case to provide a focus. At the beginning of each week there are two lectures, then a laboratory practical session on the Thursday followed by a tutorial session on the Friday.

Table 1 The series of three second year undergraduate human anatomy and physiology modules

The aim of the “Pre-laboratory” assessment (17% weighting) was to try and ensure that students prepared more thoroughly before arriving at laboratory practical class. It consisted of questions relating to the content of the laboratory practical session and should have been straightforward to complete if the details of the practical session were read before taking the assessment. It was available for the 48 hours preceding the practical session.

The aim of the “Post-laboratory” assessment (28% weighting) was two-fold: to consolidate both the information from the practical and the problem case for the week. It therefore had two parts; the first was a labelling task relating to activities in the practical session and the second consisted of a number of questions relating to the practical session and/or the problem case for the week. The assessment was available for 36 hours after the practical class. For each assessment, the general subject area of the first part was described in the module workbook.

The aim of the “Tutorial” assessment (55% weighting) was to ensure that the students made a reasonable attempt to answer the tutorial questions. It comprised of between 8 and 15 multiple choice questions adapted from existing paper-based assessments and concerned all aspects of the week’s topic. This task was available for 2 days prior and 8 days after the practical session. Students were given up to 3 attempts at the exercise. Previous experience had shown that when more than one attempt at an assessment was available, a minority of students had engaged in a process of trial and error in order to determine the correct answers to some questions. Feedback was therefore given as a score band (less than 40%, between 40 and 70%, greater than 70%) rather than the precise percentage mark, with the intention that the students received some indication of their progress whilst at the same time making both trial and error approaches, and collusion, more difficult. The students were, however, free to discuss the questions in the tutorial classes.

The online assessments were developed and delivered using the Perceptionä online assessment system (Questionmark, London, UK) which provides a web browser based interface to participants generated by a proprietary webserver linked to a database. The students had already gained experience of using the system in a previous module. The package allows assessments to be designed and configured by the lecturer and provides a number of comprehensive report formats. The scheduling of the assessments was designed with periods of overlap, such that at any one point in time, students could complete at least two microassessments.

The original assessment regimes for the series of three anatomy and physiology modules were similar; each had an end of module exam that contributed 60-70 % of the total mark, and continuous assessment marks derived equally from the submission of a completed workbook at the end of the module and from attendance. The introduction of the microassessments into A32HH2 replaced the workbook component and at the same time, the attendance component was reduced to 6% of the overall mark.

In order to assess the impact of the introduction of microassessments into A32HH2 on student performance in the end of module exam, performance within the cohort of students was compared across the series of three anatomy and physiology modules (Table 1). In addition, end of module exam performance was compared with that of the cohort of students from the year prior to the introduction of the microassessments. Analysis of variance was undertaken to compare performance in the three end of module exams both within and between cohorts. A p-value of less than 0.05 was accepted as indicating a statistically significant result.

Results

In the session 2002-3 (Year1), 27 students undertook the series of three anatomy and physiology modules. The microassessments were introduced into the second module (A32HH2) in session 2003-4 (Year2), this time with 34 students taking the three modules. The mean mark in the A32HH2 end of module exam increased significantly after the introduction of the microassessments into this module (Figure 1). Furthermore, the improvement in exam performance resulted in the exam failure rate (mark less than 30%) of 26% in Year1 falling to zero in Year2. In contrast, in the other two modules there was no difference in the mean performance in end of module exams between the cohorts (Figure 1).

Figure 1 Mean exam marks in the end of module exams of the three linked modules (Year1=2002-3, n=27; Year2=2003-4, n=34)

In order to gauge the attitude of the students to the microassessments, two free response questions were included on the anonymous module feedback forms. In response to what they liked most about the microassessments, replies included “Gave me a deadline within which to work”, “Regular study required to keep up” and “Helped me to see how well I was progressing”. Comments in response to what they liked least related to the time restrictions and short deadlines of the assessments and a request for more feedback.

Discussion

The online assessments were designed to increase student engagement with specific learning activities, whilst at the same time being short enough so as not to be overly burdensome, hence the term “microassessment”. The short deadlines were deliberately chosen so that students could not delay engaging with the learning activities. Due to security concerns with the server installation running the system, the assessments were constrained both in time and location, in that the students had to be at a computer on campus to take them. This may have been the reason behind some of the negative feedback comments. It is intended to remove this restriction when the security of the server installation system has been passed fit to be accessible from out with the campus network.

The introduction of the microassessments had the desired effect; not only did performance in the end of module exam improve significantly, but the failure rate of 26% seen in the preceding year was reduced to zero. This was unlikely to be due to differences in the two student cohorts, as the results in the end of module exams of the other two modules in the series were similar across the two groups.

The “Pre-laboratory” microassessments appeared to have a positive impact, in that the students came to the laboratory practical sessions better prepared, having been required to read the workbook beforehand. Furthermore, the students seemed more focused on completing their workbooks whilst in class as this would help them to prepare for the “Post-laboratory” assessment. In the tutorial sessions, the students freely discussed the tutorial questions with only limited direction required from academic staff.

Introduction of microassessments raises a number of potential issues. First, there may be concerns that the marks gained for this continuous assessment component may not be all due to each students’ own work. There was, in fact, no guarantee that this was the case with the completed workbooks submitted at the end of the module which was replaced by the microassessments. Furthermore, supervising students taking the large number of microassessments implemented in the module would neither be practical, nor desirable. It is view of the author that making each microassessment count, was essential to ensure student engagement with the learning material. However, the requirement to get a mark of at least 30% in each of the assessment components including the end of module exam and the fact that the exam mark was the largest component of the overall module mark minimised the potential impact of this scenario.

Second, the provision of only limited feedback after a given attempt was intended to prevent a student who had completed the exercise from passing useful information to a student who has yet to take it. It would have been desirable to give students immediate feedback, a feature which was requested in the student feedback responses. However, this would require the development of a large question bank from which to draw the questions, in order to reduce to an acceptably low level the probability that two students in contact both receive the same question in order for one to provide feedback to the other. An alternative approach would have been to “time release” the feedback as soon as the scheduled time for the particular microassessment had ended. Unfortunately, the Perceptionä system does not provide a way to “time release” the assessment diagnostic reports. They are either available for all assessments immediately after completing an attempt or not at all.

Third, the impact on student engagement with the course content was evaluated in terms of end of module exam performance. Thus, while concern may remain as to depth of learning that can be assessed with online assessments, the enhanced engagement with the module content has to be set against the context of a series of three anatomy and physiology modules in the second year of a four year degree programme where the aim in is to give a broad overview of the anatomy and physiology of the various organ systems of the body. More in depth study takes place in the subsequent two years of the programme.

Fourth, there were no significant periods of absence by any of the students that resulted in a request to reschedule any of the assessments. Given that there were three microassessments each week for eight weeks, the impact of missing one or two out of 24 would have been minimal. Furthermore, whilst the Pre- and Post-laboratory assessments were only available for relatively short time periods, the Tutorial assessments had the highest weighting and were available for a period of 11 days. In practical terms, because the aim was to engage students with the module content as they went along, it would probably be better to compensate a student’s mark rather than reschedule missed assessments as this would avoid over burdening them as they seek to catch up with the missed material.

Learning how to use the online assessment system took about five hours. The ready availability of some paper-based questions reduced the amount of development time to around one hour for each weekly topic. Ongoing scheduling and monitoring took about 30 minutes each week. The ability to monitor class performance in the assessments meant that it was possible to provide feedback regarding the key areas where significant numbers of students were having difficulty and suggest areas for remedial study.

In conclusion, improved end of module exam performance after the introduction of carefully designed microassessments supports their use as a tool to enhance student engagement with the learning material in a module throughout term time.

Communicating author: Alasdair G Thin, School of Life Sciences, Heriot-Watt University, Edinburgh, EH14 4AS, United Kingdom.

Tel +44 (0) 131 451 3469; Fax +44 (0) 131 451 3009; A.G.Thin@hw.ac.uk

References

Chickering, A.W. and Gamson, Z.F. (1987) Seven Principles in Good Practice in Undergraduate Education. Racine, WI, USA: The Johnson Foundation Inc

Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education E-journal, volume 1 available at
http://www.glos.ac.uk/adu/clt/lathe/issue1/index.cfm
(accessed 12 May 2006)

Gunn, A. and Pitt, S.J. (2003) The Effectiveness of Computer- Based Teaching Packages in Supporting Student Learning of Parasitology. Bioscience Education E-journal, volume 1 available at
http://www.bioscience.heacademy.ac.uk/journal/vol1/beej-1-7.htm
(accessed 12 May 2006)

Horgan, J. (1999) Lecturing for Learning. In A Handbook for Teaching & Learning in Higher Education. ed. Fry, H., Ketteridge, S. and Marshall, S. pp 83-94. London, UK: Kogan Page

MacFarlane, B. (1992) The 'Thatcherite' generation of university degree results. Journal of Further and Higher Education 16, 60-70

Maier, P., Barnett, L., Warren, A. and Brunner, D. (1998) Using Technology in Teaching and Learning. London, UK: Kogan Page

Overfield, J.A. and Bryan-Lluka, L. (2003) An Evaluation of Factors Affecting Computer-Based Learning in Haemostasis: A Cultural Experience. Bioscience Education E-journal, volume 1 available at
http://www.bioscience.heacademy.ac.uk/journal/vol1/beej-1-9.htm
(accessed 12 May 2006)

Snyder, B.R. (1971) The Hidden Curriculum. Cambridge, MA, USA: MIT Press