Research Article

Making a Level Playing Field at Master’s Level – an application of self-directed learning

Lesley-Jane Eales-Reynolds

School of Biomedical and Life Sciences, University of Surrey

Date received: 28/04/03                     Date accepted:14/07/03

Introduction

The modular degree structure has been adopted widely as a means for effectively delivering curricula at both undergraduate and postgraduate levels. There is a wide range of formats for modular learning but perhaps the most restrictive is that where the material is delivered in short concentrated periods and after a short period, the student is examined. Clearly, this does not give the students time to synthesise or consolidate the information or to relate it to other specialist areas of their degree programme. This lack of deep learning (Biggs, 1999) is particularly problematic with Masters students and modules that are not components of their main discipline but are vital to a fuller understanding of those modules that are. Such subjects provide a particular problem because students often come with diverse prior learning experiences and show a wide range of ability with respect to the particular topic. It has long been recognised that we need to develop students as autonomous learners (Boud, 1982). However, prior educational experiences, lack of interest in certain subjects, limited experience of assessment procedures, lack of comprehension of the level of work required, a preference for a didactic approach to education and the strictures of the modular system mean that the majority of students on our Master’s programme in Medical Microbiology have been resistant to the idea of self-directed learning (SDL)

The Problem.

For more than 25 years, we have run a successful Master’s programme in Medical Microbiology at the University of Surrey. It is a part-time programme run on a day-release basis for persons working in an appropriate laboratory and having suitable academic qualifications. A component of this Institute of Biomedical Sciences validated programme is a module in Immunology. Until comparatively recently, most students accepted on the programme had little or no immunological knowledge. However, in recent years the range of ability of the students with respect to immunology has been considerable. Although we offer APEL and CATS, many students choose not to be exempted from courses where they have prior experience since they can use this module to boost their overall degree mark or use it to compensate for poor performance in another module. Having such students in a relatively small group (between 25 and 30 students), can be distracting to the others (who feel disadvantaged) and can lead to a false sense of security where students meta cognition out strips their true cognition.

The learning outcomes of the module state that at the end of the module the students should have a thorough knowledge of the terminology associated with immunology and an understanding of the basic concepts and their relevance to medical microbiology. In previous years this was assessed by both simple and more searching multiple choice questions (to assess their knowledge of the relevant terminology and basic concepts) and a data analysis paper (which assessed their ability to relate the basic concepts to medical microbiology). The results of this approach to assessment suggested that it was probably not appropriate since the mean module mark was significantly higher than that achieved for the majority of other modules in which the students had far more experience. In addition, students did not exhibit retention of the information where it was applied in other modules. Although the students were obtaining high module marks, this was largely due to their exceptionally high achievement in the multiple choice questions. When questioned they said that they could practice and recognise the correct answer despite the fact that the programme shuffled the answers each time. Their poor performance in the examination demonstrated that whilst they might be learning the terminology they were not developing an understanding of the subject that they could then apply. In order to determine whether we could overcome these problems and ensure the students achieved the learning outcomes, we took a student-centred approach to the course and adopted assessment by portfolio and open book examination. This meant that the stronger students were encouraged to work with the weaker ones and through tutorials and set goals, the students had to develop a more in depth approach to their studies.

Methodological theory

The SDL approach to learning and teaching has been developed over many years in a wide range of subjects. When considering the design of this module, the model proposed by Grow (1991) derived from the model of Hersey and Blanchard (1988) was considered. Grow’s structured self-directed learning model categorizes learners as 1) dependent, 2) interested, 3) involved or 4) self-directed. It also identifies the type of teacher such students require being respectively 1) authoritative coach, 2) motivational guide, 3) facilitator or 4) consultant, delegator. In considering the range of abilities within the group, from past experience it was likely that the majority of the students would be interested learners, with a few dependent learners and a few involved learners. It was considered unlikely that we would have any self-directed learners. Thus from Grow’s model the best fit approach was considered to be one in which the lecturer played the role of “motivational guide” providing an “inspirational lecture”, set goals and learning strategies and guided discussion.

The approach

The Masters programme is run on a two-year rolling basis such that the intake from the current year joins the intake from the previous year. This means that in any group, approximately half the students will be on the first year of the programme. The immunology module runs in January and comprises 6 one-day taught sessions. In previous cohorts, this has been delivered as a lecture course. Assessment comprised two, self-administered, computer-based multiple-choice tests (in which the questions and answers were randomly sorted and students could practice the test and submit their best result on specially coded paper. The final assessment consisted of an examination comprising data analysis questions of which the students had to answer two out of four in two hours. Students were given practise at data analysis questions during the course of the module and were given feedback and guidance about answering such questions.

In the 2001-2002 academic year, students starting the immunology module were given a short answer paper previously set for HE2 students without prior warning. They were given an hour to answer the paper and were told that this would not contribute towards the assessment of the module. The students were then given short module guide describing the structure that the course would take, their expected role, and the nature of the assessment. This was reinforced by a question and answer session about the format of the module. Their papers were assessed immediately and the students graded as either “novice”, or having “basic” or “advanced” knowledge. The class was then divided into groups of 5-6 students each with an even balance of novice, basic and advanced students. The course was divided into a series of “key topics” (for example “ Cells and Tissues of the Immune System”) and each was given a themed “key question” e.g. “Describe how and where the cells of the immune system become physically distributed within the body”. Students were told that the “key topic” described the broad area to which the contents of their portfolios should be relevant. Their answers to the “key questions” were expected to provide a summary of all their obtained knowledge for each topic and to link together the contents of their portfolios for each particular topic. They were also given further information about the type of information that would be acceptable for inclusion, how the material should be presented, (and its inclusion justified) and how it should be referenced. Students with different abilities were expected to use different sources of material (e.g. those with more advanced knowledge were expected to be using review and research articles to increase their knowledge whilst those who were novices were expected to use text books and more basic resources available on the Web). Students were also given advice about where to find material and how to manage their group work.

Students were given an overview lecture, with notes and then asked (in their groups) to explore the key question. They were allowed library time and reconvened for a tutorial session where group discussions were held on the key topic. The “advanced knowledge” team member usually led the discussion between the students. In addition, I acted as a facilitator for these discussions. Students were told that they needed to produce a portfolio of material (notes, photocopies of articles etc.) for each “key question” and a two-page summary of the topic. I reviewed this summary, gave suggestions as to deficits in their study and once it was approved, the students were allowed to take the work into their final examination. To ensure that the work submitted by each student was his/her own, the summary, and abstracts/notes on included published material had to be handwritten. No limit was put on the amount of material to be included but this was carefully monitored and advice given during the tutorial sessions so that students became selective in the material they included.

Each subsequent session was handled in the same way. At the end of the module, the students had two weeks during which they could polish their final key question summaries and have them authorised. They were allowed to take these summaries into the final examination but the examination questions were rephrased to ensure that the students had to re-interpret their notes during the examination (e.g. “Give a detailed account of how lymphoid cells circulate throughout the body giving special emphasis to the role of lymphoid tissues”. At the end of the exam, the students submitted their papers, portfolios, and key question abstracts. The latter were assessed along with their portfolios for the coursework section of the assessment.

Students were asked to complete the standard “student assessment of module form”. In order to assess the effectiveness of this approach, students who completed this module in the second year of their studies were compared with the equivalent cohort from two years earlier who had completed the taught immunology module with the old form of assessment. Statistical analysis was performed using Student’s t-test for the comparison of means. Where appropriate paired analysis was performed. In addition, analysis of variance and correlations were performed.

Results- Quantitative Analysis

The two cohorts comprised year two students who studied immunology under the old system of lectures, multiple-choice and data analysis questions (OLD) or under the new system of self-directed learning, portfolio, and open book examination (NEW).

The OLD cohort comprised 10 students and the NEW cohort 14. The total taught course results for each cohort were found to be significantly different (Figure 1a; OLD 61+7%; NEW 56+5%; p<0.04) suggesting that the cohorts had different academic abilities. In order to determine if this was solely due to changes in the immunology module, the immunology results were removed from this total. However, there remained a significant difference between the two cohorts (Figure 1b; OLD 61+6%; NEW 57+4%; p=0.05) demonstrating that the NEW cohort appeared to be weaker academically than the OLD cohort. When the results for the immunology modules were compared, there was no significant difference between them (Figure 2; OLD 64+11%; NEW 53+10%) owing to the large standard deviations in both groups.

Upon further analysis it was found that the examination results for the two cohorts were similar with the OLD cohort having a slightly lower mean percentage than the NEW cohort in contradiction to the evidence suggesting that the NEW cohort was weaker than the OLD (Figure 2; OLD 52+10%; NEW 55+9%; p=NS). However, when the coursework marks were compared, the OLD cohort performed significantly better than the NEW cohort (Figure 2; OLD 76+14%; NEW 51+13%; p=0.0003). The OLD cohort coursework mark was derived from the self-assessed computerised MCQ tests and an highly significant difference was found between the coursework and examination outcomes for this cohort (OLD coursework versus OLD examination p=0.0001). When their coursework or exam results were compared to their overall performance (omitting the immunology module) it was found that in this cohort their coursework performance was significantly higher and their examination performance significantly lower than might be expected from their performance in the other modules (coursework (76+14%) versus course total –the immunology module mark (61+6%) p=0.0013; examination (52+10%) versus course total – the immunology module mark (61+6%) p=0.0003).

The results obtained in immunology by the NEW cohort did not significantly differ from their overall performance in the other modules. From these results, it may be inferred that the new form of assessment was more representative of the students’ abilities.

In order to assess if the new approach to the module improved retention of the subject, student’s performance in immunology was compared to that in a related module. All students have to complete a practical module that runs some weeks after the immunology theory module and comprises several applications of the theory provided during the earlier module. When a comparison was made between the OLD and NEW practical modules, it was found that the students performed similarly in their bacteriology practical, a subject with which they are very familiar and have considerable experiential learning (OLD mean 65 + 9%; NEW 65 + 12%). In addition, the students’ performance in immunology was not significantly different owing to the large variation in individual performance. When this data was considered with reference to the students’ performances in the theory module, an interesting pattern emerged (Figure 3). Sixty percent of students in the OLD cohort showed a highly significant decrease in their performance in the practical module compared to the theory module (p<0.008). Analysis of the results of the remaining students showed no significant difference. By contrast, in the NEW cohort, although 50% of students showed a significant decrease in performance in the practical module (p<0.006), 50% showed a significant improvement in performance (p=0.015). This suggests that the new modes of delivering the module may have improved student retention of information and/or improved their ability to apply theory to practical situations.

Results -Qualitative analysis

Obviously, quantitative analyses in such studies have many shortcomings. Cohorts may have very diverse backgrounds and levels of academic or experiential learning. In addition, whilst this new approach to delivery may have influenced the learning outcomes for the students, if they have not developed as independent learners and have not enjoyed the experience, then the increased workload for the academic involved may not be a valid use of time. Thus, qualitative assessments were also carried out. Such assessments have to adhere to School policy, which requires student assessment of modules to be anonymous. Thus, we have no means of associating our standard assessment outcomes with individual student feedback. In addition, feedback is usually obtained immediately after the teaching of the programme is completed and prior to students having any results or their final assessment. The feedback consists of three parts two completed by the students (one about the module, the other about their own performance) and one completed by the module organiser about student performance and involvement. The feedback forms request written comments on various aspects of the module such as speed of delivery, clarity of objectives, quality of audiovisual aids etc.

The comments from the OLD cohort who received the module through lectures and handouts were generally very positive. Students found some aspects of the course difficult to understand and felt some lectures were rushed/overloaded for the time allowed. The students enjoyed the freedom of the self-assessment multiple-choice questions but admitted that after practice they could recognise the correct answers without necessarily comprehending the underlying concepts. When assessing their own performance they generally felt that they had spent an adequate amount of time on their study (the average being 2.5 hours per week, the recommended time being 10 hours). In this cohort, 35% of the students gave the module an overall “satisfactory” mark (i.e. 3/5) and 65% gave an overall “good” mark (i.e. 4/5).

The comments from the NEW cohort were extremely diverse ranging from “excellent, extremely enjoyable, I finally understand immunology” to “I didn’t expect to have to teach myself!” In this cohort, the breakdown was 9% “excellent” mark (i.e. 5/5), 39% “good”, 35% “satisfactory” and 17% “poor” (i.e. 2/5). The majority of students reported that they had been extremely uncertain of what was required of them at the beginning of the module (an observation confirmed during the mentoring sessions in the early part of the course), which resulted in them spending many hours on study initially. During the progression of the module, their self-assurance increased and their study hours reduced considerably. However, their average study time per week was approximately 8 hours. By the end of the module, these students felt that they had gained in confidence tremendously, had a good understanding of immunology and had very much enjoyed the module. Despite this, a small proportion of the students (17%) did not like the mode of delivery at all and did not feel confident of their understanding and more than 50% of students expressed a preference for a more didactic approach. Despite the general agreement that students felt they would do better with a structured lecture course, only 29% of the NEW cohort failed to score over 50% on their examination compared with 40% of the OLD cohort.

Some students were informally interviewed subsequent to completing the module and receiving feedback on their portfolios. The one student who failed the NEW module scored an overall mark of 52% for his taught programme. Although giving a weak exam performance, his portfolio was even weaker. When asked if he understood why he had performed so badly his reply was “I did not come to university to teach myself. I want lectures, an essay and two examination questions”! Interestingly, this student failed to attend several of the lecture and group discussion sessions, repeatedly failed to produce work for feedback, (despite frequently being encouraged to do so) and repeatedly replied that he was perfectly happy and that he knew what he had to do. 

Discussion.

I have been encouraging students to develop their independent learning skills for many years (Eales, 1993), particularly at undergraduate level. In deed in an effort to encourage self-assessment, a programme was developed which was published with my textbook (Eales, 1997) to allow students to undertake randomly generated MCQ tests. In the present study, this method of assessment was used by the “OLD cohort” to provide a coursework mark for the module. However, this was my first implementation of an SDL approach at Master’s level.

The purpose of this study was to determine whether or not an SDL approach to delivering the Immunology module of the MSc in Medical Microbiology would improve student performance and provide a deeper understanding of the subject, which could be applied in related modules. The quantitative analysis appears to demonstrate that the SDL approach engenders a more representative outcome and improves student ability to recall and apply the information accrued. However, the qualitative analysis appears to indicate that although many students appreciated the value of this approach, by choice they would prefer a more didactic method. Candy (1991) noted that although a student may be independent in some aspects of any teaching/learning situation, this does not mean that (s)he is broadly autonomous. This is clearly supported by our qualitative analysis where students performed well independently but would have preferred a more traditional approach.

In designing this module, I used Grow’s model of structured SDL and made assumptions concerning the dependence/ability of the students (Grow, 1991). Candy also states that: "the fact that a learner does not choose to exercise control in a particular educational setting cannot be taken as evidence that he or she lacks personal autonomy in the broader sense".  Thus, our student who failed and expressed a wish to be “lectured to” cannot be automatically classed as a dependent learner. In deed, this preference for deductive presentation (i.e. tell me what I need to know) is common amongst students and may not necessarily reflect a lack of autonomy; it could be reflecting prior educational experience (habit) or even laziness! 

Overall, it appears that this approach has been reasonably effective in meeting our aims. However, there is little evidence to suggest that it has encouraged the students to become self-directed learners. This may be because the model applied failed to precisely match the needs of individual students. Thus, this module will be developed further incorporating a method to identify the category into which each student falls according to Grow’s model. In addition, I propose to identify each student’s preference for particular learning styles (Felder and Silverman, 1988; Felder, 2002) so that the learning strategies can be tailored to meet the needs of individual students. It is proposed to use the Index of Learning Styles Questionnaire (Soloman and Felder, 2003) to allow students to identify their learning style preferences and to provide a guide to students as to the type/format of material they should be using to achieve the objectives of the module. 

Communicating author Dr L-J Eales-Reynolds, School of Biomedical and Life Sciences, University of Surrey, Guildford, GU2 7XH: Tel +44 (0)1483 689736; Fax +44 (0)1483 300374;  l.reynolds@surrey.ac.uk

Acknowledgements:

I would like to thank all the MSc students who were unwitting volunteers for this study.

References

Biggs, J.B. (1999) Teaching for quality learning in university. Buckingham: Society for Research in Higher Education and Open University Press.

Boud, D. (1982) Towards student responsibility for learning. In Boud, D. (Ed). Developing Student Autonomy in Learning (pp. 21-37) London: Kogan Page.

Candy, P.C. (1991) Self-Direction for Lifelong Learning, San Francisco: Josse-Bass.

Felder, RM (2002) Authors preface to 1988 article. http://www.ncsu.edu/felder-public/Papers/LS-1988.pdf [Accessed April 12^th, 2003]

Felder, RM & Silverman, LK. (1988) Learning and Teaching styles in engineering education. Engr. Education, 78(7) 674-681.

Eales, L-J.(1993) Practising student centred learning in the Biological Sciences. In: CVCP Publication - “What is active learning?” Module 1 part 2. pp 31-33.

Eales, L-J. (1997) Immunology for Life Scientists. A student-centred approach. Wiley & Sons Ltd. (Second Edition, 2003).

Grow, G.O. (1991) "Teaching Learners to be Self-Directed", Adult Education Quarterly, 41 (3), 125-149.

Hersey, P & Blanchard K. (1988) Management of Organizational Behaviour: Utilizing Human Resources. Prentice Hall PTR.

Soloman, BA. & Felder, RM. (2003) Index of Learning Styles Questionnaire. http://www.ncsu.edu/felder-public/ILSdir/ilsweb.html [Accessed April 12^th, 2003]