Research Article

Evaluating University Masterclasses and School Visits as mechanisms for enhancing teaching and learning experiences for undergraduates and school pupils. A pilot study involving biotechnology students.

ANGELA TODD AND DENIS J MURPHY

Biotechnology Unit, School of Applied Sciences, University of Glamorgan, CF37 1DL, United Kingdom

Date received: 06/08/03                     Date accepted:18/10/03

We describe a pilot study to investigate the impact on teaching and learning experiences in 3^rd year Biology Honours undergraduates who were challenged to work interactively with 15-18 year old school pupils in a topical area of modern bioscience, namely the use of GM (genetically modified) crops. We were also interested in enhancing the knowledge base and decision-making skills of the school pupils in relation to this complex and contentious area of contemporary scientific and public discourse. Using statistical data generated by students, we compare here the effectiveness of two contrasting interactive formats, namely University Masterclasses and School Visits.

Our major conclusions are:

·         Course marks and general motivation of participating undergraduates were improved and excellent student feedback was received

·         Both the Masterclass and School Visits formats were quantitatively effective in terms of pupil outcomes (i.e. regarding knowledge acquisition and decision making skills), as measured by statistical analysis of questionnaires

·         Of the two formats, the School Visits format was judged to be more flexible, more suitable as an individual student activity and more easily assessed as an independent activity

Both formats were strongly supported by teachers, who also wished to participate in similar activities in the future.

Introduction

The teaching of molecular biology and biotechnology at undergraduate level can sometimes be challenging to educators. These topics are often regarded as technically demanding by students and many courses and textbooks dwell on the molecular mechanisms underlying biological development, most especially focussing on gene function. However, at the same time, many students who may not be “natural” molecular biologists are intensely interested in the wider issues raised by, for example, human cloning or GM crops. Biotechnology is, therefore, both a relatively difficult technical discipline, but also one that also raises many complex and contentious questions in areas as diverse as public awareness, commerce, ethics and politics (Polkinghorne, 2000). Previous experience of teaching biotechnology and related subjects has given us anecdotal evidence that a broadening of teaching methods, to encompass direct discussion and role-playing, can improve student-learning outcomes. This was true for students in general, but was especially so for those who were less comfortable with the conventional didactic approach to the subject.

In order to study further the utility of broader teaching methods, we established a limited pilot study focussed on a previously established university-schools partnership in biotechnology. We studied the impact on learning experiences of third year biotechnology undergraduates when they were given direct practical experience of the explanation and discussion of biotechnology issues to school pupils. We hypothesised that the need to develop teaching resources and to communicate with school pupils might improve overall student comprehension, not only of more general issues, but also of the core scientific content of molecular biology and biotechnology.

The major aim of this pilot study was to investigate the impact on the development of additional cognitive and communication skills in 3^rd year Biology Honours undergraduates by challenging them to discuss issues relating to GM (genetically modified) crops with school pupils. For the sake of clarity, in this study we refer throughout to university undergraduates as “students” and to the 15-18 year-old school learners as “pupils”.

As a further challenge to the students, we specifically selected schools from socially disadvantaged areas of the Welsh Valleys, having historically low participation rates in further and higher education (Further Education Funding Council for Wales, 2000).

Methods

The academic context of this study is an ongoing, yearlong classroom-based 3^rd year undergraduate module entitled “Plants, Communities and Biotechnology”. The module is normally centred on lectures and tutorials, with additional coursework consisting of a series of paperwork assignments ranging from essays to shorter Blackboard-based discussion items (Blackboard). During 2002-03, we ran a pilot study whereby we compared the impact of two additional alternative activities involving discussions about biotechnology with school pupils, namely visits by students to schools or university-based Masterclasses for pupils. Both activities required undergraduates to prepare teaching materials for use in classrooms with pupils who were studying biology at GCSE and AS/A2 levels, i.e. ages 15-18. In all cases, the biotechnology topics that we covered were included in the school curriculum.

Masterclasses

Masterclasses were structured, daylong sessions in which groups from several schools, numbering up to 80 pupils and teachers, visited the university campus. Each Masterclass included a seminar, visits to research and teaching labs and a full-scale debate on the motion “biotechnology has gone too far". A typical debate format involved four speakers, two speaking for the motion and two against. Following the speakers' presentations, the debate would be thrown open to discussion from the audience.

Schools Visits

These visits were undertaken by individual students to selected schools in the local area, i.e. within a 30 minute travelling time. Visits were normally of about 2-3 hours and for each class they included a 20minute presentation after which the class was invited to complete questionnaires and then engage in a discussion on the issues raised. The pupils could ask questions on the presentation, testing the student’s background knowledge and communication skills. The discussions undertaken were often initiated by the teachers who had also listened to the presentation, thereby ensuring a productive visit for the student. These activities were designed to cover a single lesson, allowing for the involvement of up to three classes during one school visit. The arrangement of these visits required a degree of organisation from the students and liasing with the teachers to ascertain suitable times, lesson structure and class sizes. Students helped to develop questionnaires, information leaflets and a Powerpoint presentation that were then used in schools to support the interactive sessions with the pupils. A useful background text for students to use in the formulation of their discussion session was Barnes and Todd (1977) supplemented by the more recent reports by Simonneaux (2002a, b).

Monitoring by questionnaires

The results of Masterclasses and School Visits on pupil perception of GM-related issues were monitored by a series of questionnaires, supplemented by the recording of direct feedback from pupils. The full questionnaires and some of the detailed responses are included as Supplementary Information to this paper. In most cases, three questionnaires were used for each group of pupils. The first questionnaire was sent to schools for completion about a week before pupils were exposed to any new teaching materials. Pupils were then sent brochures designed to supplement their preparation work with teachers. Pupils completed the second questionnaire on the day of the activity, immediately after its completion. This questionnaire was designed to assess the effects of the activity on pupils’ perceptions of GM-related issues. We were particularly interested in any topics where pupils had changed their opinions as a result of their interaction with the student. The third questionnaire was completed by pupils several months after the activities and was designed to measure the extent to which changes in pupil perception may have altered over the longer term, rather than immediately after the activity, i.e. were their opinions merely altered transiently by the activity or are any observed changes more long lasting?

Results

The pilot project was particularly successful in two regards.

1.      The students developed a range of transferable skills, enhanced motivation and improved comprehension of their academic modules.

2.      We developed a close relationship with teachers and pupils at several local schools. Pupils particularly valued the interactive sessions with the students that took them beyond their everyday lessons.

Some undergraduate comments on the project outcomes:

"I feel that this project has given me added confidence and aided my development in areas of communication and public speaking."

"I found that all the different aspects of the project stimulated my personal interest and made me more enthusiastic."

"I think that the increased confidence in communicating, public speaking and organisation of events gained here are all skills that will be useful in future employment."

Student involvement in teaching materials and data analysis   

Students helped to develop Powerpoint presentation for pupils and to formulate questionnaires. In both cases, basic templates were made available to the student but the adaptation of these templates for the different pupil audiences was a key aspect of assessment. In future, we plan to vary the extent of student input into preparation of teaching materials, e.g. by including the production of the brochure or any other aids that may be deemed useful. Students could also be encouraged to volunteer additional material. Data analysis was an important aspect of this study because it gave the students qualitative and quantitative measures of the impact of their interactions with the school pupils. The collection, presentation and analysis of the data by students formed part of their assessment.

In Box 1, we present some data and analysis from one student who participated in assisted in two Masterclasses and two School Visits. This shows some of the issues that can be explored with these formats and how they relate to academic aspects of biotechnology. The data include verbal feedback and analysis of questionnaires from the student activities with the schools.

BOX.1 An example of student data obtained from Master-classes and School Visits  Some comments made by participating pupils: "I found the debates more useful than the talks and would like to have more in the future "I found the debates more interesting and would like to have more talks in the future covering topics not covered at school such as, bioplastics" "I think the Masterclasses are a good idea" Feedback from participating teachers: ·         Teachers used both activities as supplementary lessons to support their formal science curriculum teaching ·         They commented that the students had pitched their presentations at the right level for pupils to understand ·         They wanted to be involved in any future activities Analysis of the questionnaires An example of one of the questionnaires used in this study and a more complete set of pupil responses are appended as “supplementary information”. Some general conclusions from these data are presented below and some of the key findings are summarised in Figure 1 and in Tables 1-2. In general, pupils felt that both the Masterclasses and School Visits helped them understand better the issues covered and thought these activities were useful. For example, after the Masterclasses, 88% of pupils reported that they understood more about biotechnology and 86% said they were better informed. On the wider issues discussed, pupils felt that GM foods could be beneficial to developing countries. However, they did not believe that GM food could ‘feed the world’. This suggests that they were aware of and were able to discount some of the more exaggerated claims made on behalf of GM foods. In regard to the GM Nation public debate on GM crops that was ongoing in the UK during the study in early-mid 2003 (GM Nation and GM Public Debate websites), pupils felt that we would eventually have GM foods and crops in this country anyway. The pupils felt that any reservations they held about the technology would be unlikely to influence government policy in this area. From the Masterclasses, 81% of pupils felt that GM was an important issue to discuss but only 29% of pupils thought that their views could have an effect on whether GM foods and crops are eventually accepted in the UK (see supplementary information). Other data from Masterclasses The data generated from questionnaires completed before and after each Masterclass were compared in order to assess its immediate impact on pupil opinion. A particularly important finding was that the proportion of pupils that were undecided in many of the key topic areas dropped considerably after the events. Some of these data are summarised in Figure 1 (see supplementary information for full data sets). In many cases, the pupils became more enthusiastic about some of the potential benefits of GM technology immediately after the Masterclass. However, not all opinions changed in the pro-GM direction. Pupils tended to become more cautious when asked if they would eat GM foods - whereas 46% said they would eat GM foods before the Masterclass, only 36% would do so afterwards. Figure 1a        Master-Class Interestingly, the general tendency to be more supportive of GM technology immediately after the Masterclasses was not maintained in the first follow up surveys of pupils conducted 6 months after one of the Masterclasses during March 2003). Here pupils became increasingly undecided on the topics that they were initially enthusiastic about after attending the Masterclasses. Table 1 compares four of the key questions. For three of these questions, pupils tended towards being more ‘undecided’ after the follow up study. We interpret this as a positive change of opinion to a less “black and white” pro- or anti- GM stance (perhaps after recognising the complexities of the issues) rather than as a mere abstention from any opinion at all. Table 1 Masterclass responses This table compares pupil responses to four key GM related questions asked on three different occasions, a) before the Masterclass, b) immediately afterwards and c) a follow up study 6 months later (in March 2003). Question Agree/Strongly Agree Neither Disagree/Strongly Disagree Say how much you agree or disagree with the following statements A          b          c A          b          c A          b          c I think that GM foods may sometimes be more nutritious and, therefore, healthier 0          60         30 84         40         55 16         0          5 GM food is more suitable for poorer regions of the world, e.g. in sub-Saharan Africa and Asia 35         45         20 84         40         55 10         30         5 Enough research has already been done and I am convinced that GM is safe to eat 5          10         20 69         45         35 26         45         40 I think that GM foods are unnatural and it should not be produced 21         10         5 47         40         60 32         50         35 Sample sizes for the questionnaires are respectively 19, 20 and 20. All answers are shown as percentages (*NB any missed answers are not shown in the data analysis and, therefore, will not add up to 100%). Pupils were originally asked how much they strongly agreed, agreed, neither, disagreed or strongly disagreed with the statements and for ease of interpretation the results for strongly agree/ agree and strongly disagree/ disagree have been calculated together. Other data from School Visits Comparison of questionnaire data before and after the Schools Visits demonstrates that these activities have helped the pupils form opinions as reflected in the lower numbers of pupils being ‘undecided’ after the events. These results are summarised in Figure 1b. The overall trend in opinion change for this study was in favour of GM technology. As with the Masterclasses, a 3 month follow up study showed increasing uncertainty in some areas, e.g. about whether GM foods could be more nutritious and therefore, healthier (see Table 2). However, at the same time, pupils became, significantly more supportive of statements like “some types of GM food may be good” (increased from 77% to 100%) and “I approve of GM on practical and scientific grounds” (increased from 39% to 75%). Figure 1b      School Visit Figure 1 Comparison of pupil responses to GM related questions before and after participating in either a) Masterclasses or b) School Visits. Masterclasses sample sizes were 52 for the “before” questionnaire and 53 for the “after”, with 4 local schools taking part. School Visits sample sizes were 34 in both cases. Pupils were asked how much they strongly agreed, agreed, neither agreed nor disagreed, disagreed or strongly disagreed with the statements. For ease of presentation here, the results for strongly agree/ agree and strongly disagree/ disagree have been combined.

Discussion

Our major conclusions from this pilot study are as follows:

·         Course marks and general motivation of participating undergraduates were significantly improved compared with previous performance

·         Both the Masterclass and School Visits formats were quantitatively effective in terms of pupil outcomes as measured by analysis of questionnaires

·         The School Visits format is more flexible, more suitable as an individual student activity and is more easily assessed as an independent activity

·         Both activities were strongly supported by teachers, who also wished to participate in similar schemes in the future

The pilot study described here shows that relatively open-ended activities involving outside contacts with schools can be successfully incorporated into a full 20-credit, third year biotechnology module. At first sight, it may appear that the additional workload involved in a Schools Visit programme, including all the associated preparation and analysis, may divert students from the primary scientific goals of the module. However, we found that this was not the case: indeed, student enthusiasm for and comprehension of the “hard science” parts of the module were definitely improved by their participation in the school-related activities. The students needed to master enough science to be able to design their presentations and to engage in informed discussions with pupils. This stimulus gave them a strong motivation, not only to understand the lectures, but also to read around the topic and especially to research into public concerns and ethical implications of biotechnology. An alternative format to teaching wider principles of biotechnology transfer is the service-learning approach as recently described by Montgomery (2003). This can involve community-based activities using partners such as local non-profit organisations, rather than schools. Therefore, the use of any one of a variety of approaches can result in favourable outcomes for students. In our case, we preferred a school-centred approach as a way of addressing additional issues in relation to low rates of educational achievement in our local region (see below).

The data analysis used in this study involved skills that were also used in other course modules, e.g. ecology. However, by using these techniques to analyse pupil perceptions, the students were strongly reminded of the limitations and potential misinterpretations involved in such activities as opinion polls and consumer surveys. Students also reported personal satisfaction at being able to engage in constructive discussions with teachers and pupils in an area where, for once, they were regarded as “the expert”. It is not possible to quantify the extent of empowerment felt by students but it is apparent that participation in such activities can bolster their confidence and general motivation. We are also aware that the students gain useful transferable skills in a wide variety of areas from public speaking to preparation of handouts and brochures.

It is apparent that these sorts of activities may not be effective or suitable for all students. The scheme should, therefore, be run as an option alongside alternative learning methods. This will enable students who are more suitable for schools activities to self-select with lecturers then carrying out additional screening as required. Nevertheless, for those students who volunteer, it appears that these activities can help many aspects of their performance in addition to conferring transferable skills, especially in relation to communication and organisation.  

Another attraction of these sorts of activities is that, in addition to being beneficial to our own undergraduates, they also succeeded in stimulating interest in biology among school pupils as well as involving their teachers. Our first partner schools were in the relatively deprived former mining communities of Aberdare and Beddau, which are in the Welsh Valleys and within 10 miles of the university. Rates of entry into tertiary education (especially universities) from such areas are well below the national average and are the subject of great concern, both in Wales itself (in the National Assembly) and in the UK as a whole (Further Education Funding Council for Wales, 2000). Despite these gloomy statistics, we found the pupils and teachers to be receptive and keen to engage in dialogue with our students.

In the future, we plan to perform a follow up study where we will use quantitative indicators to compare two groups of students taking the same module, where one group will participate in the Schools Visit programme while the other group spends equal time on a more science-focused activity. We will also initiate collaborations with colleagues in other regions of the UK in order to extend our knowledge base and to refine a set of activity formats and principles for possible use in the wider higher education community.

Communicating author: Denis J Murphy, Biotechnology Unit, School of Applied Sciences, University of Glamorgan, CF37 1DL, United Kingdom: Tel: +44 1443 483 747; Fax: +44 1443 482 285; dmurphy2@glam.ac.uk,

Acknowledgements

We thank Karan Simmonds and Jean Morgan of the University of Glamorgan Centre for Education and Lifelong Learning for their assistance with the Masterclasses and Dr Colette Murphy of Queens University Belfast, for comments on the manuscript. Financial support was from Wales Science Year, Biochemical Society and the Learning and Teaching Support Network (LTSN).

References

Barnes, D. and Todd, F. (1977) Communication and Learning in Small Groups. London, Routledge

Blackboard – this refers to the Blackboard Learning System™, which is a Web-based server software platform and course management system, http://www.blackboard.com

Dawson, V. (2003) Western Australian school students’ understanding of biotechnology, Int. J. Sci. Educ. 25, 57-69

Further Education Funding Council for Wales (2000) Further and higher education statistics in Wales. http://www.wfc.ac.uk/stats/statsvol/1998-99/pdfs/section_2.pdf
Accessed 14 October 2003

GM Nation and GM Public Debate websites (2003) http://www.gmpublicdebate.org http://www.gmpublicdebate.org.uk/gmdebate.htm
Accessed 14 October 2003

Montgomery, L.B. (2003) Teaching the principles of biotechnology transfer: A service approach. Electronic Journal of Biotechnology, 6 (1), 13 - 15
http://www.ejbiotechnology.info/content/vol6/issue1/teaching/01
Accessed 14 October 2003

Polkinghorne, J.C. (2000) Ethical issues in biotechnology. Trends in Biotechnology, 18, 8 – 10

Simonneaux, L. (2002a) Analysis of debating strategies in classroom in the field of biotechnology. Journal of Biological Education, 37 (1), 9 -12

Simonneaux, L. (2002b) Analysis of didactic strategies to help pupils develop argumentation skills in the field of biotechnology. www.iubs.org/cbe/pdf/simonneaux.pdfAccessed 14 October 2003

Supplementary information (PDF version only)