Research Article

Facilitation and Assessment of Group Work using Web-based Tools

Trudy Hartford

Leeds Metropolitan University, School of Health and Human Sciences, Calverley Street, Leeds LS1 3HE

Date received: 27/05/2004       Date accepted: 03/05/2005

This paper describes a final year biosciences module consisting of workshops to which web-based tools were introduced. The ability of the online tools to facilitate group work and assess the students was investigated. The online module was delivered via WebCT as the VLE (Virtual Learning Environment). The initial concept was to enhance rather than substitute current teaching methods. The major part of this study involved implementation of online discussion groups to facilitate group work and to determine whether the online messages could be used to inform assessment. This proved successful from both the student and tutor perspectives, group work was certainly enabled and further modifications devised to assess the online interactions.

Introduction

There is a strong link between social interactions, critical thinking and deep learning (Resnick et al, 1991) and group work is an excellent way of allowing social interaction and thus developing cognitive skills. The importance of student-student interaction as well as tutor-student interaction has been emphasised (Entwhistle and Ramsden, 1983) and the use of an online discussion facility could increase the opportunity for these interactions. Collaborative work among students is often hampered by the students’ personal circumstances. For example: part-time students, mature students with families, or in fact students who work part-time to fund their studies. Collaborative learning and discussion-led learning can be achieved through asynchronous communications and student-centred and resource-based learning can be encouraged with a range of materials. This article outlines attempts to utilise web-based asynchronous discussion to facilitate, support and assess group work.

As teamwork is a skill frequently requested for comment when supplying references for future employers, it would be desirable to have some evidence to enhance personal opinion. Thus with this module it was hoped to investigate means to facilitate and mediate group work in some way and to assess group work in more depth, rather than just the final product of the group. Getting students to work in groups often involves the co-construction of knowledge - interactions which produce new understandings (Vygotsky, 1978); however this is very difficult to observe using current group work methods.

Pros and Cons of Group Work

Problems previously encountered with group work have included individuals giving low or no contributions to the group. Group journals or diaries do not highlight problems early enough, as the tutor does not usually view them until completion of group work. Other problems may arise when one student is academically brighter than the rest of the group; they then feel resentment that they would have done better without the rest of the group. However on this module, where the assessment has two components, students’ group mark has always been higher than their report mark, in agreement with Gibbs et al (1986), who found students produce work of higher calibre when they work in groups.

Giving the same mark is inequitable; one way to overcome problems is to involve students in the assessment. Benefits of peer assessment include encouraging responsibility (Goldfinch and Raeside, 1990) increasing maturity and confidence, and enhancing social relationships and relevant skills (Earl, 1986). Peer assessment can be used to enhance the fairness of assessment and encourage students to both participate in and reflect on the group work process. It is imperative for the students to have a complete understanding of the assessment criteria, feel comfortable about assessing and also confident that peers are assessing fairly (Orsmond et al, 1996). Peer assessment has been documented for a range of assignments in biological sciences (Falchikov, 1986; Orsmond et al, 1996; Stefani, 1994) and a range of grading and weighting rubrics reported (Cheng and Warren, 2000; Conway et al, 1993; Goldfinch, 1994; Lejk and Wyvill, 1996; Li, 2001).

Electronic Communications

Adoption of Virtual Learning Environments (VLEs) by Universities has increased the opportunity for student-student and student-tutor dialogue. Electronic interaction manifests in various forms: tutor-student, tutor-small group, tutor-whole cohort, student-student, and student-many other students. Communication may be synchronous, that is occurs in real-time, or asynchronous, where there is a time lag between interactions.

Research around online communications has focused on enabling and monitoring discussions, known as e-moderating (Salmon, 2000) and how to assess an individual’s contribution to a discussion. Henri (1992) was one of the first to investigate qualitative assessment of asynchronous discussion rather than quantifying contributions. Messages were divided into units and ascribed to one of four categories: social, interactive, cognitive and metacognitive, others have used similar approaches (Garrison et al, 2000, Popping, 2000). Gunawardena et al, (1997) based assessment on detection of knowledge construction, by identifying five phases that learners go through from observation to metacognitive statements. Newman et al, (1997) devised a similar system using a range of questions addressing critical thinking.

The method of incorporation and integration of online communication into the educational setting can be critical, as always it is how the tool is used that affects the enhancement of the Teaching and Learning process, not the tool itself. Asynchronous online discussion gives learners time to think about their comments and those of others before they compose coherent responses or pose questions. Therefore it can involve more students than would take part in the equivalent face-to-face conversation. Online communication appeals to learners who need time to digest information and it encourages reflection. It is suited to learning that involves argument, debate, conversation, and as here, collaboration between students.

Module Design

The original module, Integrative Studies, was delivered to final year bioscience undergraduates. The overall structure of the module is outlined in Table 1. The students were presented with a fictitious situation and were divided into groups to study different aspects of that situation (based on an idea from Sockett, 1999). For example topics have included bioterrorism, the MMR vaccine, and the crash of a train carrying radioactive waste. The group work culminates in a presentation at a ‘conference’. The students were put in control of their own learning to a certain extent as the detail of the topic to be researched was determined by the group. Each group was given a tutor as a resource which they could use for a maximum of two hours plus an initial consultation agreeing their group aims.

Table 1  Module Format

A WebCT module was designed to accompany the tutor-led workshops this comprised:

• a communication tool used for asynchronous discussion, this consisted of:
  • a communal area for FAQs and general feedback
  • private discussion forum to evidence group work, enhance and summarise face-to-face interactions
• chat rooms for synchronous discourse for students who were unable to meet face to face so that they could hold an online meeting
• some content and electronic references
  The content included a description of the scenario, ideas for the focus of their research, deadlines for the tasks given to each group, details of assessment including peer assessment.

There was a record kept of all discussions so students could digest these and reread at their own pace this was seen as an advantage over face-to-face sessions. Online group tasks with deadlines were given which included clarifying their group objectives and posting minutes of meetings evidencing group interactions. Lurkers or browsers were encouraged to participate by use of structured activities, e.g. a requirement to provide a reference online. Initial tasks were designed to encourage use of the WebCT aspect of the module, particularly the discussion tool, give a sense of achievement and open up tutor-student and student-student dialogue. Familiarity with the electronic environment is crucial to participation (Mason and Bacsich, 1998), so an online tutorial was given early in the modules to reduce frustration, anxiety and confusion that can occur with innovative modules (Hara and Kling, 2000).

Assessment

Originally, the module assessment comprised the products of group work (conference presentation and leaflet) moderated by peer assessment, and a report on the Conference itself. This was subsequently adapted, so the portfolio of assessment, worth 15 credits in total, contained:

·         Staff and Peer assessment of the products of group work (25%)
The leaflet, produced to assist the students in their critical evaluation of the conference (see below), was replaced by a webpage incorporating key references and a summary of the relevant presentation.

·         Completion (and quality) of online tasks (10%)
Initially marks were given for completion of tasks only

·         Peer assessment of group members (intragroup assessment).
Each student’s group mark was adjusted using their intragroup assessment (Cheng and Warren, 2000). Emphasis was placed on contribution and effort rather than academic ability. Students grade their peers on their level of contribution to group discussions, research and evaluation materials, preparation and organisation of output, reliability, response to criticism, whether they were helpful and supportive. A grid system containing definitions of what is/is not expected from a group member was used to collect the marks.

·         Self-evaluation and reflection (5%)

·         Critical Evaluation of the Conference (individually crafted report, 60%)

Initial Outcomes

Use of the online module

The use of WebCT was monitored using the integrated tracking facility.

This provided the tutor with a range of information, including the frequency at which students’ access the module, the popularity of the pages accessed, which tools were used. After session 1 only 6 students accessed online information. After the hands-on session only 6 students had not accessed the web material and this was due to technical difficulties. The module was accessed a total of 5849 times, an average of 390 times per student (29 students in total, range 0-650). The tool pages were accessed an average of 100 times per student over a nine week period - this included the discussion facility and references organiser page.

As an outcome of one of the online tasks, the references provided by each group on their topic were displayed on the module so that they could be utilised by all. At the students' request their own presentations in the form of PowerPoint or word files were added to the online content (five presentations in total, three online). These 'conference presentations' were very popular and were accessed over 50 times in total. A student presentation area has subsequently been added to the online module to accommodate presentation notes. This has replaced the A4 leaflet that groups produced as a product of their effort and has been incorporated into the group assessment.

Message Analysis

There was no clear relationship between use of the discussion facility and seminar quality. However the group with the fewest online interactions also produced the lowest quality group product (Table 2).

Table 2 Message Analysis

	Group 1	Group 2	Group 3	Group 4	Group 5
Number of Messages pertaining to Group Work	55	31	58	23	10
Interactive/cognitive messages (exhibiting skills relating to processes)	6	1	6	0	0
Meta cognitive statements (indicators of critical thinking)	5	0	0	1	0
Group Work Mark (%)	60	65	69	64	48

The quality of the online discussion messages were evaluated using evaluation questions based upon those of Gunawardena et al, (1997) and Lally (2000). This was to determine whether the discussion facility could evidence learning and group work. The process of reading, dividing and analysing the messages was time consuming. There was certainly evidence of sharing and comparing information, as found by others (Lally, 2000; Newman et al, 1995) but to use this method for group assessment, students would require explicit instructions on formulating and providing their evidence. The majority of the critical thinking evidence came indirectly from the minutes of the meetings posted by each group. Thus groups 1 and 3 wrote particularly good minutes evidencing their thoughts and discussions while the other groups stated outcomes only (Table 2). For the purpose of evidencing group work, assessment of messages using a series of questions based on detailed assessment criteria was relatively easy to apply and adopted for subsequent runs of the module.

Of the 64 messages posted by tutors 38 provided information or answered questions, 29 directly to groups and 9 in the whole cohort. The remaining messages provided feedback and encouragement to the whole cohort and to individual groups (23 messages). Individual feedback was given privately, that is through the e-mail facility included on the module.

Students’ perception of module

The student’s personal view of the delivery and assessment, whether they felt it help them progress etc was evaluated by a questionnaire. This very subjective and qualitative approach consisted of a combination of multiple choice and open-ended questions. The questionnaire was available online (anonymous) and as a hard copy so as not to introduce bias against those who had not used the WebCT element.

Compared with the responses obtained with hardcopy module evaluations in previous runs of this module, the students were much more vocal online. This might have been because the tutor was not present, but also as the release of results (on a WebCT e-mail page) was dependent upon completion of the evaluation. Results from the online module evaluation are shown in Figure 1. 

Figure 1 Student Perceptions of the Module

 

Two students gave an off-line evaluation, one of these objected to group work in his final year, saying that he did not like relying on others who were not as able as him. Although he could see that he would need group skills in a working situation, he felt he could not give or take instruction here ‘it’s different in a work situation where you know who’s boss’.

89% thought the assessment was appropriate; of those who did not their comments were regarding the proportion of marks allocated to group work (they wanted it increased). The final question on the module evaluation was open ended to allow students to add any extra comments; seventeen students typed in comments some were quite extensive. Several students commented on the development of a range of skills during the module: time management, presentation skills, oral and written communication, research and IT skills, teamwork, and social skills: ‘I also got to know the people on my course a little more’; ‘(we were) able to develop a good social working atmosphere...’; ‘...I have not appreciated group work included in past modules...’; ‘gave me more confidence...’; ‘I miss our group meetings...’. Overall the online aspect was successful from both the tutor and student perspective. The comments received from students were largely positive (26 positive points compared to 6 negative ones).

Successes and Challenges

Once WebCT tools are mastered then the preparation time is low, a selection of tips for development and incorporation of online group work are listed in Table 3.

Table 3 Tips for Online Group work

The online part of the Integrative Studies module allowed more effective use of time with students. There was also a notable reduction in the amount of paper generated during the module. The module acts as an excellent filing system for both staff and students, so if anyone (staff or student!) missed a session or misplaced a document, or didn’t make a note of something important, there was always a back-up. From the tutor’s perspective, students seemed a lot less anxious about the assessment and enjoyed the social aspect of learning in groups. It was also apparent that not all students want to use the Web-based learning aids, the students split into two groups - non-users and active users with very little in between.

Feedback

One of the great bonuses of the discussion facility was the increased capacity for giving feedback. By using the discussion facility the comments came from peers as well as staff. Detailed feedback was given frequently throughout the first part of the module; this complies with recommendations on assessment produced by the open learning foundation (Race, 2001).

This could be given on continuing work not just grades. In the past, a lot of time had been spent giving the same or similar advice to students individually. However, students like to have the feedback at their fingertips when they are actually doing the work and used to try to contact me for feedback, reassurance etc which was exhausting. Being able to reply to queries online reduced that time and also aided other students as they (if relevant) could view the same feedback. The feedback was there to help students who find it uncomfortable to approach a tutor for help. Over the first few weeks of this module, one group used the online discussion facility more than any other; however after the first couple of weeks it became obvious that the members of this Group were working individually and the discussion facility was used as a means of contact only. After tutor feedback, the students elected a leader and really utilised the web well.

The online feedback had the advantage that the tutor had a permanent copy (without copying students’ work) and it was much quicker than handwriting feedback. Although cutting and pasting can be used, the feedback can remain personal and individual; it is also much easier to read!

Use in Assessment

The assessment of group work using online discussions is very difficult without giving the students goals or tasks, but this itself may interrupt group work. The discussion facility, as used here, provided insight into how each group was working, that all members were happy and contributing. To assess this fairly and accurately the tutor would have to enforce an online meeting and/or give more tasks. It is difficult to devise tasks that actually encourage group work/ active learning; however, if the assessment aims are crystal clear this is made easier. When analysing messages it was easier and more precise to use an approach of asking questions regarding each message (Gunawardena et al, 1997; Mason, 1991; Newman et al, 1995) rather than trying to split the messages into units (Henri, 1992; Popping, 2000). Interactive, cognitive and meta-cognitive messages were less abundant which is in keeping with other studies (Lally, 2000). There was evidence of reflection and co-operation and student collaboration. There was evidence that they gained knowledge and help from each other, also there were a couple of examples of students providing material for others in their group and giving supportive feedback to other members of the group over their work. Within the minutes there was evidence of co-working and sharing tasks, however, it was difficult to judge whether students changed their understanding or constructed knowledge as a result of group interactions. From the module evaluation, students, by their own admission, changed their opinion to group work and how to go about it, which was also evident from the self-evaluation sheets.

In this study, no real conclusions could be gained by counting the number of messages as the extent of the use of the discussion facility was largely the student’s choice. At the very minimum students had to make 6 postings to fulfil their online tasks; all groups posted significantly more than this. Social messages increased over time as expected. Although the students had studied together for over 2 years it was apparent that new relationships were forged over the course of the module.

One of the negative aspects of trying to assess group interactions online is the tutor time involved in analysis. Here the groups did not confine their interactions to WebCT, yet one group managed to produce over 100 messages between them in a few weeks. Analysing all messages could be very labour intensive especially with large classes. For subsequent runs of the module the online tasks and evidence that groups are functioning appropriately were assessed by searching electronic interactions using a checklist of criteria.

Table 4 Summary of Value to Learning and Teaching

Advantages	Issues
·      Allows for rich, rapid feedback ·      Gives a wider range of feedback ·      Discourages lurkers and encourages contribution ·      Increases awareness of assessment criteria ·      Encourages reflection ·      Allows monitoring of progress with opportunity to intervene at an early stage ·      Having to make explicit their interactions, how they approached tasks, a log of their debates/discussions, improves the clarity, coherence and emphasises aims of their group work ·      Creation of an informal support network through social interactions	·      May be assessing the ability to evidence group work rather than ability to work as a group, although partly recompensed by peer assessment ·      Analysis of individual messages can be time consuming
In tandem with peer assessment:
·      In combination with peer assessment can be used to assess group work ·      Encourages critical evaluation and helps develop critical appraisal skills (Jaques, 2000)	·      Unless criteria are clear, students may not be assessing what the tutor wants

Although both students and staff were new to WebCT, there were some immediate benefits to both in the module developed here (the value to teaching and learning is summarised in Table 4). The biggest bonus with the asynchronous discussion was the increased ability to give feedback. The group work in this module was definitely stress-reduced and with some feasible alterations to the marking scheme and student instructions, including how to evidence meetings, at least some of the online discussions could be assessed.

Corresponding author: Dr Trudy Hartford, Leeds Metropolitan University, School of Health and Human Sciences, Calverley Street, Leeds LS1 3HE. Telephone: 0113 283 2600 ext 5975
Fax: 0113 283 3124 Email: t.hartford@leedsmet.ac.uk

References

Cheng, W. and Warren, M. (2000) Making a difference: using peers to assess individual students’ contributions to a group project. Teaching in Higher Education, 5 (2), 243-256

Conway, R., Kember, D., Sivan, A. and Wu, M. (1993) Peer Assessment of an individual’s contribution to a group project. Assessment and Evaluation in Higher Education, 18 (1), 45-54

Earl, S.E. (1986) Staff and peer assessment - measuring an individual’s contribution to group performance. Assessment and Evaluation in Higher Education, 11(1), 60-69

Entwhistle, N and Ramsden, P. (1983) Understanding Student Learning. London, UK: Croom Helm

Falchikov, N. (1986) Product Comparisons and process benefits of collaborative peer group and self-assessments. Assessment and Evaluation in Higher Education, 11 (2), 146-166

Garrison, D.R., Anderson, T. and Archer, W. (2000) Critical inquiry in a text based environment: Computer Conferencing in Higher Education. The Internet and Higher Education, 2 (2-3), 1-19

Gibbs, G., Habeshaw, S. and Habeshaw, T. (1986) 53 Interesting Ways to assess your Students. Bristol Technical and Educational Services.

Goldfinch, J. (1994) Further development in peer assessment of group projects. Assessment and Evaluation in Higher Education, 19 (1), 29-35

Goldfinch, J. and Raeside, R. (1990) Development of a Peer Assessment Technique for Obtaining Individual Marks on a Group project. Assessment and Evaluation in Higher Education, 15 (3), 210-231

Gunawardena, C.N., Lowe, N.A. and Anderson, T. (1997) Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing.  Journal of Educational Computing Research, 17 (4), 397-431  

Hara, N. and Kling, R.(2000) Students’ Distress with a Web-based Distance Education Course: An Ethnographic Study of Participants’ Experiences  http://www.slis.indiana.edu/CSI/WP/wp00-01.html (accessed 18 May  2004)

Henri, R. (1992) Computer Conferencing and Content analysis. In Collaborative Learning through Computer Conferencing, ed Kaye, A.R. p.117-136  Berlin, Germany: Springer-Verlag

Jaques, D. (2000) Learning in Groups, 3rd Edition. London, UK: Kogan Page

Lally, V. (2000) Analysing Teaching and Learning Interactions in a Networked Collaborative Learning Environment: issues and work in progress. Paper presented at the European Conference on Educational Research, Edinburgh. Available at http://www.leeds.ac.uk/educol/documents/00001648.htm(accessed 18 May  2004)

Lejk, M. and Wyvill, M. (1996) A Survey of methods of deriving individual grades from group assessments. Evaluation in Higher Education, 21 (3), 267-281

Li, L.K.Y. (2001) Some Refinements on Peer Assessment of Group Projects. Assessment and Evaluation in Higher Education, 26 (1), 5-18

Mason, R. (1991). Moderating Educational Computer Conferencing. The Distance Education Online Symposium News, 1 (19), available at http://www.emoderators.com/papers/mason.html  (accessed 18 May 2004)

Mason R. and Bacsich, P. (1998) Embedding Computer Conferencing into University Teaching. Computers in Education, 30, (3-4), 249-258

Newman, D.R., Webb, B. and Cochrane, C. (1995) A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology Journal, 3 (2), 56-77. Available online at http://www.helsinki.fi/science/optek/ (accessed 18 May 2004)

Newman, D.R. Johnson, C., Webb, B. and Cochrane, C. (1997) Evaluating the quality of learning in computer supported cooperative learning. Journal of the American Society for Information Science, 48 (6), 484-495. Available online at http://www.qub.ac.uk/mgt/papers/jasis/jasis.html (accessed 18 May 2004)

Orsmond, P., Merry, S. and Reiling, K. (1996) Importance of marking criteria in the use of peer assessment. Assessment and Evaluation in Higher Education, 21 (3), 239-250

Popping, R. (2000) Computer assisted text analysis. London,UK: Sage

Race, P. (2001) The Lecturer’s Toolkit A Practical Guide to Learning, Teaching and Assessment, second edition. London, UK: Kogan Page

Resnick, L., Levine, J. and Teasley, S. (eds) (1991) Perspectives on socially shared cognition (pp. 384-397). Washington,USA: American Psychological Association

Salmon, G. (2000) E-moderating: the key to teaching and learning online. London, UK: Kogan Page

Sockett, R.E. (1999) Learning about bacterial pathogens via a fictional bio-warfare situation. Society for General Microbiology Conference, Edinburgh

Stefani, L.A.J. (1994) Peer, Self and tutor assessment relative reliabilities. Assessment and Evaluation in Higher Education, 19 (1), 69-75

Vygotsky, L.S. (1978)  Mind in Society: The Development of Higher Psychological Processes. Cambridge, USA: Harvard University Press