Using on-line assessment is becoming more of a main-stream skill for
many bioscience academics. On-line assessment offers opportunities for
staff and students. Many of the issues and technologies were discussed
with respect to a number of typical products used currently in HE.
Following the recent publication and adoption of a number of standards
the on-line delivery of content and assessment is changing. Products are
becoming more interchangeable and robust, leading to more widespread adoption.
Academics preparing material may no longer have such steep learning curves
to climb or be tied to one particular product.
Around 20 people attended and enjoyed a very interesting day at the well-equipped
Netskills Centre. Participants had the opportunity to discuss pedagogical
and technical issues of on-line assessment as well as to try out a number
of example products. The example products were chosen to demonstrate different
aspects of content delivery and exchange.
Terry McAndrew, LTSN Bioscience
This icebreaker identified the issues and concerns relating to
on-line assessment. On-line assessment is gaining in popularity and the
experiences gained thus far proved useful as we looked at different approaches
during the day. Groups were asked to isolate and discuss issues.
View
the output from the discussion.
Terry McAndrew, LTSN Bioscience
Terry gave an overview of some of the recent developmental initiatives
and suggested how the new IMS standards may offer an opportunity to overcome
common obstacles (e.g. exchange of content between different projects).
Four different products, typical of the 'types' of assessment software
were introduced.
Richard Parsons, University of Dundee
Richard evaluated commercial solutions to deliver online assessment at
Dundee
This was followed by a hands-on session building MCQs using a commercial
tool; Questionmark Perception. This is an established commercial product
which has made significant steps towards adopting standards that allow
the exchange of assessment content between systems (- the IMS-led standards
for Question and Test Interoperability IMS QTI).
View
Richard's slides
Demonstration and further hands-on session using an alternative commercial
product (Canvas Learning Author).
This is new product developed from an internal development tool. It is
specifically targeted at producing content in a 'universal' IMS QTI format
and so can be rendered in a number of systems in addition to its own delivery
mechanism.
The session gave delegates the opportunity to use IMS QTI to format questions
(sourced from a Bioscience questionbank) in both Canvas Learning Author
and Questionmark.
Lunch.
EQL co-sponsored the event but technical difficulties prevented the installation
of the software in the PC cluster. Fortunately, two of the developers
were able to squeeze a brief demonstration at the end of the lunch break.
EQLs i-Assess product plans to
be IMS QTI compliant in the spring of 2003.
Richard Rayne, Birkbeck College
Richard gave a presentation on how on-line assessment has been introduced
to, and used by, mature part-time students at Birkbeck College. The assessment
was developed using TRIADS
- a HE developed solution. The presentation was followed by a hands-on
session with QuickTri - similar software developed by the same group.
Richard is also a co-holder of an FDTL phase 4 grant for the OLAAF
project. OLAAF's primary aim is to develop and disseminate generic
guidelines for the construction of Computer Based Assessment and Feedback
(CBAF).
View
Richard's slides
.
Richard Parsons, University of Dundee
Richard presented a solution to the peer-marking of student work on-line
with his system for the assessment of textual answers. The system allows
students to assess their own and each others work on-line in a robust
and consistent manner before generating an 'immediate response'. These
responses can be sent directly to a mobile phone. The quality control
and benefits of this type of marking were discussed. Once again a hands-on
session followed the presentation. Richard's system proved very popular
with attendees and is available for demonstration/ testing by bioscience
tutors in other institutions.
View
Richard's slides
Visit
the website for the Self and Peer Assessment Tool
Graham Smith, University of Leeds
Graham offered a view from the perspective of an educational software
consultant on the problems of inconsistent adoption of a standard for
exchange of content. Many product vendors are interpreting the IMS QTI
standard for assessment content in a flexible manner - generally one that
suits the specific needs of the product vendor. Concerns are being expressed
about this as content is likely to move between delivery systems. Graham
also demonstrated a facility for the web-based delivery of on-line assessment
developed at the University of Leeds. This solution seeks to adopt the
IMS QTI standard in a robust manner. The advantages of using this as a
test site for IMS QTI conformance were discussed. Again a hands-on session
followed allowing delegates to test the portability of CAA content on
a web-based test system.
Terry McAndrew, LTSN Bioscience
Terry demonstrated the use of a 'Content Packaging' tool to combine materials
for subsequent re-presentation in alternative systems. A common problem
is the uploading and management of a collection of learning resources
(video, slides, notes, web pages, assessment etc.) in a simple transparent
fashion without having to have use specialist web-authoring skills. The
problem is universal in nature and another standard (IMS
standard -Content Packaging) is being used to separate the 'logic
of collection' from the 'presentation or resources'. The example used
was from the Microsoft LRN
content packaging tool but other notable
alternatives are being developed in UK higher education. Terry would
like to express thanks to Boon Low for additional materials regarding
the use of XML for content packaging.
A discussion of the days events followed.
Aspects of the software solutions were discussed with respect to the initial
S.W.O.T. analysis undertaken at the start of the day. It was agreed
that many of the concerns have been, or could be overcome with careful
consideration and selection of the most appropriate tool. The standardisation
of the logic of online assessment was welcomed and it was agreed that
it was necessary for teaching staff to be aware of these, without having
to have deep technical knowledge of the underlying XML based mechanisms.
A number of alternatives were also suggested. The value of a subject based
questionbank, issues of the quality
management of delivery, content, and acknowledgement for authorship
were discussed.
Note: During the day it was not possible to schedule
time to try WebOL-lite, a tool for the production of assessment for on-line
delivery. WebOL-lite was being developed to produce QTI standard output
at the time of the presentation. This has now been updated. Further
details about this software, free for use in UK Higher Education, are
available.
