True or False

One of the most important aspects of formative assessment is the feedback given to learners that allows them to advance their knowledge or skills. Dylan Wiliam argues that without this mechanism, assessment can’t be considered to be ‘formative’,

“Assessment is formative only if the information fed back to the learner is used by the learner in making improvements”

The key in that sentence for me is the bit about ‘only if… used by the learner’. I’ve spent a lot of time in the past writing detailed feedback on assignments only to find students scanning the page for the grade and ignoring the additional commentary. I know this to be true, because I used to ask the same students about something I’d written and be faced with blank stares.

Sigh.

That’s why I was quite taken by the JISC- funded project, ‘Sounds Good: Quicker, better assessment using audio feedback‘. Although as a language teacher I’d used audio feedback before (recording comments onto cassette tape in a language lab), I hadn’t come across the practice in other disciplines. The initial findings from the project seem to show that students are genuinely engaged with this style of feedback, considering it to be more personalised and detailed than the traditional written forms.

The more I think about it, with open source applications like Audacity, recording feedback should be a breeze. I’m conscious that not everyone is keen to record themseleves speaking - it’s something that takes a bit of practice to get right (you’ll find some useful tips in the initial findings linked above). If you do decide to give it a go, then please let me know how you get on!

(Oh, I noticed the project has two sites - the older site has a few interesting documents not available on the newer site that are worth checking out)

Wow, has it really been 4 and half months since the last post? Where has all of the time gone? And why is my local supermarket selling Xmas decorations ? Surely, it can’t be that time of year already…

Ah well, I’ll try and make for my absence with some interesting bits and pieces that I’ve across:

Launched at ALT-C in September 2008, Effective Practice with e-Portfolios (PDF, 781kB) draws out key points from established and innovative examples of practice in further and higher education and from a selection of significant JISC-funded projects on e-portfolios.

The publication complements an infoKit on e-portfolios which synthesises the main outcomes from JISC-funded projects in this field and covers the main drivers, purposes, processes, perspectives and issues around e-portfolios. Audiences for both resources will be those whose roles might involve use or implementation of e-portfolios as a tool for learning: learners, practitioners and managers in FE and HE, including IT and network staff; professional bodies; those involved in initial teacher training and in the management and implementation of PDP practice.

Now I have to admit that I don’t actually own a mobile phone, though I do occasionally borrow my wife’s (much to her annoyance). It’s amazing what you can now do on such a device - even those that aren’t ‘smart‘.

Mobile Study off you the ability to create quizzes (with images) that can be downloaded onto most modern mobile phones either directly or via a computer. These quizzes then run as standalone java applications. For the techies out there, if your phone conforms to the MIDP 1.0 specification (most of them), then all should be well.

There’s also a Moodle plugin that allows you to export quizzes created in the VLE to your students’ phones. The scores of these quizzes aren’t recorded in Moodle, this shouldn’t be a problem for most.

Multiple choice questions are supported by feedback, though this is limited to text (without hyperlinks).

It’s been my experience that students respond well to these simple short quizzes, especially if they are delivered in a format that allows you to try them ‘anytime, anywhere’.

We’d love to hear from any of you that have tried to use mobile quizzes with your students. Tell us what you think of the format and how well your students have responded by leaving a comment below!

The JISC-funded WebPA project has won a Bronze at the IMS 2008 Learning Impact Awards in Austin Texas. The open source online peer assessment system was the only UK entry in the field of twenty-three nominees from across the world.

I recently attended a presentation by Nicola Wilkinson from the WebPA development team and was impressed by the simple but effective model that underlies the system.

Students assign themselves and members of their group grades according to a defined set of criteria: e.g. ‘Participation’, ‘Attendance of group meetings’, etc. The lecturer then awards the project an overall grade, which the system adjusts for individual scores according to the input of the students.

It’s a neat uncomplicated solution that allows students to have a say in the assessment process, especially the part that isn’t always visible to the lecturer.

Download the software here.

The NHS e-Portfolio was a finalist in the 2007 e-Government National Awards in the category of ‘Excellence in Learning & Skills’ (though the prize finally went to Gloucestershire College for their StARs & StRIPEs system).

It’s currently being used by over 16,000 doctors in training across the UK as well as 700 dentists and over 100 pharmacists in Scotland. Activity on the system remains high, with 81,000 clinical (skills) assessments, 55,000 multi-source feedback assessments and 130,000 educational log entries in the system to date. Summative assessments submitted by other healthcare professionals to demonstrate competency of individual doctors account for 60% of these entries.

August 2008 will see the launch of the second version of the portfolio. Select this link for a sneak preview of e-Portfolio 2.0 and more information on the project. If you’d like to compare it with the current system, then demo logins for the original portfolio are also available.

I was looking around the Microsoft website for a solution to a problem I was having with Word (a ruler line that refused to be deleted no matter what I tried!). After ten minutes of fruitless searching, my mind (and cursor) started to wander and I came across a section on Office Quizzes.

Obviously, these only cover Microsoft products, but could be useful to students to want to quickly revise their understanding of particular applications, or as part of some additional ECDL/PC Passport classes?

Do you know of any other useful assessment resources on the web? If so, please add a comment and share the knowledge!

As I was trawling through the net, I came across this concept map on the University of Bath site describing some of the key issues surrounding e-Assessment. It’s somewhat dated, and the embedded links don’t really lead anywhere, but it’s still a neat representation. If you happen to know of something better out there, please let me know!

BTL has completed an 18-month project to convert 13 National Qualification English for Speakers of Other Languages (ESOL) units into interactive formats for the Scottish Qualifications Authority (SQA). The 13 units will be created as a SCORM package and also made available on CD.

Follow this link to find out more about the SQA’s e-Learning Support CD for ESOL, available to centres from the week beginning 11th February.

Now, strictly speaking, this isn’t really a post about e-Assessment software; it probably has more to do with my fascination for (serious or otherwise) computer games. However, as I can’t quite justify writing about my latest high score in Wii Sports, this will have to do.

Coffee Shop is an online version of the Lemonade Stand game that used to come bundled with old Macs. It’s a simple premise, as the owner of a coffee shop, you’re responsible for purchasing supplies, determining the recipe and setting the price of your coffee. Various additional factors will influence how well your coffee sells - reputation, the weather and perhaps most importantly, the taste!

Once you have set all the variables, the first day commences (you have 2 weeks to maximise your profit). Customers come and go, giving feedback on your performance. You can keep an eye on your inventory and can adjust the price of your coffee. All going well, at the end of the day, you should be showing a handsome profit and a boost to your reputation.

You can monitor your progress through a serious of graphs that display detailed information about your activities. There are just enough variables involved to make this a challenging, but at the same time engaging activity for students.

So how can this be used as part of an e-Assessment? Well, if you were introducing your students to some basic economic principles, then Coffee Shops allows them to apply this knowledge and gain feedback on their understanding based on performance.

There are also opportunities for groupwork, with students devising a strategy for each day together. The assessment of their performance could be conducted as part of class discussion - allowing students to become more involved with the analysis of their achievements.

If you do decide to try this (or something similar) out with your students, then I’d be delighted to hear about it - please post your comments below! Now, let me have just one more try at making that perfect cup of coffee!

Multiple choice questions (MCQs) form the bread and butter of e-Assessment, though not everyone agrees with their widespread use, arguing that they promote memorisation, rather than actual learning. Nevertheless, more and more students find that MCQs are becoming a familiar part of their assessment experience.

Unfortunately, they don’t always come across as being the most stimulating of assessment formats. The thrill of making a choice between A, B, C or ‘none of the above’ is generally quite muted. So it comes as a relief to discover more innovative or entertaining versions making their mark in education.

Confidence-based marking (CBM) is great way of engaging your students, but perhaps the most successful method is to add an element of ‘edutainment’. An article in NewScientist highlighted the work of Tzu-Hua Wang at the National Hsinchu University of Education in Taiwan. He developed an online quiz that borrows liberally from the popular TV show ‘Who Wants to be a Millionaire’ and discovered that students were (unsurprisingly) more keen to be tested using the system and that (more interestingly) they seemed to have higher comprehension levels after using it!

Perhaps it’s time to take another look at the faithful old MCQ?