Test design - meet the professionals!

Originally published to eBridge on Friday 16 November 2012

My anti-bribery e-learning has now been in play for two full weeks, and there's a mixture of good and bad in the feedback. The biggest positive is that well over 400 people have completed every part of the e-learning, with a fairly positive reception to some of the more experimental aspects, such as the decision making scenarios. But...

Working for an organisation that designs assessments has shown that people have some very rigid ideas about what constitutes a valid test. In particular, one colleague in our research department seemed to expect that the questions should conform to the standard knowledge tests designed to discriminate between different abilities at GCSE. Others have attacked the complexity of the wording, and the variation in how questions are to be answered, for instance having to decide how many and which statements are correct from a set of judgements about scenarios. There are some points that I will take on board for future - perhaps having questions alternate between spotting statements which are either true or false was a step too far!

However I'm definitely not alone in thinking that the test should be complex, after all it is designed to protect the pupils who take our tests from bribery, fraud and corruption, so a test that's too easy and doesn't prompt recall later on is a dangerous falsehood. My reasoning behind the complexity of the questions was inspired by an article by Dror (2006) where he suggested that having learners make judgements about the information would encourage better recall in future. The reason for my being so passionate about this topic is having read accounts by Bennett (2001) and Chapman (2002) of how bad corruption in education can be, I'm determined that we need to take an attitude of 'the buck stops here' rather than doing the minimum requirement to meet our legal obligations. I also have a deeper, more subtle motivation for taking active ownership of this policy - Ronfeldt (1996, p.17) lists corruption as the key risk associated with hierarchical organisations (i.e. governments). With the government tightening its grip on the education system, I must ask the eternal question: 'Who watches the watchers?'

References:

• Bennett, N. (2001). Corruption in education systems in developing countries: What is it doing to the Young? Available at: <http://www.10iacc.org/content.phtml?documents=112&art=126> [Accessed November 2012]
• Chapman, D. (2002). Corruption and the education sector. USAid / Management Systems International. Available at: [Accessed November 2012]
• Dror, I., 2006. Shall I remember? How learning technologies should facilitate, but too often hinder, memory. A paper for Learning Light. Available at: <http://www.e-learningcentre.co.uk/Resource/CMS/Assets/5c10130e-6a9f-102c-a0be-003005bbceb4/form_uploads/Shall_I_Remember_Dror.pdf> [Accessed November 2012]
• Ronfeldt, D. (1996). Tribes, Institutions, Markets, Networks: A framework about societal evolution.  RAND Corporation, Santa Monica. Available at: > [Accessed November 2012]http://www.e-learningcentre.co.uk/Resource/CMS/Assets/5c10130e-6a9f-102c-a0be-003005bbceb4/form_uploads/Shall_I_Remember_Dror.pdf> [Accessed November 2012]