The “sham test” controversy – a wasted opportunity?

Psychometric testing is in the news today – and not in a good way. According to a report in the Guardian, the Department of Work and Pensions is forcing unemployed people to take “bogus” psychometric tests, and threatening to take their benefits away if they refuse to do so. The report, and the accompanying video, go on to say that the test is a “sham”, and will spit out the same results no matter what answers are given. Some websites and blogs go further, saying for example that “this ‘test’ is a tool for abuse and psychological torture”.

So, a fair cop? Well, perhaps not quite. Looking at the questionnaire and the report that is produced (try it for yourself here), it feels to me as if someone has genuinely tried to put something together that will help people. You fill in a few short questions about yourself and then you get a short report that tells you how to put your best foot forward in applying for jobs – possibly making you think about strengths that you didn’t realise you possessed. Unfortunately, the way in which this idea has been implemented is not perfect:

• The questions are a little unusual in places, for example “In the last month, I have been thrilled by excellence in music, art, drama, film, sport, science or mathematics”, or “I have not created anything of beauty in the last year”. Some may not be easy to understand for people with lower levels of literacy or who do not have English as a first language (“I hesitate to sacrifice my self-interest for the benefit of groups I am in”). This may be an issue given the people the questionnaire is aimed at. There is also at least one grammatical error (“I always look on bright side”). None of these are fatal, but without some additional explanation (eg “some of these questions might seem a little unusual, but...”), people will not take the questionnaire seriously. In the jargon of testing, it will have low face validity.
• One criticism has been that people will get the same report no matter what answers are put down. This is not actually the case – you will get a different report if you answer the questions in a different way; although this is not obvious, as the reports are very similar. 
• The reports themselves list your five key strengths, so they give a very positive message – but it isn’t immediately obvious that this is what the report is doing.

The biggest issue, however, appears to be how the questionnaire may have been used. Forcing people to complete an open personality questionnaire against their will (if this is indeed what has happened) is unlikely to have a useful outcome. Doing so on a nationwide programme in the full glare of publicity could be seen as asking for trouble. This is a real shame, because we know that accurate personality questionnaires, with focused reports, can be incredibly useful in career counselling.

Any personality questionnaire is a means to an end; how it is used is just as important. Unfortunately, the world seems to have decided that the DWP questionnaire has been designed from the outset to be used in a less than positive way. Any questions such as “is it reliable?”, “is it valid?” or even “could it be useful?” are going to be drowned out. Let’s hope that this does not detract from the many places where psychometric tools are used in a positive way to develop individuals and to widen and inform their career choices.