There are two styles of testing in corporate and academic assessments:
- Subjective / Descriptive: The test has a question and the test taker – “assessee” in Learning & Assessment Management System-speak – writes their answer on a piece of paper in the case of offline assessment or types their answer into a form in the case of online assessment. In either case, the answer is open-ended.
- Objective / MCQ: The test has a question followed by multiple choices aka Multiple Choice Question. Typically, an MCQ has five choices, labeled from A through E, with E typically being “None of the above”. The assessee selects one of the five choices by blackening the corresponding oval on the answer sheet with a pencil / pen in the case of offline assessment or by clicking the radio button beside the corresponding choice on a screen in the case of online assessment. (Some MCQs permit more than one answer. But, that’s a detail we can ignore in the context of this blog post).
Both assessment styles have their own pros and cons in terms of intrinsic fitment-for-purpose and administrative considerations.
How do we know this?
It so happens that “Learning & Assessment Management System” has been a hot domain for our outsourced product management and business development practices. Click here, here, and here for a few of our case studies in the LAMS space.
LMS will sell better if it focused on improving learning outcomes instead of exhorting people to learn more & more.
https://t.co/fvhb8NcmCc— Ketharaman Swaminathan (@s_ketharaman) October 11, 2016
Companies and academic institutions regularly need to test certain skills when it’s not self-evident which style of assessment they should use.
This debate came to the fore in a recent interview conducted by Times of India with Prof. Sarit Kumar Das, Director of IIT Ropar.
Prof. Das bemoans the drop in standard of IITians and holds the objective nature of JEE responsible for this problem.
For the uninitiated, JEE or “Joint Entrance Exam” is the common test for admission to the prestigious Indian Institutes of Technology. Conducted once a year all over India – and in a few countries outside India with Indian diaspora – JEE originally used the subjective / descriptive style but switched to the objective / MCQ format around 20 years ago.
According to Director Das, MCQ lets students crack the right answer purely by guesswork and does not test problem-solving capability.
This is hardly a new critique of MCQ.
During my days at IIT Bombay in the early eighties before JEE had switched to MCQ format, I remember reading an article about SAT. For the uninitiated, the American Scholastic Aptitude Test used MCQ even back in that era (probably it did right since inception). The authors of the piece claimed that SAT could be easily gamed. According to their exposé, the sequence of correct answers to SAT’s multiple choice questions followed a few preset patterns that could be cracked by the “seed value” i.e. the answer to the first question. Published in a reputed journal (Scientific American?), the article left such a strong impression on my mind that I remember its title 35 years later: “E. None of the above”.
Notwithstanding this exposé, SAT has continued to use MCQ. The same format has subsequently spread like wild fire and is now de rigueur in GRE, GMAT, JEE, CAT and dozens of other competitive tests worldwide. The sky hasn’t fallen.
It’s tempting to attribute the popularity of MCQ entirely to its administrative benefits such as higher speed and lower cost.
But that’d unfairly be selling it short. IMO, MCQ’s ubiquity must mean that it’s basically fit-for-purpose.
Prof. Das doesn’t seem to buy this argument. In his interview, he actually advocates going back to the old subjective / descriptive system (although he fears a legal backlash against the switch).
I wonder about this and wanted to solicit your views.
Hence this post. While it’s triggered by an academic test, it’s equally well relevant for corporate assessment scenarios around technical, presentation and other skills tested regularly in the workplace.
While “E. None of the above” is not one of the choices, you may feel free to weigh in with other alternatives.
PS: See what I did there? To expand an old Oscar Wilde quip, the best way to resist the temptation of featuring an MCQ in a blog post about MCQ is to succumb to it!
