What triggered this post was this story about an SSLC student who obtained the first rank after re-evualuation by beating the previous first ranker by one mark. The episode created enough flutter for it to be featured on the front page of the daily newspaper and for some readers to retort,
It so happens that the re-evaluation changed marks in subjects like English and Kannada. Now I wonder, how one could prove that the correct marks for an essay answer was not 8 marks, but 9 marks..
Our obsession with ranks has reached a level where it is hardly distinguishable from rank idiocy. In this process, we are paying a heavy price by wrecking the mental peace and happiness of the next generation and we are blissfully unaware of it or choose to ignore it altogether in our quest for social glory.
Education, intelligence, skill, wisdom, etc. are such intractable, immensely complex entities; yet we find it convenient to project them all onto a single, ordinal (not even categorical) dimension for evaluation.
It is somewhat like explaining how an elephant looks like, by looking at its shadow on a single line (not even on a plane).
If a student has obtained 99% in Mathematics, does it mean that the student knows 99% of all there is to know in Mathematics? Or, if one student has scored 99 and another has scored 98, does it mean that the former has one unit of knowledge greater than the latter?
It means nothing. And yet, this number is horrific enough to wreck the mental peace of several students -- even leading them to commit suicide on the one hand or to get an inordinately high element of self worth on the other.
To explain the adverse effects of a single-dimension ranking system, even on the rank holders, here is a small anecdote:
Last week I'd met some people in Sweden who have started their own company based on an innovative product. They were explaining to me how the first version of the product were developed by some IIT graduates. The graduates had marketed themselves into this job by focusing on their high all-India ranks that they'd got in JEE. Basically, they sent out the message that they are genuises and if others didn't understand them, well it was just too bad. So these guys were mighty impressed and hired them to develop the product. And of course, the end product was lousy. The application was bloated, it did not scale, it was not documented properly, etc. So much so that the entrepreneurs decided to tear down then entire codebase and re-implement their product from scratch. I was just thinking, "Why am I not surprised.." Heh.
Some years ago one of my cousins passes the CET with a good rank and promptly all relatives are asking me how much I had scored when I had taken that exam. People, I had written my CET almost 20 years ago!! These things never have an expiry date, is it? Apparently these exams -- the three hours that you spend -- brands you for the rest of your life.
There is also this story of a couple of friends of mine who were extremely talented in two different aspects of software development. One was very good in developing efficient algorithms, while the other was very good at understanding the user experience and develop good user interfaces. But they never got together to team up and start a company or pursue research together or anything like that. It so happened that one had got a somewhat lower GATE percentile than the other and there was this perinneal rivalry and complexes between them.
There are also cases of some young cousins of mine who are very good at skills like sketching, art, design, etc. But they all end up with very low self-esteem because they did not do well enough to impress people around them in their engineering entrance exams, etc.
The amount of damage we are causing by this mind-numbing single-dimensional evaluation system is phenomenal. Far from teaching problem-solving skills that are necessary for leading a life, we are just training our students to be performance artists. They are somewhat like the gladiators of the ancient Roman empire -- fiercely competitive fighters who fought and killed one another -- just for the recognition and glory they obtained from the emperor.
Anyway, rather than ranting away, here are some concrete suggestions:
- First develop a faceted evaluation system. There are several dimensions to learning a subject and keep those dimensions separate. Don't add them up. For instance, when evaluating an engineering subject, consider dimensions like the student's comprehension of the concepts, his/her ability to apply insight to real-life situations, his/her comfort level in extending the body of knowledge, his/her ability to analyze any system based on this subject matter, etc.
- Encourage the formation of teams and group activities. Have an element like a group project that is considered for final evaluation. And don't just evaluate the final project report. Instill skills in students to document their group activity right from its beginning and evaluate the project not just on the end result, but also on the way it was approached. For instance, get the students to make a movie about the project and the way they got it done. This makes it difficult for them to "outsource" the project to the friendly neighbourhood computer center. Even if someone else makes the movie for them, they will have to play-act the project plan -- and get to learn something in this process.
- Encourage peer review. Instill this culture right from primary school or high school. A lot of learning happens by evaluating other projects and others' understanding than by our own learning. Much of professional life requires peer review skills.
- And finally, for God's sake please don't combine all of these metrics to a single rank. Let the report cards contain multiple dimensions and a histogram of the student's performance along all these dimensions. Better still have some dimensions that have categorical values (something which cannot be placed in an order, like numbers).
Develop university programs that can enable each student to focus on what they are good at, all the while having a reasonable knowledge of the other dimensions. We need to develop experts who can collaborate with one another and deliver sound solutions. We should not be creating jack-of-all air-bags or gladiators or mercenaries.
14 comments:
Good points. Others have said (or still say) what you are advocating here. The problem is, there is too much to change and most of these changes are institutional.
To modify the grade system is to meddle with many interrelated organizations: the school, the other professors, the students (many that prefer the one dimension grades), companies, government, etc.
To exemplify, Brazilian government did attempt to introduce a way of measurement more or less in the way you suggested. They called it "competences" the student had to acquire in order to pass a course. These weren't combined in a single rank, so for each course the student had a list of competences acquired.
These competences were very abstract units that were thought to represent the students' acquired knowledge. For example, in a basic programming language course, these could be "identify, understand and write conditional structures", "write modular code", "construct efficient recursive functions", etc. For each competence, it was expected different kinds of activities, projects and exams. Competences could be interrelated, or be a prerequisite for another, setting a broad course structure for all professors to follow.
I'm sure you can think of many problems with that construct. For example, how do you measure one's ability to "identify, understand and write" something? What distinguishes someone capable from someone who is not? And how do you acknowledge an outstanding performance in order to keep these students motivated?
Also, professors tended to think in terms of quantifiable measurements for the competences, so some schools started to include grades (e.g., poor, good, excellent) for each competence. You can't simply change a system by changing the rules. The older system is established in many other informal norms and conventions which are difficult to change.
Finally, the same apply for other sectors of society. Companies wanted grades in order to hire students with the best scores (that became a problem for colleges with the competence system in Brazil because companies started to hire only from those which weren't using that system). Masters and doctoral programs also wanted grades and even the students were pressuring in that way.
Granted that some professors supported the idea of competences (and I like it too) but I also must admit that we evolved into a society where everything has to be measured somehow. Unfortunately, changes in that may never happen. It might be the case that the very obsession you are against in your post, will just get reinforced as more and more people are adopting it.
Good points.
Sadly, this "rank idiocy" looks like it has only just begun this year. I was just browsing through the morning's paper, and was shocked to see another front-page report about a revaluation elevating a student to joint first rank. So, now we have two such cases.
Like you said, it is extremely hard to objectively say how many marks one deserves for an answer (especially in languages, where "correctness" is largely subjective). This student had earlier secured -- ahem -- 93% in English and 97% in Kannada.
Apparently, she "conceded that 93 and 95 were great scores but she wanted a revaluation as she was convinced that her answers were correct."
She did apply for revaluation eventually, and got the joint first rank. Apparently, her parents had tried to convince her that there was no need to apply for revaluation, but she had insisted that there was.
Now, this is scary. The student thinks her answers are "correct" and that she should get more marks. In Kannada and English, for crying out loud! (And worse, perhaps she even feels vindicated now.) This perhaps goes to show that the system encourages the "what's mugged up is right" phenomenon. The whole system of evaluation actually encourages students to bother more about the evaluation itself rather than learning.
Oops! Sorry, I forgot to credit the source of that quote.
Another Mysore girl climbs to the top, The Hindu, Sunday, May 31, 2009.
I had to go through similar ignominies during both SSLC & CET, thanks to comparisons between relatives & friends. Was lucky to remain sane enough thanks to some advice from close quarters.
I thought the rank system was scrapped a while ago. Maybe that was for PUC...
@ricardo Thanks for the reality check! ;-)
The reason why quantitative metrics are always preferred is that it is scalable over a large population and can be easily compared.
But I am not convinced that given the large population, the present set up of board exams and marks is the best we can do. There is a copious lack of any substantive research papers or articles exploring better effective and scalable methods for evaluation.
I know. But I think the lack of research could be simply because we can't find a better solution yet. It's not that there isn't enough research, it's just that they aren't being translated in successful results.
My Brazilian example was to make that point. The government initiative was backed by research and pedagogical studies. At the time I was working at a software company providing solutions for these schools (all of them very prestigious public technical schools and colleges in Brazil) and spoke to some qualified professors and researchers on the topic.
They had divergent opinions regarding the government's approach but consent that there is yet to emerge a better alternative when it comes to a large, scalable evaluation system. I would argue that a switch from quantitative to more qualitative measurements implies higher and undesirable transaction costs to organizations (as I put in my previous organizations). That may be shortsighted (these transactional costs may be higher only in the short term) but how to convince most people in most societies of that?
To be perfectly clear, I'm agreeing with your premises but criticizing your solutions in terms of measurements (not the practices professors should adopt). ;)
hehe ;) Thanks for the criticisms, and do give some more ;)
Yeah, implementing any model that is even slightly more complex than the single-dimensional one that we have right now, is a lot of work. And yes, there definitely is a lot of research in pedagogy about evaluation mechanisms; but what I find copious by their absence are some really innovative ideas.
I'm thinking in terms of using modern day web search, knowledge retrieval and social network analysis techniques to find good evaluation models. Something where computer-aided analysis is an intrinsic part of the process. Without which, there is no way we can do anything more sophisticated and scalable than what we have now.
:)
Ok, I agree. But for the sake of the exercise, lets assume there is such a computer-aided model. Wouldn't it, after all, be resumed to a final, comparable number that most people will demand for?
Perhaps is a better indicator but still a one-dimensional representation of a more wider analysis aided by a computer. The problem, I think, is with the "rank" institution. People expect to be ranked and strive to be better than others in such terms.
Er no..
I was advocating the use of computer-aided models to generate the histogram itself -- and leave it at that! And resist the temptation to add up all the numbers and give one grand number.. ;)
But yes, it will take a while for a new model to be accepted. Even more difficult is the changes required in the rest of the ecosystem, as you have mentioned, companies, universities, etc. fixing their systems to understand histograms than single numbers.
"And resist the temptation to add up all the numbers and give one grand number.."
That I think is harder than finding a better alternative solution. hehe
Thanks for the responses, by the way! :)
I thought I was the only one surprised that a well respected newspaper like The Hindu would print such a news on the front page. This is blatant pandering to the readers' growing thirst for sensational drivel.
What more could I add to what you have said about our grand educational system! In one of your posts you mentioned about a 60-year old Swede building his own house from scratch. Made me wonder how many of us could do that even if we had the free time. Our schools don't have a course involving woodwork, whereas western schools do. I find it extremely shameful that when needed most of us can't even change a washer in a leaking tap at our homes.
The less I say about the practical use of "education" imparted in our schools the better. I can ramble on and on but I'll pass :)
As researchers have analysed there are different types of intelligence it should have become unimportant by now how much one scores in a particular field , a person may have the least score in one field but maybe the first one in another . The problem arises only because all these intelligence are not given equal importance , the so called 'software' engineers have the world at their beck and call whereas the lesser ranks are distributed to other streams which do not fetch enough name and fame in their family circle . The "rank" is hence of utmost importance , recently a student scored not so well in his IIT whereas he scored a lovely 70 something in CET but the situation was sombre as he did not even consider other avenues as vistas of education , he rather wanted to redo his IIT exam the next year .This is the trend that is catching on in the name of higher education , narrow perspective of the so called 'broadening of knowledge'.
Post a Comment