Search This Blog

Thursday, April 20, 2017

Merit-based admissions

People who do well in JEE Advanced will mostly tell you that merit is a one-dimensional beast, never mind that it consists of performance in three heterogeneous disciplines, mapped into a single number by the simple addition of three numbers without checking if these numbers are comparable or represent something which can be simply added. It is further believed that admission to higher education is a reward for performing well in an exam under the conditions of extreme pressures. And hence the only justified admission policy is to admit students in the decreasing order of merit or performance in that exam (modulo any laws passed by the parliament, but of course, not before they have pointed out that merit has been violated by those laws).

Should a university consider admission as a reward for performance. Or should it be something else.

Well, it has to be something else. In fact, no university will couch its admission policy in terms of rewarding past performance. University goals are varied: Who is likely to do well in our programs? or Who is likely to do well even beyond our program and be successful and enhance our reputation?

If university goals are somewhat in line with what I have mentioned above, then it is obvious that the admission through a single test is not ideal. No one can perhaps argue that performing in a particular test can both be an indicator of future performance in Computer Science, as well as an indicator of future performance in Economics. No one can argue that India has the best admission process since this is the only major country in the world that does not take language abilities into account for admission. No one can argue that the ability to answer MCQs is a strong indicator of the ability to write long notes necessary in any studies.

The reason for following a single exam is that it provides an objective, transparent mechanism while simultaneously measuring one variant of irrelevant merit. In other words, the most important outcome of JEE based admission has been that we have kept out pressure from the rich and influential as well as won most court cases. And by no means, this is a small achievement. In India, one may genuinely consider keeping pressure out and winning court cases as more important property of admission process than getting students aligned with the university goals. However, over a period of time, people doing well in JEE have started believing that the prime purpose of JEE is to evaluate merit, and that there can only be one definition of merit, which is the JEE rank.

Besides the obvious inadequacy of a single number predicting success of all programs in all disciplines in all institutes, there are other problems as well.

If a university asks the question who is likely to do well in our programs, and let us assume (despite absence of any data) that performance in PCM has a strong positive correlation with performance in Mechanical Engineering. What if we were compare the performance of people who have 80% in JEE after intense coaching, whose parents were rich and provided a good study environment whether at home or in a hostel, with the performance of people who have 75% in JEE without any coaching, and who come from modest backgrounds, and are the first generation learners. When both these groups compete, who is likely to do better. Most universities would have data to show that latter group performs better, and hence they would actually create admission policies to offer more admissions to the latter group.

When we point this out, an immediate reaction is, but how do you evaluate deprivation, and what has been the impact of that deprivation, and how much extra "credit" should be awarded to compensate for that deprivation so that we can compare the merit of the two. Now, in this question, there is an inherent belief that JEE score is a strong indicator of future performance of every sort. Gold standard of transparency and honesty (?) has been converted to Gold standard of merit. Do you ever question why people with same total in JEE score are ranked differently based on marks in some subject or the other. Does that imply different merit. Isn't it true that luck plays a part. If you had a headache that day, your merit may be way down. That any exam performance is only an indicator within a significant band is conveniently forgotten. But when it comes to any affirmative action, we want exact data, why 5% and not 4%. Why not ask data about JEE also.

Let us do another thought experiment. Again, let us assume that PCM performance is an indicator of future performance in all programs. Consider two possible admission decisions. One, take the top 100. Two, take the top 90. Study whether there is sufficient diversity in the class. If certain backgrounds are missing, let us fill the last 10 seats through them. Now, there is a belief (and educationists may even have data, I am not one) that diversity is good for education. That diverse inputs will cause more innovative projects. That having diversity in your peer group in college will prepare you for a much more diverse workplace that you are likely to face. So both performance within the university and success beyond the university is likely to be enhanced if the class is more diverse. Because of these reasons, universities encourage student exchange programs besides having diversified student admissions to begin with.

Let us for the sake of argument assume that there is data to show that diversity helps. What should a university do? Should you admit 100 students whose average performance will be 60, or should you admit 90 students whose average performance will be 70 (helped by diversity) and 10 students whose performance would perhaps be only 50. There is certainly a plausible argument for not admitting strictly through merit (if at all merit can be determined by a single number). But ask someone who was 91st in our imaginary list. Will he agree. If you assume that admission is a reward for performance in a particular test, you will not like what the university is doing.

Universities may have other goals. For example, should we admit those who are the smartest and will benefit very little from university education, or should we admit those who are likely to gain a lot more from university education. So on a scale from 1 to 10, should we admit students at 9 who can be helped to reach 9.5, or should we admit students at 7 and take them to 8.5. In other words, should "merit" be the only criteria for admission or should we also look at what will benefit the society most. This question is particularly important when the society is funding the cost of that education. Again, I know what the "merit" crowd will say, but I will only suggest that most universities around the world will admit both kinds of students in some proportion. (We also are mandated by law to do affirmative action, which means admitting students at lower "merit" but they are likely to benefit the most from our education.)

A lot of universities have a stated goal of helping the society that nurtures them. One way to do this is to prepare entrepreneurs who will create jobs in that society. Depending on the geographical location (and particularly so, if the location does not attract a lot of investments), universities may prefer to have some weight to the nearness criteria, since students belonging to that society are more likely to stay there beyond graduation. A lot of universities will, therefore, admit more students from within the state even when that magic number representing merit is lower for these students.

In summary, the insistence on admitting based on a single number is misplaced. As long as the single number is considered a proxy for transparent/honest process, the insistence has a logic, but when this number starts getting treated as a proxy for merit, there is a problem. On the other hand, it is necessary for public funded universities to not only admit the best, but also give an appearance of admitting the best. As long as public universities like IITs have a transparent/honest process, they should have flexibility in both defining the merit (as combination of multiple parameters, perhaps), as well as bypassing merit to achieve the multi-objectives of  the university.

Sunday, April 16, 2017

Improving the gender ratio in IITs

Recently, there have been media reports on a decision taken by Joint Admissions Board (JAB) of IITs where by IITs would create some extra seats for women so as to improve the gender ratio in the under-graduate programs. Two of the reports are here and here. Earlier, JAB had asked a committee headed by Director of IIT Mandi, Prof. Timothy Gonsalves, to look into the ways of improving gender balance in IITs. This decision is apparently one of the recommendations of the committee.

Though the details are sketchy, it seems that there is a goal of having at least 20% women in the under-graduate class in stages. For 2018 admissions, the goal has been set as 14% which will increase by 1% every year to reach 20% in 7 years. In recent years, number of girls admitted to IITs number around 9%. To achieve a 14% ratio in 2018, they will have to increase the number of seats by 6%, and all these 6% will be filled exclusively by women candidates.

A little over a year ago, I had suggested in this blog that we must do some research into why women are not getting selected in larger numbers despite their performing extremely well in Physics, Chemistry and Mathematics in class 12th and also they perform well after they get selected in engineering. My own contention is that there is an inherent bias in the society which restricts the coaching options for women. The fraction of women in Kota coaching classes is very small, for example. Even in larger cities, the fraction of women in JEE coaching is less than the fraction of women in science sections in schools. Sometimes, it could be lack of willingness to pay high amounts for a girl child. Or it could be the concern for their safety.

I am not sure what data the committee looked at, but apparently they did find that part of the reason for lower number of women is societal bias, and hence that bias needed to be compensated by some mechanism. And what other mechanism do we know of in this country but reservation.

Is reservation the best mechanism to achieve gender balance?

The obvious shortcoming of reservation or a quota system is that its benefits are not directed towards the disadvantaged class but a larger class. The additional seats may not all go to those women whose parents have refused to send them to outstation coaching, or even to a more expensive coaching within the city. At least some of the seats will be taken up by women who actually go for expensive or outstation coaching.

But note that this is the problem of reservation based systems in general. Aren't OBC reservations benefiting students whose both parents are well educated and can hardly be called "Educationally Backward." But in India, we always argue that if we don't use simple proxies for disadvantages faced by people, the whole system will be gamed by rich and influential. In case of simple proxies (like caste for socially and educationally backwardness), some non-deserving people may sneak in, but it helps those who need such help. Something similar is likely to happen with women reservation as well. A few non-deserving women will get admission, but overall, it will help compensate the societal bias to some extent.

I think something more interesting may happen here. Once the parents know that getting admission to IITs is somewhat easier for women, they may actually be more willing to get them coached. Currently, one of the reasons for not investing in their coaching is that the chances of success are so low, and the expected return on investment is consequently low. They are willing to invest in sons' coaching even with that lower expected return, but will invest in daughters' coaching if the expected return is higher. As a result, a greater percent of women may succeed in the admissions process on their own. My gut feeling is that the percent of women in the normal process will keep increasing and they will need only 5-6 % supernumerary seats even as the goal improves from 14% to 16% and all the way to 20%. And because of this hope, I am positive about the reservation.

Are there other methods that they could have used to increase women admission?

Absolutely. JNU has had the scheme of deprivation points in their admission process. Under this scheme, they would add a few points to the other pieces of evaluation based on some criteria of background of the candidates. One of the criteria is gender, and a small benefit accrues to female candidates.

IITs could do something similar. They could increase the marks of every women candidate by some small number in a way that in the top 10,000 ranks, there are exactly 1400 women. And now women have ranks based on this new marks. One could a priori decide what is the maximum number of marks to be added, and if to ensure that there will be 1400 women out of 10,000 ranks require a higher number of marks to be added, then we will still stick to the maximum marks. (And, of course, we would know how many of them were in top 10,000 before these extra marks, and how many have been added, and create that many more seats to satisfy the current policy of not reducing the number of seats for categories not part of new reservation.)

Of course, it is easy to expand this mechanism to implement all sorts of reservations, and we will have data on exactly how much difference there is between various categories. So we could increase the marks of all SC students in a way that there are exactly 1500 of them in the top 10,000 (subject to the maximum number of marks, note that even now there is a limit on how much lower we will go in the merit list).

This mechanism is very useful when you have very small reservations, for example, in case of Physically challenged students. A PH-ST student is competing for a 0.2% quota (3% of 7.5%), which means that in a large number of programs, there will be 0 reserved seats in any given year. But if you add enough marks to their score that they represent 3% in 10,000, they will be able to seek admission to any seat that they deserve at their performance level.

This mechanism will be useful if we want to compensate for any other bias or discrimination or deprivation that candidates have faced.

Of course, what we are arguing now is that a 15% reservation over 10,000 seats means that the reservation is overall and not in each program. This would mean that they may get slightly less than 15% in some programs and slightly more than 15% in some programs. We only need to make sure that the distribution of marks are such that it won't lead to very high or very low presence in the popular programs (which I suspect will not happen). I don't know how courts will look at it, but it is worth trying.

Can we improve gender balance without any affirmative action?

That would be the least controversial and best method, in general. But that would require a lot of research, and we normally want to solve the problem without doing research. For example, if the hypothesis that lower women representation is due to societal bias and consequent lack of investment in their coaching turns out to have some merit, then perhaps we need to have the entrance exam (or at least some components of it) which are not impacted by such high pressure coaching. Small amount of coaching would be enough. One way to do that is to have speed tests, I am told, instead of very difficult to remember tricks. On top of that, something that government has already asked IITs to do, we can get study material prepared by IITs. And, of course, we are also seeing development of apps where by a candidate can practice for speed tests and get feedback sitting at home, all at a very low cost.

Of course, if the research shows some other reasons behind gender imbalance, we will need to tackle that properly.

Will this lead to more demands of diversifying student population?

Tamil Nadu has a little over 5% population of India, but it does not send 5% students to IITs. Muslims have about 15% population in India, but the fraction of Muslim students in IITs is much smaller. Wouldn't there be demands for increasing their representation.

Of course, there will be. But note one thing. The system proposed is saying that if 50% population does not have even 20% representation then there is something wrong somewhere, and we need to do something about it. So, the goal is not to ensure representation aligned to population fraction. Also, this is the population which seems to be doing much better in pretty much every exam in the country, except JEE advanced. If there are other groups which meet these criteria, that too can be studied.

Summary

It is a difficult decision. But one that I think could lead to attracting better talent by IITs. I am hoping that the "quota" part will remain very small and will eventually go away, and that IITs will implement other ways to attract talent, including changes to JEE.

Added on April 22, 2017:

Prof. Timothy Gonsalves, the Chairman of the committee on improving gender imbalance in IITs has made a posting on his FaceBook wall giving a summary of what went behind the report. I strongly encourage everyone to read that.
 An excerpt from the same:
--------
Would admitting girls with slightly lower ranks compromise on quality at IIT?
 A study in IIT-Delhi looked at the final CGPA of male vs. female students. Over a period of 13 years (2003-2015), females outperformed males consistently by an average of 1 grade point, despite having lower JEE ranks! This amazing finding gels with our experience as teachers in other IITs also. It is an indication that this cohort of young women is extraordinarily talented and highly trained despite the disadvantages of growing up as girls in India.
--------
I am told that the difference in grade between girls and boys of similar JEE ranks at IIT Delhi is a whopping 1.5. It proves beyond a shadow of doubt that giving some push to women whether through quota or bonus marks, or whatever, will actually admit better students to IITs.

Tuesday, April 11, 2017

International Student Exchange Programs

In the last 2 years that I have been in Delhi, I have attended many a meetings with folks from different countries trying to figure out how the institutions in India can engage with institutions in their respective countries. There are two models which seem to working well.

One is that of twinning wherein a student in India joins an Indian institution, does some course work for two years, and then seek admission in a partner university abroad, which recognizes the credits completed here. The student spends two more years at the foreign institute, and get a degree from there. This is a win-win situation for everyone involved (commercially, at least). The student wants a degree from abroad, and gets it at a cheaper price than spending all 4 years abroad. The parents are happy that they didn't have to send their ward abroad when s/he was too young. The foreign university is happy that they are getting at least two years' tuition from a foreign student. The local university is happy because they can actually charge a bit more for the two years than what they may charge for their 4-year program.

The other is that of research collaboration. Two researchers meet somewhere, may be in a conference, and they decide to collaborate. Much of the interaction can happen over Internet, and a few visits can be supported by their respective projects. On top of that, there are government to government schemes under which they can apply for projects jointly, and while getting big moneys in international projects is difficult, collaborations can certainly happen.

But what about things beyond this. In terms of teaching programs, can we have student and faculty exchange programs where our students can spend a semester or two in the foreign location, and their students can spend a semester or two on our campuses. The same could be done with faculty. Can our PhD students work in their labs and their PhD students work in our labs. This is where one does not find any solution. For our students to go to North America or Europe is very expensive - travel, lodging and boarding, as well as tuition. For their students to come to India, well they don't think of it as an option. If they were coming to our campus, it would be easy to argue for tuition waivers. They don't pay tuition here, and our students don't pay tuition there. But we don't see a 2-way exchange.

So in every such meetings, there will be complaints about lack of two-way exchanges.

In a recent meeting, I asked the representatives of various universities whether they have an MoU with a university from Bangladesh or Sri Lanka or Mayanmar. The answer was on expected lines, none of them had an MoU with any university in these countries. The reason was supposedly obvious. Their universities were not great (though I can tell you that most universities represented in that room would not be better than good universities in these countries). Now, your student does not want to go to a university which is roughly similar to that of your quality, and you are wondering why an American or a European student does not want to spend a semester at your campus.

I recall that long time ago, when we were discussing relationships with top universities in Senate of IIT Kanpur, one professor had said that only after IIT Kanpur has had a good working relationship with HBTI, MNNIT, and other decent colleges of UP, would it realize how to have a relationship with MITs of the world.

A gentleman from US asked a question, "What is the goal for student exchange?" A pin drop silence. Frankly, the goal is only to brag about it, or enable our students to go to US/Europe with fee waivers. If the goal was what normally universities say, greater cultural diversification in the class, then we could achieve that by having more students from Africa, Central Asia, South East Asia, etc. Our focus on these regions is fairly limited right now.

I believe that the right thing to do by our universities is to attract exchange students from comparable or less developed countries than us. This will help us in multiple ways. It is more likely to succeed than to try wooing students from US/Europe. Thus the first goal of cultural diversity in the class (and a side goal of doing better in ranking) will be achieved. This will make us learn what are the challenges that foreign students face, including but not limited to getting a visa, police registeration, finding accommodation in nearby localities, and so on. It would be easier for our students to spend a semester in these places and gain an exposure of different cultural setting. Once we have all this knowledge and experience, we will be able to come up with better ideas to expand the scope of exchanges to richer countries. Right now, there are many attractions that we can market to students of richer countries, but we really don't know how to leverage them, how to prepare a program suited for different classes of students.

And just like we are offering a twinning program to our students, we could get into agreements with universities in less developed regions that they will send us their students after two years of training and we will train them for 2 more years and give our degrees. Why should we always be importer of education service. We should try to become an exporter of education service.

At a national level, bringing students from such countries also projects our soft power, create goodwill, create ambassadors for life.