How do you say that a country is doing better than another? If I were to look at two people from the two countries and tell you one is well off and the other struggling to make ends meet, could I say anything about the nations, and if yes is that all the information we need? Most racists will not believe racism exists so how do we quantify racism for them? Would that make identifying racists easy? Economists have a knack of quantifying social phenomena and rightfully so. People put a certain faith in words if they are backed by numbers. In social sciences, individuals have been trying to quantify qualitative phenomena for decades. Evidence based policy is trying to take forward that notion, convincing people to believe in a policy decision based on extensive statistics based on past experience.
Since the time of tribes and kings, humans have always had an inclination towards collective good, even if it was for their own community. This idea of a collective welfare of the people made rulers stay rulers and let communities prosper. When societies developed to make more complicated political and economic structures, they developed governments to make decisions on welfare. Public policy over the years has largely been influenced by historical contexts, individual ideals (of those in power), both collective and personal experiences. The system was working well for most part of history and suddenly there was an explosion of computing capacities. We are now experiencing more informational exchanges than at any point in history. The capacity for this exchange of information with the increased prosperity of societies has allowed for the policy debates to entail data into consideration.
On the flip side, With this rampant spread of information, we are now observing record numbers in the spread of fake news (which spreads almost 6 times faster than actual news). More than ever before, policymakers require objective and empirical support for the right kind of policies. Evidence based policy is aimed at bringing a systematic empirical review of all existing research regarding a policy (which they usually refer to as an intervention). It gives a targeted assessment by policy intervention on indicators in the area of the policy.
A special kind of approach in the field known as Randomized Control Trial has been popularized of late, pioneered and promoted by the 2019 Nobel Prize Winning Economists Abhijeet Banerjee, Ester Duflo and Michael Kremer. The approach is essentially like a medical RCT which focuses on producing consistent and unbiased estimates. One collects samples for treatment and control groups from a population and randomises such that the differences in the characteristics of the two groups are not statistically significant. The control group becomes the counterfactual which explains when the intervention had not been made. RCTs have made giant leaps in helping us understand policies that work and the ones that don’t. Studies on microfinance by Abhijeet Banerjee, Ester Duflo, Rachel Glennerster and Cynthia Kinnan in Hyderabad have shown that the erstwhile celebrated policy has very little impact on empowering women. The studies help bust myths about policies that one would assume works and give an objective view of the unbiased facts. They have also been subject to a lot of debate due to the ethical consideration raised on account of the subjects, especially in the developing world who may be withheld from access to services due to the misfortune of being in the control group(or if they were not the study sample).
A recent study on the impact of cutting off water supply of Nairobi’s slum dwellers on their payment behaviours has garnered a huge attention on twitter (and since then has been redacted by the authors for reconsideration). It stirred up the everlasting conversation of ethics in the field of social research. Another study examining the impact of “how violence exposure affects the way Kashmiris feel about the Indian state and how they respond to Pakistani irredentism”(Nair and Sambanis 2019). The study being approved by the Yale Institutional Review Board raises questions on what is acceptable for research. Such studies exemplify the ethical concerns for RCTs and the need to publicise and strengthen review boards.
The parameters that should ideally come from any ideal RCT (or any research for that matter) is its discovery of new facts or addition to the already existing pool of knowledge, its economic, political and social costs and the benefit/harm caused to the study group. The Nairobi study shows that after cutting off the water supply the landlord is mostly induced to pay the fee but to counter it adds it to the rent that the tenant has to pay (and subletting extra areas for additional money). The Kashmir study too, states the obvious that after viewing violent content they are less likely to be in favor of India. While getting results accepting and rejecting a hypothesis are used and accepting a hypothesis can be called a good research if it brings out any other factor like efficiency, cost, or other comparisons. The political costs of the Kashmir study were phenomenal considering they studied half the state legislative assembly constituencies in proportion to population size in all ten districts of the Valley and the research may essentially influence polls significantly for the elections and the social costs of swaying public sentiments should have outweighed the benefits from the study. The Nairobi study in that way would have soured the landlord-tenant relationship and further increased difficulties(by making them bear the extra costs) for the already disadvantaged tenant. Another very important thing missed out in both the studies is that there was little to no effort made on taking informed consent of the subjects. Proponents of RCTs in the field of social studies talk extensively about informing the subjects (who are already disadvantaged) who wish to partake in the study.
These studies are not singled out because of any malicious intent but to demonstrate how easily seemingly harmless (and often interesting) research questions can have detrimental impacts on the social fabric. RCTs usually have low external validity and need to explain every local phenomena that can explain changes in order to successfully demonstrate a causal relationship. Rarely is a study devoid of personal bias, there are always underlying biases that motivate the studies. In that sense RCTs are a reliable method of evidence collection that should complement the existing research and should not be treated as the “gold standard” for policy making. Evidence based on historic context and collective experiences and needs to be valued just as much as the ‘objective’ measures of research.
The future of science, social life, businesses and public life lies intertwined with data in this digital age and so the reliance on data-based research should come as no surprise. The goal of the thought leaders should be on the promotion of good/ethical practices and the checks on bad practices to set up a framework for policy making.
By-
Angira Shukla (angira.shukla@gmail.com)