The Truth is Out There, It’s Just Very Hard to Find

How government makes evidence-based policy difficult to practice

Mike Davis

Poverty is a shockingly complex problem. Governments have struggled for decades to solve it. Fascinated by this complexity, I have dedicated much of my career to understanding it.

Some suggest ‘allowing’ people to live in poverty is unacceptable. Unconscionable. Unfathomable. Not to mention the cost of poverty. It is estimated to cost Ontario around $30B annually and Nova Scotia $2B annually (which is about 15% of their respective provincial budgets).

Modest investment has been made to generate evidence that can be used to address poverty, but nowhere near as much as has been invested in areas such as health and climate. Research on even the most promising solution to poverty, a universal basic income, is still in its infancy.

If poverty is such a huge societal problem, why haven’t governments invested more in research on how to solve it?

I recently wrote about what evidence-based policy is (and isn’t). In summary, ‘evidence-based policy’ refers to using previous research and the ‘scientific method’ to make government decisions. It has the potential to improve policy and programming (and therefore society) as it allows governments to assess the impacts of their decisions more rigorously and ensure investment is being maximized to improve lives.

If we know that evidence improves policy-making, why don’t we use it more?  Why is it so hard for government to embrace?  One reason it’s hard for government is the same reason that most things in government are hard – because of the nature of government itself.  Here are four reasons that speak to this.

First, the term ‘policy process’ would suggest that there is a nice, linear approach to identifying a societal need and working through established steps to reach a policy outcome — a race that has a start and a finish. In reality, policy-making is chaotic, looking more like a game of tag than a race with a defined route. It involves multiple individuals at both the political and bureaucratic levels of government, numerous stakeholders, the public, and the media.

There are a variety of inputs, some data-driven and some opinion-based. Policy-makers take these inputs, conduct some type of analysis, and navigate a variety of bureaucratic, decision-making processes (again, picture a game of tag where someone is weaving in and out of others without knowing exactly who is ‘it’ or where even to run). A decision is then endorsed — and, ta-da, policy has been made! As a result, it is hard to figure out where best to insert a scientific approach into a policy ‘process’.

Second, even with policy-makers advocating for an evidence-based approach, setting studies up can take considerable time that those responsible for making key policy decisions don’t have. Think about how quickly officials needed to respond to the ever-changing Covid-19 situation. While some governments have been lauded for their ‘science-based’ approach, even the good ones have had to limit their focus to epidemiology and reducing the spread of the disease. They were far less able to gather evidence to consider the impacts that lockdowns and travel restrictions might have on mental health, marginalized communities, small businesses, or climate, for example.

Third, there is a bias against running experiments in government. Experiments often require people to be segmented into groups and treated differently, with some receiving a promising intervention while others do not. This raises obvious ethical questions, but I don’t believe that they are legitimate. In a ‘worst case scenario’, people not receiving the intervention being tested in an experiment usually receive the same treatment that is already in place, meaning they are in the same situation they would be in if there was no experiment at all (i.e. no worse off). And evidence of the effectiveness of the intervention being tested has yet to be collected (so we don’t know who is being treated better). Finally, programs are rolled out all the time without strong evidence of their effectiveness, which is arguably a more ethically-questionable practice than using the structured approach of a scientific experiment.

Finally, privacy regulations make it very difficult to collect the necessary data. The data needed to conduct a study might already exist within government systems, but the administrative burden and confusion regarding privacy rules deter even the most ambitious policy-makers. Where data doesn’t exist and could be collected through experimental studies, privacy protections still need to be addressed. Privacy regulation and ethics review processes allow for all of this to be done, but each study is treated like a brand new concept forcing policy-makers to answer many of the same questions and concerns over-and-over.

So, how can we improve this? Restructuring governments or the policy process are probably unrealistic goals. In my opinion, the answer lies in closer collaboration between government and those that have expertise in conducting extensive research — the academic community. I’m certainly not the first to suggest a closer working relationship between policy-makers and academia, but there are a lot of opportunities to improve this collaboration.

In my next piece, I’ll talk about the opportunities and challenges to evidence-based policy presented by academia, where much of the research capacity and evidence currently resides.

Be sure to sign-up here and receive my next blog as soon as it goes live.

Related insights