Failing and Succeeding Out Loud: Embracing Evidence-Based Policy Innovation


In a world full of complex urban problems, decision-makers can use all of the help they can get. But the politics involved can leave leaders reluctant to invite researchers to evaluate their successes when they must also share their failures. We imagine a future of cities openly exploring what works.


Status Quo Governance

We all have ideas about what works, what people in our communities will like, what has seemed to work before, and what will pass a city-council vote. It is important to consider the politics of city-governance. After all, making the right policy choice will not matter if no one is willing to pass or implement the policy.

Working local politics and basing decisions off your experience is not a bad way to govern, but it is not the smartest way to govern either. It is simply the status quo.

The Ethics of Finding Out

Source: Matthew Wiebe, Unsplash

Imagine that you are on the school board of a district with a high-dropout rate and you are responsible for deciding which interventions to fund. There are after-school programs, assistant teachers, and mentorship programs. All of them sound great, but if your district is going to move forward, it needs to to do one thing really well instead of trying to juggle everything. When boards deliberate, they typically try to reach a decision based on what matters most to families in their community.

What we know is really great for science, can however, seem maligned for policy. Randomized-trials don’t feel fair, because they require some people to be excluded from a particular program or ‘treatment’. Altering the services people receive is scary and getting it wrong can have immediate social and political consequences.

What’s worse than failing? Continuing to fund ineffective programs and becoming complacent with mediocre governance. When resources are limited and how cities spend their dollars really matters.

Source: Samuel Zeller, Unsplash

Enter Academia:

There are thousands of researchers in the United States interested in the outcomes of your town or city. Urbanists, economists, public health researchers, criminologists, sociologists, and psychologists would be thrilled to help you evaluate and innovate your social and economic policies.

Conducting a randomized study of a program will help you find out what works, while separating your outcomes from other changes in your city.

In a randomized trial of high-schoolers, students will be randomly selected to receive a treatment while students in the control group will receive a placebo treatment or no treatment at all. This means researchers will be able to tell what the cause of change is. For example, is your program keeping participating kids in school longer or is something else in the community — like a growing local economy —at play?

While it may be politically difficult to conduct a randomized trial — especially if you like the program under consideration — your outcome, good or bad, will help city leaders for years to come govern better.

If every city in America openly experimented with their social and economic policies, we would all be able to make smarter and faster decisions about what is best for our communities.

Success on the Southside of Chicago

Becoming a Man participants meet with with President Obama 2013. Source: The White House, flickr

The Chicago school district discovered how to reduce violent crime arrest and increased graduation rates for pennies on the dollar — all through a trail of a cognitive behavioral therapy program called Becoming A Man or BAM. You can listen to the whole story on this Freakonomics podcast.

BAM takes the concept of cognitive behavioral therapy, which is typically used in psychology, and applies it to a social-work setting. The program challenges students to reconsider how they address conflict and social negotiations with alternative solutions. Young men are given a space to figure out what masculinity should look like in their neighborhoods.

By conducting a lottery to randomize the program, students did not feel singled out and now that the program has proven to be successful, there is plenty of support to fund it. By embracing research-based innovations, Chicago public schools improved the social and educational outcomes for their students.

Solving Problems Out Loud

Source: MH Merhi, Unsplash

Be open about the challenges that your city faces and crowd-source solutions. Invite researchers and local universities to examine your policies and give them freedom to conduct experiments. Trial-and-error in the short-term will mean strong innovations in the long-term.

You might discover that a policy you really like, one you were advocating for, isn’t good enough (yet). Share that too. With this strategy, the way we approach education, crime, poverty, and housing can only get better.

Are you an urban problem solver? Speak with someone today to learn more about using data to find patterns in your city.
About the Author: Michelle Stockwell is more of a qualitative researcher herself, but is always thrilled to see cities do more with numerical data.