---
id: "workshop:innovating-in-the-public-interest-winning-early"
type: "InnovateUS workshop"
title: "Innovating in the Public Interest: Winning Early"
url: "https://innovate-us.org/innovating-in-the-public-interest-winning-early"
date: "2025-04-09"
duration: "60 minutes"
language: "English"
series: "Innovating in the public interest"
instructors: ["Anita McGahan"]
recording: "https://www.youtube.com/watch?v=JbF-znn48ws"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Innovating in the Public Interest: Winning Early

InnovateUS workshop · 9 April 2025 · 60 minutes

Great projects begin with early wins. We will talk about the importance of driving for early wins – and of using them to build understanding of how collaborators are each evaluated and rewarded for successes.  Early successes can become the foundation for longer-term initiatives.

## Learning goals

- **Design Early Wins** – Learn how to structure quick, targeted successes that build trust, demonstrate impact, and reinforce public-sector credibility.

- **Navigate Cross-Sector Dynamics** – Understand how different organizations measure success and leverage early achievements to strengthen long-term collaboration.

- **Overcome Skepticism** – Explore strategies to gain buy-in from internal and external stakeholders by demonstrating clear public value.

## Speakers

- **Anita McGahan**, Senior Research Scientist, The Burnes Center for Social Change

Part of the series [Innovating in the public interest](https://innovate-us.org/workshop-series/innovating-in-the-public-interest).

Recording: https://www.youtube.com/watch?v=JbF-znn48ws

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/innovating-in-the-public-interest-winning-early.txt (plain text) and https://innovate-us.org/innovating-in-the-public-interest-winning-early.json (every segment with its start and end second).

**Audience**

[0:27](https://youtu.be/JbF-znn48ws?t=27) Good afternoon.

**Anita McGahan**

[0:29](https://youtu.be/JbF-znn48ws?t=29) Hello, everyone. Thanks for being here. We will start in about 10 seconds, 15 seconds. Just want to make sure everyone gets in through the waiting room. OK. All right. So let me just begin by welcoming everyone. My name's Anita McGann. I'm a senior research scientist at the Burns Center for Social Change at Northeastern University. Today, we'll be talking about early wins in improving people's lives through lasting change in the public sector. This is the second in a six-part series called Innovating in the Public Interest, Partnering for the Public Good. And those of you who are signed on are eligible for a certificate both for this session and if you participate in four of these workshops for a certificate that reflects your broader participation. So thanks for that as well. OK. So we're going to be talking about winning early.

[1:31](https://youtu.be/JbF-znn48ws?t=91) And let me just jump in by reminding you that what we're talking about winning early about is innovation through the public sector that has a lasting impact on people's lives to improve people's lives. So that sounds very broad, but as I'll explain momentarily, we actually want to be very specific about what those innovations will involve and how to accomplish them. This course is about the idea that you want to improve outcomes through your organization. So if you're working in the public sector, you want to use your platform, your job, your scope of impact already as a way forward to improve people's lives. You generally need to take on some creativity to do this because most public sector organizations are generally not designed for change.

[2:34](https://youtu.be/JbF-znn48ws?t=154) And we're going to talk a little bit about through throughout the workshop series here about how to use resources from partnering organizations like in the private sector or even among other, you know, your friends in other jobs in the public sector to actually accomplish change together. And there are some very specific things to avoid and things, very specific things to take advantage of that we have discerned through our research, studying, you know, initiative after initiative, some things that work and some things that don't work. So we're, the focus here is on trying to provide you with a roadmap for how to actually accomplish this in ways that are stress tested for their effectiveness. So we talked last time just to remind you about getting how you get started. Here, our focus was on, you know, thinking about what do you want to change?

[3:35](https://youtu.be/JbF-znn48ws?t=215) Do you want to provide a new service? Do you want to streamline a profit process? Do you want to create some evidence, build capabilities and restructure? What is it that you want to innovate on exactly to improve people's lives? So we talked about that. And by the way, the workshop video is available through the Innovate US website. So you can access that either directly through Innovate US or through pysquared.burncenter.org. We had a number of different examples that we talked about. We talked about a number of different organizations that sought to improve employment and what it was like to be an employee in the public sector. IBM was doing skills-based hiring instead of hiring people based on college degrees. We talked about Tulsa Remote in Tulsa, Oklahoma that sought to attract remote workers to reside in the Tulsa community that had been designated for that effort and paid people to move to Tulsa.

[4:42](https://youtu.be/JbF-znn48ws?t=282) We talked a little bit about Oaks Rehabilitation in Toronto, which was working with local community leaders, particularly in policing, to identify candidates for construction jobs. We talked about Georgia Works, which is in Atlanta, that was retraining mostly homeless men for jobs in the city. There was a whole category of examples from education. We talked about innovative educational delivery models like Giddy Mobile that has online gaming as an entry point for high school students. Binti that was working to improve placement and matching of students with educational opportunities, volunteer placement. We talked about developing cities, women's entrepreneurship programs all over the place. In New York City, we talked about how restaurant grease in sewers was remediated through a public-private partnership that was innovative. Banco de Providencia in Rio, the Build From Within Alliance in cities like Detroit and Minneapolis.

[5:48](https://youtu.be/JbF-znn48ws?t=348) And I'll come back to that, building Build From Within Alliance and some of these other examples today. We want to marry examples of successful changes and processes with contrasting examples of some problems. And we want this to be focused for you and accessible and concrete so that you can really make a lasting impact now in your workplace starting tomorrow, starting today. So what we want to avoid is what I'll call the play pump problem. That play pump is illustrated on the right-hand side. Just to remind you, this was a merry-go-round style pump. So you've got kids playing on a merry-go-round as they play and turn the pump. That's gonna drive a pump to push water up into a cistern represented on the upper left-hand side there. And this thing was a total failure. There were a lot of reasons that we went over last time, but some of the big traps were that it wasn't appropriate for the community.

[6:54](https://youtu.be/JbF-znn48ws?t=414) There's not enough groundwater. There wasn't, these things would break down. There were no spare parts. The community, the community's practices were not aligned with the storage of water. We had all sorts of issues of gangs and thugs taking over control of this apparatus here. So there's a whole set of problems that arise from pushing technology into a different context without really understanding the context. There's also another set of problems that comes from analyzing something to death or pursuing media headlines for initiatives by making big, glorious announcements that are not substantiated by actual media activity in the background. We don't want you taking unnecessary risks with your resources or your careers. We don't want you to bet at all. We don't want you to become vulnerable by opening the door to well-intentioned companies that don't have a strong history of doing work in these areas and don't know what they're doing and are experimenting on your desktop.

[8:03](https://youtu.be/JbF-znn48ws?t=483) And pushing tech is really a big problem that's very widespread. And so we don't want people chasing uninformed ideas like a play pump. So last time when we talked about getting started, we had five things that we were focused on. So let's imagine that you've done this. You've built your team. And to remind you, that means you've pulled together a few friends that share a passion for accomplishing some sort of change. It's very specific. It's very well-defined. And you want to work together. You've got an idea. The second thing is you've narrowed this. You've measured it. You know how to measure whether or not you're actually having an impact. You've narrowed your focus. The third thing is you've done a case study to bust myths. You've got some sort of a story or some sort of a kernel of evidence that what you can do that provides optimism is going to overcome some sort of trade-off that was baked into the system when it was originally devised.

[9:05](https://youtu.be/JbF-znn48ws?t=545) So for example, in a particular community, for example, it may have been that construction jobs were not available without certifications to the general public. And there was this sort of idea that trying to pull people into these construction jobs would happen through trade schools, let's say. And let's say it was hard to get people to take these jobs, these construction jobs, because they were dangerous. That may have been true 20, 30, 40 years ago when the industry was initially structured and when certification was initially established. But let's say that today, you've got a shortage of construction workers in a city like Toronto, and you can show that you can get people certified quickly without putting them through trade school by training them specifically on the certification criteria. So you've got a kernel, a story.

[10:07](https://youtu.be/JbF-znn48ws?t=607) You've got evidence that it's possible that you can get really high-quality trainees through a new pathway, okay? So you've got some evidence that you can overcome some sort of a problem that constrained the design of the system historically. You've tied the agenda to a redefined public good. So you're thinking about how you're innovating, how your project will rebalance trade-offs between different elements of the public good. So for example, you've got at Oakes Revitalization Association, this idea that training young people for construction jobs is also going to reduce, it's going to reduce criminality and reduce violence in communities that are burdened by gangs. And it's going to have an impact beyond the construction industry directly to improve communities. And you've thought about how you're going to create successes for your partners. You've thought about how the people that are working with you to try to do this are going to benefit.

[11:09](https://youtu.be/JbF-znn48ws?t=669) So you're able to, for example, at Oakes channel really well-trained people into construction companies that had previously had a shortage of talent. And that creates a success for your partner that goes beyond the public good element of your approach. Today, I'm going to focus on three different examples that were, for which there are core teams that are represented on this slide. On the left-hand side, there's these three wonderful young people are Jim Kim, Ophelia Dahl, and Paul Farmer. They are three of the five co-founders which is a clinic, a series of clinics now, of Partners in Health, that provided healthcare for the very poor, first in Haiti, in the plateau around Cange, Haiti. Nobody believed that you could provide HIV care or high quality medical care in such an impoverished area.

[12:12](https://youtu.be/JbF-znn48ws?t=732) And these three young people go on to prove that belief to be incorrect. They became champions of pro-poor healthcare and innovation in the healthcare system. On the upper right, you see Joe Williams and Mark Tenaglia of Oakes Revitalization, which is that organization here in Toronto that I had just been describing. And then in the lower right, we have Ginni Rometty, the former CEO of IBM, Jamie Dimon, the CEO of JPMorgan Chase, and Alex Gorski, the CEO, former CEO of Johnson & Johnson. The three of them together with 178 other CEOs signed the business round table commitment to stakeholder capitalism. And we'll talk a little bit over the course of the next hour about some of the pros and cons of the way that played out after they did that. So the goal, the first thing that we wanna do before scaling up any sort of effort, once you have your team established, once you have your idea in mind, is to structure an early success.

[13:23](https://youtu.be/JbF-znn48ws?t=803) So we'll talk a little bit about how to do that, structuring early successes. Our agenda for this hour will also show the importance of demonstrating impact and the value that you can create by doing things differently. And finally, we'll talk about building a shared understanding of what's possible. So these are our three agenda items for today's session, today's workshop. Structuring an early success. Now, I wanna make clear, structuring an early success is not the same thing as creating an organization and building out a whole scaled apparatus for actually implementing something. We're gonna talk about structuring early success in terms of framing and really deepening an understanding of what can be done. So I'll tell you a lot more about that in just a minute. We wanna demonstrate impact. So we wanna actually have an evidence-based analysis of the way that value can be created.

[14:24](https://youtu.be/JbF-znn48ws?t=864) And I'm going to give you a couple of different frameworks for demonstrating impact. And then we wanna build a shared understanding. So we wanna create evidence with skeptics. So we wanna bring together people that are worried that what we wanna do is not gonna work into a conversation to construct some evidence that gives them a sense of possibility, a sense that they might have been too skeptical, a sense that they might be wrong about what is actually possible. As we talk about this, what we're doing here is trying to demonstrate. Great early wins. And this is a little bit different than what I talked about last time when I talked about busting this. Last time, we were just looking for a story. We were looking for, you know, a one-off opportunity, some singular piece of evidence that could be inspiring. Now what we're doing is we're looking for more robust evidence. We're looking for systematic, stress-tested, hardcore data that innovation is going to get us new value creation, and that there's a lot of new value to be created through innovation. And we want to make sure that skepticism about that potential is confronted and addressed with evidence. So today is going to be about producing that evidence. Let me begin by talking a little bit about the business roundtable. So I'm going to end up this hour by saying that there were some problems with the business roundtable. The 2019 business roundtable announcement that was pitched by Alex Gorski, Jamie Dimon, and Jenny Rometty, it was championed by them. What that was, it was a press release by 181 CEOs of some of the most

[16:25](https://youtu.be/JbF-znn48ws?t=985) important companies, particularly in the United States, but these are globally significant companies like JP Morgan Chase and Johnson & Johnson and IBM, that went directly to the idea that companies should be in the business of promoting an economy that serves everybody, all Americans, ultimately everybody, and not just shareholders, not just investors. And the basic logic here was the CEOs were saying that if we focus on creating value for our customers, if we focus on making our suppliers successful, our employees and communities more robust, more successful, more fulfilled, then shareholders will ultimately benefit. That if we focus on value creation, that there will be a trickle down of the effect of all that value creation, and investors will prosper. Okay, so there was a declaration by 181 CEOs in a belief that this was possible. But as I'll describe later, there wasn't a comprehensive plan to address skeptics, and there wasn't a lot of detail on exactly how this would work. So that's going to end up being a problem later. We'll come back to that. But at the outset, the idea here was a showing that this is possible. We got 181 CEOs saying that they believe this. That's a really compelling statement about what's possible.

[18:03](https://youtu.be/JbF-znn48ws?t=1083) And that's an important starting point. This is a picture of the plateau in Haiti, where Partners in Health ultimately built out Zanmi Le Sante. It's a hospital complex up on the upper plateau. They started on this dusty, you know, dry, you know, place by building one or two buildings here to provide first class health care to the poor. And the goal was that this complex would serve some of the most impoverished Haitians in the country, and therefore some of the most impoverished people in this hemisphere with first class health care in a remote place, and that this would be possible. That was the goal. And what Zanmi Le Sante did was incredibly successful. So there is going to be a contrasting success here. In fact, Zanmi Le Sante's problem was that it was so compelling to do this that they were overwhelmed in some sense with a demand to scale. And scaling too soon is a problem. But not having the capacity to scale also is a problem. That second issue will be the topic of something that we deal with later. We're also going to be talking a little bit about the Oakes Revitalization Association here in Toronto. This crowd of a community liaison committee in the Toronto Police Services 31 Division, which is marked here in red on the map, you can see Toronto, the name of the city of Toronto, sort of toward the bottom right hand side of this picture in Lake Ontario, looming large down at the lower right hand corner. This neighborhood, the 31 Division serves, includes an area called Jane and Finch, which is in the city, analogous to an area where my parents

[20:11](https://youtu.be/JbF-znn48ws?t=1211) hailed, which is the South Bronx in New York City. It's a place where historically, there's been a lot of gang activity, a lot of violence, it's been very difficult to provide sufficient services to preserve safety in this part of the city. And what the community liaison committee in the 31 Division was concerned about was how to address this problem at its core. And you'll see some members of that committee on the on the picture here, the photo here. At the far right, the upper far right is Mark Tenaglia, who is one of the co-founders of Oakes Revitalization. He became incredibly inspired and committed and put pretty much his entire, everything he had at his disposal, all of his personal wealth, all of his time, all of his effort into trying to address this problem and started Oakes Revitalization on the idea that we can provide a pathway for high risk young people that are in this area that are tired of the kind of lifestyle that is the default choice for young people such as themselves.

[21:33](https://youtu.be/JbF-znn48ws?t=1293) They're tired of that, and they want good, secure, high paying jobs. They want a ticket to the middle class. If we provide that to them, we can keep them out of the justice system and provide them with a much brighter future, a safer future, the possibility of having a family, having homeownership, and that that would be compelling. And that turned out to be enormously successful as well, although faced with some scaling challenges that I'll describe a little bit later. Okay, so we got to structure the early success by expressing what the value creation opportunity is. Now in business, we have a framework and I want to show you this framework because I think it could be useful to you as you start to think with your friends about what kind of value your idea creates. So we're going to define value creation here as the difference between the buyer's willingness to pay for whatever you're selling, in the Oaks revitalization example, it's trained construction workers with good work habits. The pay, the entry level pay is something like 75,000 Canadian dollars per year rises up to 150,000 and even more if you're effective at your job. There are construction companies out there willing to pay salaries to people that are that are relevant. So that's kind of creates a sort of benchmark or an anchor for trying to express one piece of the value creation. A second piece is down at the bottom of this value creation equation is the supplier in this case, the young person at Jane and Finch or the young person in the South Bronx, who is faced with a choice. Am I going to potentially make a lot of money in the very short run, but that

[23:34](https://youtu.be/JbF-znn48ws?t=1414) money's going to get run through quickly? Am I going to choose a gang to pursue illegal transactions and put myself in danger and at risk, but with the potential to make a lot of money very fast, very regularly, but with the potential for a tremendously violent life, for incarceration, for conflict, for meaningless territorialism, prison sentences, constant threat of death? Or am I going to pursue the Canadian dream and try to get out of this situation and have a quiet, calm life with homeownership and, you know, a family and peaceability and a membership in a community? The gap between even the conceptual gap between the buyer's willingness to pay and the young person's choice set is where the value creation opportunity lies. Okay. And that gap has to be sufficiently great to compel an organization to pursue it. There has to be enough there that, you know, it's worth pursuing, enough value to create to be worth pursuing. So the organization comes in and seeks to fill that open opportunity and bridge that gap. So we can think of the organization's added value as the amount of value that would be created if it didn't, if it exists, minus the amount of value that would be created if it didn't exist. So let's imagine that occasionally an untrained worker would find their way into a construction job, but it would be rare. There wouldn't be any sort of support

[25:35](https://youtu.be/JbF-znn48ws?t=1535) for that person to overcome some of the constraints associated with their origins. There would be a kind of systematic set of challenges with getting transportation to work, with learning good work habits, and so on. But there's a little bit of value that occasionally would be created. The difference between the value created at scale or the value created, the creation potential, with an intervention like Aura, minus the value created with occasional successes that can occur without Aura's involvement, is the added value of Aura to create this value. And in this instance, it's quite large. You know, in quantifying this and showing this, and thinking about the value of the innovation that you're interested in pursuing, in terms of its value creation, is often incredibly compelling. If you can provide evidence that there are enough people that want this kind of opportunity, enough construction companies that would be interested in hiring young people that have this profile, you know, you can start to illustrate the potential of an intervention here. Partners in Health, for example, on the Haitian Plain, the idea here was, we can innovate in healthcare by hiring community health workers, and putting in a relatively few physicians into this community, leveraging their impact through trained community health workers that could implement doctors' orders much more efficiently and effectively, much less costly way than in conventional hospitals. So there was an organizational innovation through community health workers

[27:35](https://youtu.be/JbF-znn48ws?t=1655) roles in the system, to lower the cost substantially, and provide really high quality healthcare that was paid for by government. So you always want to follow the dollar to find the buyer. So when the government is paying through a budget, let's say, Zambi-Lacente was originally supported by a donor, a donor provided the funding for this. But over time, government grants, including from USAID and other US government agencies, but also through the Gates Foundation and other donors, saw a lot of value in what in the model that Partners in Health was pursuing, relative to higher cost alternatives in other places. And the trio here, accompanied by two other co-founders, was able to demonstrate empirically that the quality of the care was not diminished by this approach. So as you get this sort of demonstration of value creation that becomes very compelling. I'm going to argue that in the case of the Business Roundtable, this demonstration of value creation was almost taken for granted. It was offered almost on a theoretical level. If we serve our communities, serve our customers, serve our employees, serve our suppliers, we're going to create more value for them. And that's going to allow us to capture more value for investors. That argument, which is, you know, defensible case by case within each of the three companies represented by the CEOs and the other 178 companies that signed that agreement, was never fully articulated because of its complexity. And that ends up

[29:36](https://youtu.be/JbF-znn48ws?t=1776) becoming a challenge for those organizations. Let me keep going. I'm going to try to end in about 15 minutes, so that we have a little time for questions. So what happens in the case of Partners in Health in Haiti, is they have so much demand that the challenge becomes scaling for them. Figuring out how to meet the need for all this value that they can create becomes their biggest problem. And of course, that's a problem that you want to have on one hand. On the other hand, if it gets out of control, it becomes itself the next mountain that you have to climb. This is a picture of some hospital beds outside the physical building, you know, on the dirt outside because there's not enough room internally just to illustrate that challenge. You know, at Oakes Revitalization, you get the province of Ontario so compelled by this approach, they build a training center for 50,000 new construction workers. One of the critical elements of the Oakes Revitalization approach is they do a lot of work with the young people that come into the program to help them address some of the experiences that they've had at Jane and Finch, in their communities, and to start to develop better work habits, good work habits, like showing up on time and making sure that they can, you know, really perform well in the context and that they know how to ask for help at work if they don't know how to complete a task and so on. So, you know, you worry about too much too fast when the scaling is accelerated without a very strong shared understanding of all of the elements of the value creation project that need to be in place for it to be ultimately successful. So, you've got to demonstrate all the different elements of the thing and show their essentialness before rushing to scale and, you know, the key here is making sure that that's really carefully spelled out and articulated.

[31:44](https://youtu.be/JbF-znn48ws?t=1904) In the case at the lower right hand corner of the business roundtable, Larry Fink, who is the CEO of BlackRock, jumped in, and this is a picture of him on a news show, saying that BlackRock was now only going to invest in companies that had this orientation towards stakeholders. But this, the problem here is when an organization like BlackRock, which is the largest source of investment capital in the United States, private source, rushes to support a particular initiative before it's fully understood and fully spelled out and fully articulated is that the approach starts to become politicized. So people don't understand it. The problems of politicization haven't been resolved. We see a big value creation opportunity, but we still have to make sure that we have the right intervention and people understand its nuances. And that gets me to my final and third major point for today. We have to build a shared understanding of how the intervention that we are proposing actually is going to cause a better outcome.

[32:56](https://youtu.be/JbF-znn48ws?t=1976) This seems simple. We want to show that X is going to cause Y, but there's going to be skeptics. And we have to make sure that we design a process of getting evidence to address that skepticism before we rush to try to scale this thing. Okay, the Zami Lasante ultimately overcame the challenges of scaling. Oak's revitalization is in that process right now, and things are going well. The business roundtable, it has been much, much more difficult. Because of that politicization, people have different theories about the efficacy of stakeholder capitalism in delivering better outcomes for investors, and without it, it's not going to work. And without a shared understanding of how that is going to happen, that skepticism has a life that cannot easily be overcome. In fact, I would argue that the Pi Square project is entirely constructed to address that issue. You know, how do we get the private sector to actually get aligned with what our goals really are for value creation?

[34:08](https://youtu.be/JbF-znn48ws?t=2048) So we have to build a shared understanding. Some of the questions that have to be addressed are, are we sure that Y doesn't cause X? Are we sure we don't have reverse causality? So one source of skepticism might be to say something like the availability of construction jobs is causing people to show up claiming that they want the Canadian dream. So the reason that we have all these people that want to get these construction jobs is because there's a boom in construction, let's say. Or, you know, the availability of medical care on the Haitian plain may cause people who are not sick to show up and ask for medical care. And that's why we get such a rush to the doors. Or we have to make sure we reject these alternative kind of causality arguments. People will come out and say, you know, the availability generally of social support will induce people to want to give up work or worse, to not even try in the first place to work. And we have to show that those problems are relatively readily managed or can be ruled out for some reason. We have to control or deal with the potential for reverse causality. There's a lot more to say about this, but in the interest of time, let me keep going.

[35:19](https://youtu.be/JbF-znn48ws?t=2119) Another source of skepticism may be this idea that a whole series of things that surround X are causing both X and Y. This is called misattribution or misidentification. Here's an example. You know, maybe rising wealth in the city of Toronto is fueling both crime and construction. So we have to rule that out. We have to show that we can reduce crime and have construction increase at the same time. That we can address this problem without actually dampening wealth creation. So that would be, you know, you've got to design some sort of study or some sort of evidentiary process to try to address that kind of skepticism. Here's another potential argument that you may hear. X may cause a whole bunch of stuff, maybe bad stuff, in addition to Y.

[36:20](https://youtu.be/JbF-znn48ws?t=2180) Okay, so this is part of the play pump problem. When the play pumps did work, they pushed water into a cistern, but they also encouraged thugs to dominate the water source. They made the area dangerous. They disrupted cultural routines. They drew in unwanted tourists. They diverted money from better projects. So we have to show that we can prevent all those kind of public bads from surrounding the public good that gets created by this. We have to make sure that we can rule out that X only appears to cause Y. So there's a lot of issues in the public sector around this issue of appearance. So, you know, we could ask this question like, if you look at the bill from within Alliance in Detroit. So the bill from within Alliance goes into an industrial city like Detroit or Minneapolis and goes into the downtown core and starts to support entrepreneurs and business success in the downtown core.

[37:26](https://youtu.be/JbF-znn48ws?t=2246) A question that might come up is, would new business formation in Detroit have increased even without them doing that? So maybe it appears that they had an impact, but maybe they didn't have any impact at all. That would have happened anyway. You know, maybe we have to rule out that HIV rates would have declined in Haiti, even without St. Milosante because more information was available or more care was available in Port-au-Prince or something like that. Maybe the construction training facility would have been successful even without this source of talent coming into them through a special program designed to bring people from this neighborhood of Jane and Finch into the facility here. So we have to make sure we can actually identify the intervention with the precise outcome. I'm going to talk about how you do that, by the way, in a second. It may also be that something that causes X, there may be something in between. X may cause Z, Z may cause Y. We have to make sure we have the logical chain right.

[38:27](https://youtu.be/JbF-znn48ws?t=2307) So, for example, we have to be sure that Tulsa Remote's program to draw remote workers is what's causing tax revenues to rise in Tulsa. Even if Tulsa Remote is drawing in remote workers, so we can see Tulsa Remote's intervention, that's the X, we can see remote workers, that's the Z, can we really show that those remote workers are the source of the increased tax revenue, Y? Going directly from Tulsa Remote to Y, somebody may say, hey, you can't prove to me that it was your thing that caused that outcome. And you have to address that. You have to make sure that people's skepticism is addressed here. And then finally, you know, maybe people sort of themselves. So let's imagine before that all of these dots, both groups of dots, all the dots here, caused outcomes that we can call, that we might call, let's say, let's call them Y1. The outcome variable at time one. Now we see a better Y. Y2 is bigger than Y1.

[39:29](https://youtu.be/JbF-znn48ws?t=2369) But maybe that's because only some of the dots signed up for X. Maybe only the high tax generating dots signed up for Tulsa Remote. So if you look at the tax revenues generated by the people that signed up for Tulsa Remote, they're higher than before. But only high income people signed up for Tulsa Remote. So by only looking at those people, you don't have an apples to apples comparison with before. And that's a source of skepticism. So these are some of the, this is the structure, some of the skepticism that will almost always come up with a public sector intervention. People will say, is X really X? Maybe it's not X. You know, we take police out of an over-policed place and the arrest rate goes down. Is crime down? Or is it just that crime is actually not leading to arrests, right? So you take the police out of a community, the crime rate went down. Is that because the police were being jerks before and over-arresting people?

[40:31](https://youtu.be/JbF-znn48ws?t=2431) Or is that because the absence of the police making arrests makes it look like crime came down when it didn't? So these are the kinds of questions that will come up. You want to build a shared understanding. We need to know that more X will generate more Y and it won't cause other problems. And that means we have to evaluate and assess the potential of X by developing some evidence. So how do you do that? Especially how do you do that before you have your intervention deployed? We have to define a research question. We have to figure out what our claim is going to be. What's the impact of X? How are we going to measure that? How are we going to measure Y? We have to figure out what kinds of evidence people are going to believe. So that's our criteria for effective testing. We have to build a culture around testing it that takes the skepticism seriously. And that's often pretty hard, right? Because people are passionate advocates of an idea. So there's some notes here on why this research is important.

[41:33](https://youtu.be/JbF-znn48ws?t=2493) But the fundamental insight is that we have to answer these questions and rule out, you know, alternative explanations for, you know, what could be going on so that we can sustain a kind of fact based advocacy for X for the intervention, you know, and overcome skepticism that's almost guaranteed to show up. Now, we have to also do this in a way that's legit. So, you know, some types of research are not legitimate. You know, let's say that we want to show that parachutes are necessary to preserve life while jumping out of a plane. The prediction that we want to say is that we should put parachutes in planes. You know, how are we going to show that these parachutes are necessary to preserve life while jumping out of a plane? It's not testable. We cannot. It's a non-researchable question because we can't have people jumping out of planes or having planes discharge people from the doors in the air without parachutes here.

[42:47](https://youtu.be/JbF-znn48ws?t=2567) It's a failed research design. So we have to be sure that the way that we construct our evidence base is actually, you know, legit. So here's some ways to do that. There are better ways to develop evidence about parachute designs. You do a simulation or maybe you do an observational study or a case analysis, but you can't do an experiment. You can't send people out of the doors of an airplane without parachutes and say, hey, they would have survived had they had parachutes. It's not legit. So we've got to choose an approach to evidence creation that's actually going to be legitimate. There are two types of evidence. One's called qualitative evidence. This is more stories. Deep, rich understanding, often developed through interviews or through observations. Or you could do quantitative studies. These are data driven assessments where you get a lot of data.

[43:48](https://youtu.be/JbF-znn48ws?t=2628) They each have their advantages and disadvantages. Advantages of qualitative analysis is that getting stories is often expensive. It's time consuming and it's what we call non-generalizable, which means it's kind of difficult sometimes to draw broad conclusions from a few stories. Quantitative evidence has disadvantages. The evidence, the particular observations may not be comparable. So what's going on for the first person that signs up for the construction training may not be the same as for the 10th or the 100th person. There may be other factors that are interfering, that are changing over time, like the HIV rate in the central plateau of Haiti may be going down as we do our interventions. And so we have to, you know, we have to control somehow for that in our analysis. And it may be hard to get the whole picture through just looking at quantitative evidence. There are a number of different ways of doing research studies.

[44:54](https://youtu.be/JbF-znn48ws?t=2694) And, you know, I will put into the notes some references if you're interested in getting more. Background on this, but cross-sectional research design is where you look, you know, at one point in time, you make comparisons, like, for example, looking at different school outcomes in a city. And this is good for asking questions about different reasons for the outcomes. If you've got some schools generating much higher test scores than others, you can look and do a cross-sectional analysis with some observations, qualitative observations, and say, here's some of the reasons why this school is getting better math outcomes than that school, but that school may be keeping more students in school than the other one. So there may be trade-offs that you can expose through cross-sectional studies. Observational cohort studies either look, they either start now and then follow some people into the future, or they start now and look back at where some people came from. So those are called prospective and retrospective studies. The gold standard in a lot of science is a prospective observational cohort study that's designed as an experiment.

[46:03](https://youtu.be/JbF-znn48ws?t=2763) So here we have randomized controlled trials, or you may have heard of A-B testing or matched samples. And this is state-of-the-art in many situations for causal claims that X causes Y, because what I'll do is I'll take my Xs, I'll randomly assign some of them to this bucket and some of them to that bucket. And then that bucket will get the new innovative approach, and this bucket will get the old established approach, and then we'll look at differences in the outcomes, and we'll be able to say at the beginning, these were totally equivalent buckets of people that each got different treatments, and we got different outcomes, and that proves that one is better than the other. This is often the way that medicine, new medicine is tested. One group of people will get the old medicine that has been customarily used, and another smaller group, often smaller group, will get a new medicine, and you'll look at which performs better, and that'll constitute what's called a randomized controlled trial.

[47:05](https://youtu.be/JbF-znn48ws?t=2825) A retrospective study is an historical study, so if you want to understand really the breadth of consequences of some previous action, then you could use a retrospective study. So you look at differences, let's say, in medical outcomes, and you say, hey, how was this group that didn't do as well different historically, how were they treated historically differently than this other group that did well? So it's a retrospective cohort study. An experimental study, as I've just described, can be interventional, where it's the researcher that's imposing X on one group but not the other, or it can be a case control, which means that there's a random assignment of people across the two categories of the treated and non-treated. So for example, let's imagine that a tornado comes through a particular area, and that you can identify which people in the area where the tornado came through had cement shelters and which one had metal shelters, and you could say, here's my case control, the tornado hit all these people at the same time with the same power and strength.

[48:17](https://youtu.be/JbF-znn48ws?t=2897) The people that were in cement, let's say, did better than the ones that were in metal. Cement is, you would conclude, better than metal for protecting people in that kind of tornado. So you're looking at a group that self-selected into these domains, but whether they picked concrete or metal had no relationship as to whether or not the tornado hit them hard. So that would be a case control experimental design. The claim to knowledge here is that you're offering an answer to the skeptics' question. You're identifying new contingencies, new conditions that would give rise to the effectiveness of a particular intervention. You're talking about its limitations. You're thinking about what other information you need. So winning early comes down to really providing evidence that the idea that you have can be structured, that it will have a big impact to create value, and that you can build a shared understanding of why it works and how it works.

[49:32](https://youtu.be/JbF-znn48ws?t=2972) This is critical to accomplish early in a project's development before you start building organizational structures and start publicizing what you're doing. The goal here is to really make sure that the thing that you want to do is not just going to work and create value, but that people get it, and that you can show that the concerns that they have either can be addressed or don't have a foundation in fact. Okay, let me end there by thanking you for your kind attention. Please check out our website. It's pysquared.burncenter.org. You can sign up for our newsletter at pysquaredburncenter.org. Backslash sign up. We have a podcast at Private Innovation in the Public Interest, and my email address is adottmegann at northeastern.edu. With that, let me stop sharing my slides, and Agata, could you please take a look at the chat and let me know if there are any questions?

**Agueda Quiroga**

[50:39](https://youtu.be/JbF-znn48ws?t=3039) Absolutely. I am stopping the recording right now.

* * *

Original page: https://innovate-us.org/innovating-in-the-public-interest-winning-early · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org