---
id: "workshop:results-framework-executing-change-and-delivering-results"
type: "InnovateUS workshop"
title: "Executing Change and Delivering Results"
url: "https://innovate-us.org/results-framework-executing-change-and-delivering-results"
date: "2024-10-23"
duration: "90 minutes"
language: "English"
instructors: ["Steve Kadish"]
recording: "https://www.youtube.com/watch?v=8BTzvJH9psI"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Executing Change and Delivering Results

InnovateUS workshop · 23 October 2024 · 90 minutes

This workshop is geared for those in the public sector who never stop thinking about what it takes to improve government services to our residents, businesses, and communities. It offers a call to public sector problem-solvers—who are launching a new initiative, grappling with a project that needs a reset, or pursuing a gnarly problem—and must have a comprehensive view from problem-definition to practical results. Join us for a 90-minute virtual training with Steve Kadish, former Chief of Staff to former Massachusetts Governor Charlie Baker. Together, Kadish (Democrat) and Governor Baker (Republican) developed the [Results Framework](https://store.hbr.org/product/results-getting-beyond-politics-to-get-important-work-done/10509) to focus on the necessary but often overlooked parts of good governing: successful execution and implementation in the delivery of public services. In this training, participants will learn about the 4 key concepts in the Results Framework - People are Policy, Follow the Facts, Focus on How, and Push for Results. Participants will discuss examples of how the framework works in reality, learn key tips for successful results, and leave with new tools to tackle complex problems. Participants will gain a better understanding of how to execute change and deliver results—and have a proven method to do so—tailored to the public sector.

## Learning goals

- Learn how to improve government services with a comprehensive approach, from problem definition to practical results.
- Get insights from Steve Kadish, former Chief of Staff to Governor Charlie Baker, and explore the bipartisan-developed Results Framework.
- Understand the 4 key concepts of the Results Framework—People are Policy, Follow the Facts, Focus on How, and Push for Results.
- Gain practical tools and tips to tackle complex public sector problems and deliver successful results.

## Speakers

- **Steve Kadish**, Former Chief of Staff, Governor Charlie Baker

Recording: https://www.youtube.com/watch?v=8BTzvJH9psI

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/results-framework-executing-change-and-delivering-results.txt (plain text) and https://innovate-us.org/results-framework-executing-change-and-delivering-results.json (every segment with its start and end second).

**Steve Kadish**

[0:01](https://youtu.be/8BTzvJH9psI?t=1) Well, welcome, everyone, and a special welcome to a couple people that I recognize. I am very excited to be here. I've really decided in the last few years to devote my time to helping people in the public sector do their work as well as they possibly can. And what I'm going to be talking about today is the results framework that is something that Governor Baker and I had put together and introduce you to the book that the two of us wrote. I will share my screen with you for eight, nine, ten slides, and then come back to you all for comment, share my screen again with you guys for six or eight slides, and come back for comment, and so forth.

[1:06](https://youtu.be/8BTzvJH9psI?t=66) I really like using the chat as a way to have everybody participate or everybody that wants to participate with their comments, and it's very powerful to see what other people are thinking and can add to the discussion. I learn from every single time I participate and lead one of these workshops from the folks that are here in the room, so to speak. I've encouraged everybody to put their name and their position and the state that they work from in the chat so everybody has an idea of who's here. So, while you're doing that, I'm going to now share my screen. Anybody that wants to make themselves visible, I'm all for that, because the body language helps me see whether or not I'm on the mark or not. So, Christine, thank you for coming on, and Julia, etc.

[2:19](https://youtu.be/8BTzvJH9psI?t=139) So, I'm going to now share my screen. I have too much information to present to you. I'm going to ask Agueda to make the PowerPoint presentation available to you both in PowerPoint form and as a PDF so that you could refer to it later. You could share it with colleagues, and you don't have to read and understand everything that's in there. The presentation is dense on purpose that way, in terms of being something that could travel and hopefully be a little bit self-explanatory without having me there. I also want to say, as a caveat, relatively speaking, while I have spent 30 years in and out of the public sector, I'm new to presenting and teaching, and so I would really appreciate your feedback in terms of what's working and what's not.

[3:30](https://youtu.be/8BTzvJH9psI?t=210) So, you guys all ready? I'm going to share my screen, and we're going to fly for the next 82 minutes back and forth. So, I'm going to share my screen now. Can you guys see that? Did you wave your hand or nod? Super. Okay. Okay. Awesome. Okay. Love to start here. This quote's for you. It's one of the really the rudders of the whole effort, the whole book, the results book. A functioning government is a beautiful thing all by itself. Functioning is the key word there, but it's also fierce proof of our democracy. It can diffuse disillusionment and inspire confidence. I hope that you find that true, and I hope you find this inspiring in that way.

[4:31](https://youtu.be/8BTzvJH9psI?t=271) So, there are three things that I want to do in the next 80 minutes or so. I want you guys to have a good understanding of the four key elements of the results framework. I'm going to go through each one. There are tips and tools that I'm going to focus on. These are among the most important tips and tools that I have found to enable success, to actually move ideas into successful implementation, and I hope you find it informative, interesting, and inspiring. So, I start here. I'm on the right, sitting at the desk. Governor Charlie Baker is there on the left. This picture is almost 10 years ago now. It's in the first couple of days of Baker taking office. There's nothing on the walls. There's virtually nothing on our desks. Why do I start here? Governor Charlie Baker is a lifelong Republican. I'm a lifelong Democrat, and when he selected me or asked me to be his chief of staff, it had a lot of people wondering what was going on.

[5:49](https://youtu.be/8BTzvJH9psI?t=349) One, I was a Democrat. Two, I wasn't part of his political team, his campaign team, and many of you know that chiefs of staff often come from the political operation of the campaign. But he and I did have, and do have, a long relationship of working with one another, both in state government and in the private sector, on and off, in different roles, and I really appreciate his absolute, nonpartisan, fact-focused, problem-solving approach to tackling climate change. And so, here you have it. Here you have a Republican governor, a Democratic chief of staff, starting to tackle the issues in Massachusetts.

[6:52](https://youtu.be/8BTzvJH9psI?t=412) In the eight years that Baker was governor, I think there was only one other bipartisan team in the nation, and right now I'm forgetting which state that was in. So, I'd like to start here. This is a quote that we include in the introduction of our book. Let us not seek the Republican answer or the Democratic answer, but the right answer. Let us not seek to fix blame for the past. Let us accept our own responsibility for the future. John F. Kennedy, President Kennedy from Massachusetts. So, the book, there's a snapshot on the cover. When I was stepping down from being chief of staff after almost three years serving Charlie Baker in the Commonwealth, I said to him, why don't we write down the approach? Why don't we put down so that others can take advantage of the hard lessons that we've learned over time and of working together, both from the public sector and the private sector?

[7:59](https://youtu.be/8BTzvJH9psI?t=479) And he agreed, and it took a bit, but we got there. So, this book really is written for you. It's written for people in the public sector. It's not written from a consultant's perspective. It's not written from an academic perspective. It's written by a couple of experienced leaders who are taking some lessons from the public's private sector and translating them to the public sector. Charlie and I used to exchange books, used to exchange articles. We were super hungry for learning how to do things better, and almost all of these were from business. That meant that all of them required a translation from business into how do you have that make sense into the public sector and sometimes the nonprofit sector.

[9:00](https://youtu.be/8BTzvJH9psI?t=540) So, we wrote this book for you guys. We wrote this book for you to be able to tackle difficult problems and specifically to improve services. The book has two parts, the results framework, which you're going to get a heavy dose of today. And there's four chapters relating to each part of the results framework. And then there are four case studies that follow that show how the results framework is applied, was applied. I want to start here in terms of delivering results. You're going to see the word results a lot, but you may have very talented people. And these people, you and your team, may be very fact and data oriented, but that alone, talented people and data are not sufficient. You can get there, but to ensuring that you're going to get there in terms of results.

[10:04](https://youtu.be/8BTzvJH9psI?t=604) Results require understanding how things work, a method, and we're going to talk a lot about methods, and then the experience and the know how to actually make the change happen. I fully believe isn't the right word, appreciate isn't the right word, embrace may be the right word, just the importance of policies and programs as the guideposts of what need to be done. And there's a lot in the public sector literature about policy development and so forth. But the focus here, our focus here is on not the policy ideas, but how do you successfully execute and implement. Four parts to the results framework. People are policy, follow the facts, focus on how, push for results. And they all work together. They all are important.

[11:09](https://youtu.be/8BTzvJH9psI?t=669) You might find one piece or another resonates more with you as we talk about this, but they really do go together and work together, and then iterate upon one another. We always start with people are policy, and what I'm about to say might might sound naive or simplistic, or, of course, or what or what do you mean but you need people to actually get things done. And too many times I've been involved in efforts of all sizes of all kinds, a fix it effort, a turnaround effort, a launch of a new program, and people just start to try to do stuff without stopping and saying, who's going to do it, and do we have the leadership and the team to actually pull it off. And I can't stress enough how important it is to get the people part right at the beginning of major change efforts, the rolling out of new programs to build that capacity.

[12:20](https://youtu.be/8BTzvJH9psI?t=740) I can't emphasize enough to prioritize either filling a staffing gap or filling a hiring need. Boy, do I know how hard it is to hire in the public sector. And I also know and have seen and have helped agencies hire really quickly, really talented people really quickly, and you guys as leaders and managers need to need to spend your time and energy up front on initiatives, building your teams, selecting the leaders, prioritizing the hiring, and then you'll have a shot at having the capacity to move forward. This idea of constructive tension, we've probably all experienced this. I know I have. You're in an organization, new leadership comes in, and too often the new leadership thinks that the current folks are the problem, the current folks don't know how to do things.

[13:27](https://youtu.be/8BTzvJH9psI?t=807) And I know that in every single organization there are good people and there are great current staff doing the work. There are sometimes some bad apples or some folks that aren't as competent. And this idea of constructive tension that I've also found that when you are tackling major problems, major change, rolling out big programs, that the idea of having both new people, new ideas, new approaches, and current staff is super powerful. When you are assembling your teams, your groups, the people effort, I'm going to ask you to focus on both people who are knowledge people, your subject matter experts, and, and, and people who have the know-how to make stuff happen. That could be operational people, tech people, project managers, and change management folks. The combination of the two is super powerful in terms of getting things done. One without the other is off balance, especially in the public sector. I know this is true in every workplace, but it's especially true in the public sector.

[14:51](https://youtu.be/8BTzvJH9psI?t=891) The characteristics, the qualities of trust, humility, and knowing that you do not have all the answers, respect. Respect, respect all the way around, and compromise are essential. There's no such thing as my way or the highway, and for situations like that, those people just do not last. We like to talk about the collaborative gene, and that's either something you have or something that you can develop, but it's the ability to work with other people to succeed and just how important that is. I can, off the record, talk about the examples of working with people at all levels, mid-level directors, commissioners, and so forth, secretaries, who did not focus on these people issues at the beginning of their effort and stumbled a bit, and we've had the greatest success when we've focused on the people issues first, prioritized the people decisions.

[16:03](https://youtu.be/8BTzvJH9psI?t=963) I'm going to talk about this tip, and this may sound off the mark a little bit, again, because public sector agencies are pretty strapped and are operating pretty much full out, yet when you can, and I believe you can do this in almost every situation, I'll talk about it a little bit more in a minute, but create a dedicated team that's focused 100% on the delivery of the effort. It is this startling catalyst to moving things much faster than normally occur. This quote here, simply asking everyone involved to think differently or behave differently will not work. That's from the other side of innovation from Vijay Govindajaran and Chris Trimble, great book. This idea of the dedicated team, dedicated does not mean you're committed to the issue, you really care about the issue. Dedicated is about your time. It may be hard to get folks that are 100% dedicated, but I'm talking 80, 90, 100% dedicated.

[17:21](https://youtu.be/8BTzvJH9psI?t=1041) When I say team, that is not one individual who you've designated as the project person to make things happen. A team is at least two to three people and can be much larger depending upon the size of the effort. The dedicated team must be value-add and people see them as value-add and have the experience and the knowledge and expertise to move things along. It is hard to do this in any organization. I know how hard it is to do in the public sector. I know it is possible to do. We've done it over and over again. I have some notes here that you can refer to later in terms of some of my ideas, in terms of how I've seen to make this happen. But this idea of the dedicated team is not something we in the public sector think about. We typically ask very busy people to do even more and build efforts that take 10, 15, 20% of lots of people. What I'm talking about is grabbing the two, three people that can be the catalyst, the dedicated team, the engine to help pull those other parts and pieces together in a really powerful way.

[18:44](https://youtu.be/8BTzvJH9psI?t=1124) The second tip are some of the hard lessons I've learned about working with consultants. I'm guessing all of you have or will have had experience of working with a consultant or managing a consultant. I believe that consultant expertise is essential, critical to helping us in the public sector. But it's so important to understand what it is you're asking the consultant to do. Different consultants, large and small, are better at some things and not so good in other things. I've used consultants to help with operational issues, technical issues. For me, not so much strategic issues, although they do that plenty. Helping understand how other organizations work, bringing in resources, et cetera. Super valuable. The value of a consultant is that you can get that expertise, get that additional resource relatively quickly as opposed to doing a lot of hiring. But understand what it is you are looking for the consultant to do. Be super clear, develop a scope and a time frame, and be really clear about what the deliverables are.

[20:14](https://youtu.be/8BTzvJH9psI?t=1214) One of the things about working with consultants as opposed to staff, it's staff that just takes longer, but you actually have the responsibility. If somebody is not working out as well as you want on the consultant team, you're the manager of the taxpayer money. You should ask that consultant to find somebody else that can come in and give you what you need. That's acceptable. I know you'll all be fair about doing that and not snap judgment. But one of the opportunities that working with a consultant, a larger team gives you, is the opportunity to switch out people that may not be quite as strong. Managing a consulting contract is not the same as managing staff. Even if it's a consulting firm of one or two or 10 people, they have to have their own center of gravity, which is different from yours. It doesn't mean that you can't and shouldn't be aligned, but it's different and you have to appreciate those differences. There's some tips in terms of hopefully succeeding at a consulting engagement. I know that you guys either have or will. I'm going to go back to full screen now and ask for you guys to put into the chat your thoughts based on your experience. What do you think is important for selecting leaders and building teams? How do you think about this people or policy concept that I just discussed? I went through this bunch of things, but I'm really interested in gaining the knowledge from you all in terms of what do you think is important for selecting leaders and building teams? Jot your note into the chat, please.

[22:14](https://youtu.be/8BTzvJH9psI?t=1334) Awesome. Aztec, thank you for being the first. Honesty. I'm going to go back to full screen now and ask for you guys to put into the chat your thoughts based on your experience. What do you think is important for selecting leaders and building teams? How do you think about this people or policy concept that I just discussed? How do you think about honesty? Awa, great comment. If you see a comment you like, give it a thumbs up or a heart. These are terrific.

[23:25](https://youtu.be/8BTzvJH9psI?t=1405) They're terrific. I'm going to ask a couple of you to comment on what you wrote.

**Audience**

[24:02](https://youtu.be/8BTzvJH9psI?t=1442) Hi, this is Amy Rainer. I actually show up as Mary Joyce because she gave me her link because I couldn't find mine. Thanks for for hosting this. It's great. So far. My comment was around hiring people that aren't like you. I had to learn that having gone through, you know, some, some various Myers Briggs type things and I realized that the people that I was drawn to were just like me. Right. Kind of high energy, fairly assertive, move fast. And I realized that when I had other people on my team that were the quieter, the more thoughtful, the thinking through and almost the warriors, like I needed those people to because I wasn't doing that myself. And so I always try to remember, I think we're all drawn to people just like ourselves. I try to remember that you have to have all of the other skill sets and all of the other types of folks on the team, or you're not going to get anywhere.

**Steve Kadish**

[25:05](https://youtu.be/8BTzvJH9psI?t=1505) Awesome. Thank you, Shiva. Would you mind commenting on your comment empathy which helps leaders understand where others are coming from that you had four thumbs up.

**Audience**

[25:18](https://youtu.be/8BTzvJH9psI?t=1518) Yeah, I, all the leaders who I've followed without question have had a lot of self reflection and ability to be empathetic in an authentic way and so I think if I see a leader modeling those traits, then I'm willing to go down any path or journey with them.

**Steve Kadish**

[25:41](https://youtu.be/8BTzvJH9psI?t=1541) Thank you. Now, sir L diversity of thought ability to be both a role player and a leader, talk about that.

**Audience**

[25:49](https://youtu.be/8BTzvJH9psI?t=1549) Yeah, I thank you for the opportunity diversity of thought just we don't want all people that agree with each other we want people that can articulate their, their visions and have respectful arguments with each other. I don't want everyone just to say yes or no I want to have, you know, conversations, thoughtful conversations about solving problems. And so you have to have people that have a diversity of thought diversity of experience to bring those different attitudes and assumptions to the table. And then people need to be a role player and a leader, is that, you know, to be a leader you have to know how to be part of a team you have to know how to follow, and sometimes it takes people to step up to be a leader to to move things forward so you have to be able to do both.

**Steve Kadish**

[26:34](https://youtu.be/8BTzvJH9psI?t=1594) Henry, you wrote demonstrated knowledge of process and bureaucracy.

**Audience**

[26:39](https://youtu.be/8BTzvJH9psI?t=1599) Yeah, I think this is something I'm encountering right now which is to be able to, you know, work on the margins of a system and understand how to get things done. You need people who are both really creative and are willing to push the boundaries and push bureaucracy to bend. But you also need people who know how to execute on getting contracts through and procurement past so the balance of those two you can't have all of one or all the other, as I am rapidly learning.

**Steve Kadish**

[27:16](https://youtu.be/8BTzvJH9psI?t=1636) Well said great transition. Super comments. I'm going to ask a greater to grab all of the comments and then take your names off them but and then we'll, we'll share the comments. Either later today or tomorrow to all of you. I'm going to go back to sharing the My presentation, super stuff. Okay. So, results framework. The first item we always start with is people are policy. The second is follow the facts and why do you follow the facts. I've experienced this and I'm sure you guys have to lots of times people jump to what needs to be done without taking the really important time to understand what happened. What are the facts and and and then therefore, what are some of the possible interventions.

[28:22](https://youtu.be/8BTzvJH9psI?t=1702) So you follow the facts to define the problem and which can also point to what some of the key interventions are an intervention could be a fixed thing or it could be a positive expansion of a program. There are two kinds of facts that I found really critical data evidence or some call it hard data and points of pain, which can be referred to in some cases as more qualitative data. Or is part of what is called qualitative data. I'm a super data geek. And for the first 1015 years I still do it, but I just devour things that I can read Reports program descriptions. Budgets or charts more reports. If there are performance metrics diving into the performance metrics, trying to learn how other organizations do similar services and so forth. And I really believe that I know this is true for Governor Baker earlier on in our careers is that It was this belief that if we worked hard enough at understanding the so called data evidence, the hard data that we would know what know what the problem was and therefore what the interventions were. And again, this may sound naive or like sure took you guys a long time to get there, but I really came to appreciate it almost flip this but really came to appreciate that the data evidence is absolutely critical.

[30:00](https://youtu.be/8BTzvJH9psI?t=1800) Absolutely critical so that you have a grounding in the information that's available, but to understand what's really happening in the real world, you need to be listening to and speaking with your customers, the clients that you're serving, and super importantly, from the staff and the other providers that are providing the service, and then finally also media reports. And so the combination of the two, the data evidence and the points of pain, I find are really important. One without the other will not give you a full picture. You can't operate just on stories, and you can't operate just on reading reports. The combination is very powerful to help build out what a picture is of the facts to follow. So following the facts, the problem to solve, you start with the facts, you look at your data evidence, and you look at your points of pain, and I say you, it's you and your teams that are doing this.

[31:13](https://youtu.be/8BTzvJH9psI?t=1873) And what you're trying to do is to get a common understanding, a common description of what's happening with the service and the program. And then from there, you're able to elaborate on and enumerate specifically what are the problems and then what are some of the opportunities. And there's a couple of phrases that you're gonna hear from me a lot, or I hope that you take with you is always ask why. I was in a conversation yesterday with the chair of a board of a university, and we were talking about what was happening in higher ed. This morning, I sent him an article that said that in higher ed, there was a 5% decrease in the, a startling decrease, 5% decrease in the 18 year olds that were going to school this year. And I sent that article to him, and I had a lot of theories about it, but so I asked him why, and I'm gonna, and it's this, and he, and so he had a different explanation, which I'll talk about it in a second.

[32:29](https://youtu.be/8BTzvJH9psI?t=1949) So asking why is really important. So in defining the problem, you're not boiling the ocean. You're not trying to solve everything all at once. You are narrowing the problem and or the opportunity into something that can be solvable and can be done. That's meaningful. You're breaking down the larger problem into doable pieces, one. Two, we don't do this enough. We don't spend enough time, and I don't know if this is true, as much as true in the private sector, but in the public sector, we don't look enough to see what's working and emulate it. Do more of what works and do less of what doesn't. And then in three, sometimes it takes a step back to get a full view and not just the narrow view. And here's the punchline of the story of the chair of the board that I was speaking with.

[33:30](https://youtu.be/8BTzvJH9psI?t=2010) I had a whole different set of theories as to why the number of 18-year-olds had dropped so significantly going into higher education. And he wrote back, well, FAFSA, the Federal Financial Application Form was such a disaster this year that many, many students and families couldn't figure out what to do in time and nor could the schools respond. And so that was not on my radar as an item, but it was an example for me of stepping back, getting a fuller view and asking why in terms of how to tackle something. So you're following the facts to get at a problem to solve that is solvable, doable, and meaningful. It is so important right from the beginning of an effort to establish your performance metrics of success.

[34:35](https://youtu.be/8BTzvJH9psI?t=2075) This is also something that is many times put off till later, but part of what you're doing in the follow the facts is you're defining the problem and you're beginning to enumerate and describe what some of the solutions are. And innate in that is therefore, these are the metrics that we wanna measure to see if what we're suggesting actually makes a difference. So set your performance metrics at the beginning of the effort. They can change and they should change, and then you'll be learning throughout. Here's a tip, and there are tips throughout every chapter in the results framework, in the results book. So I love this tip. I actually like all of the tips, but this tip is something you could use this afternoon. Call it round the table. I use it in two different ways.

[35:37](https://youtu.be/8BTzvJH9psI?t=2137) I first use it often at the beginning of an effort, particularly around points of pain. And with a group, it could be a commissioner and her staff. It could be a bunch of providers. It could be key staff. The thing about round the table is everybody speaks. We are so often in meetings where a couple of people speak, our eyes and our body and all of our energy goes to the person in the room with the most authority. And then everything bounces off that. Round the table is the great equalizer. You are literally going around the room. Everybody has an opportunity to speak, ask a question, make a comment. And the dynamic is super different than a regular part of the meeting where there's a different kind of presentation and discussion.

[36:40](https://youtu.be/8BTzvJH9psI?t=2200) And it may take more than one time, a couple or three times in doing the round table for people to actually believe that they are okay speaking, that they're okay speaking in front of their boss's boss, that they're okay speaking with somebody else in the room. But it is super powerful, super enlightening and super empowering. So I use this, as I said, most often at the start of a project or initiative to get at points of pain. I also use it with projects and rolling out initiatives as part of the regular project review. And so here, you've been through your meeting, reviewing your project deliverables, performance metrics, other policy stuff. Building into the meeting agenda around the table, again, allows everybody to speak.

[37:46](https://youtu.be/8BTzvJH9psI?t=2266) They can ask a question. They can thank somebody for something. They can say, I heard you, but I still need this. They can ask a question about, so what happens next with communication, et cetera. Super powerful tool to elicit more honest thinking and perspective in terms of what's going on. So I use the round table in both of those ways. You guys could use it in those ways and others as well. The second thing, I was speaking to somebody yesterday and we were talking about common sense and he said, common sense isn't. And so I thought, okay, I need to think about that. But so this is kind of common sense, I think, but we organizations tend to believe that they have the answers within the organization, every organization I've ever been in. And it's sometimes true, but boy, is it limiting.

[38:47](https://youtu.be/8BTzvJH9psI?t=2327) And what I'm doing with the tip for you to get out of the tower is asking you to incorporate in your thinking and in your team's thinking. It's breaking the internal information bubble, breaking the usual channels of how information comes in and hearing and learning and growing and seeking what else is happening outside of the organization related to what you're doing. A whole variety of different ways to get out of the tower, literally getting out of the office and talking to all kinds of people you're working with, suppliers, advocates, especially customers. Go see other people's reports, other organizational descriptions. If there's comparative performance stuff, that's super powerful. Talk to the folks that are actually doing the work, staff and providers, and take the role of the customer. Very powerful, simple.

[39:48](https://youtu.be/8BTzvJH9psI?t=2388) It's a mindset. Once you get into this mindset, you'll appreciate what you have internally, but then always be asking, okay, so what else is happening else out there? How do I get out of the tower? How do we in our team get that information? So I'm gonna go back to you all now. Follow the facts equals problem definition. What facts do you seek to understand the problem? Where do you find key data evidence? How do you seek points of pain? So Agueda, if you would put this into the chat, if you guys would respond into the chat, what facts do you seek to understand the problem? Data, evidence, points of pain.

[40:51](https://youtu.be/8BTzvJH9psI?t=2451) Yeah, keep going, guys, good start.

[41:52](https://youtu.be/8BTzvJH9psI?t=2512) Great comments. Great comments. Nicole, you wrote, user testing is helpful in order to make sure the documents and programs we create are useful for the people who will use them. Talk about that, that's interesting.

**Audience**

[42:28](https://youtu.be/8BTzvJH9psI?t=2548) Yeah, thank you, Steve. One of my colleagues and I have really been like spearheading some new initiatives. We're creating this guide. We work for the Department of Labor in New Jersey to help explain to worker advocates how to submit wage complaints with like more information, more complete, so we can follow up on them more effectively. And we thought from the beginning, it'd be really helpful to do user testing on that guide. We create something and give them something to respond to, to make sure it's something that's useful for them so that as they're taking intake from different kinds of workers and submitting complaints to our office, it's something they can use as a training guide. Like we set up sessions, like listening sessions to go through the guide with them and understand if it was useful, if something needed to be clarified. And I think it really helped save time in the long run because instead of creating something fully formed, we were able to really kind of target that need and make sure what we were creating addressed the problems and was understandable for the general public, the audience we were trying to serve.

**Steve Kadish**

[43:28](https://youtu.be/8BTzvJH9psI?t=2608) Hello, Jeremiah, I'm trying to find your comment now. You wrote, where else is this happening and how are they handling it? So talk about how you do that.

**Audience**

[43:40](https://youtu.be/8BTzvJH9psI?t=2620) Yeah, I mean, I think that's coming most directly out of the work that I did with many of the folks on this call on COVID, whereas I think especially where you have a problem that's a lot of folks are suddenly dealing with at the same time. I think there's a lot of benefit in seeing how people are defining their, they rhyme, right? Like states have a lot of problems that rhyme. They're not exactly the same everywhere, but like they're a themes, right? And so I think that's a big part of it. They're a themes, right? And I think you lose a lot of the benefit of other people's experience if you don't. I think this is partially a research exercise, but honestly, for myself, like I will just make calls until I get somebody that I can talk to. And then you build up a reservoir of people that you can reach out to and that will also reach out to you. And they'll tell you about their problem that it's like, you know what? I'm also having that problem and now you've already like helped me start to solve it by just reaching out to me because you're having it too. So I think I'm a big believer in getting the benefit of other people's experience.

**Steve Kadish**

[44:44](https://youtu.be/8BTzvJH9psI?t=2684) Thank you. You know, one of the big differences between public and private sector is that folks in the public sector, for the most part, will talk to you about what they're doing and what worked and what didn't work. And so that's a term I just spoke about is a great example. Getting out of the tower and building your networks, networks of support, help, advice, as you go on, that's also super important. Rebecca, you wrote population-based data. I want to understand the people our work is impacting. Talk more about that.

**Audience**

[45:21](https://youtu.be/8BTzvJH9psI?t=2721) Yeah, I think primarily I think it's best to have a good landscape and idea of who we're serving. So I want to know, for example, if we're trying to push out particular guidance, we want to make sure that we're touching people who this is impacting, if it's their language spoken at home, if this is impacting their geographic region, who is this really, who should we be reaching, and how should we be best reaching them? And I think pulling some of that data allows us to understand how to meet the people where they're at, rather than having them come to us.

**Steve Kadish**

[45:54](https://youtu.be/8BTzvJH9psI?t=2754) Great, thank you. Great comments, you guys. Again, this will be part of the comments in the chat will be part of what we share. I'm going to go back to my deck. If anybody wants to thumbs up or heart any of the comments you see in here, please do, because that's also super helpful. Okay, I'm going to go back to sharing my deck. Okay, so the first part of the results framework is people are policy, right? The second piece we just spoke about is follow the facts. And the third piece, focus on how, if you put your eyes to the bottom of this slide, this is where you will spend most of your time, you should. This requires the absolute majority of your time, your mental energy, the people that are involved, the resources, and so forth. The other two pieces, people are policy, follow the facts, are super important to set upright. But this is where you are moving from problem definition to actually implementing results. It's the structure to move from fact problems to meaningful results.

[47:20](https://youtu.be/8BTzvJH9psI?t=2840) Two pieces here, what to do and how to do it. And think about what is directly coming from the follow the fact piece that you have a problem. If you did this change, this new program, this improved performance, this expanded coverage, then this result would occur. And so it is the promise, the implicit promise of a policy change, a programmatic change, a technology change, more money, a change in the legal or regulatory structure, that by doing these things, it will make this occur. It will make this problem get, it will decrease the incident of this problem, or it may increase the availability of a service that you're trying to get out there. So that's the what, the announcement, the announcement of the activities almost. The how to do it is the method. And we've spoken about putting together the people piece, the leader, the team, the dedicated team. I'm a huge, huge believer in project management at its simplest level, and then for more complex efforts at its very complex level.

[48:51](https://youtu.be/8BTzvJH9psI?t=2931) But I like simple, the simplicity and the accountability of project management. I'm going to talk about that. I also want to talk about the importance of time versus scope. And I'm going to go through some examples. I want to introduce you to the concept of releases, although you already know it, and spend a few minutes on Agile Scrum, for those of you that don't know it. And I'm not going to talk about business process analysis in this piece, just from a time point of view. So this first tip, flip the work management triangle time, time drives results. What I mean here is the work management triangle has three elements, scope, which is the set of things you are planning to do, time, the amount of time that you've allocated to it. And the resources are people, money, space, tech, et cetera, that are involved in actually helping to do the delivery. So those three items work in a kind of a triangle. They can't get out of sync. You can't have a huge deliverable scope and very little time, or you can't have a very short amount of time and expect that you're going to hit your deliverable without a lot of resources they iterate. And again, I thought I was a pretty good project manager. And for the first 15 plus years, I would start every single one of my most complicated efforts. And some of the simpler ones by focusing on scope first, what it was we were trying to get to do and spending often it was not just weeks, but months

[50:56](https://youtu.be/8BTzvJH9psI?t=3056) with a lot of people developing in detail what was going to be accomplished. And these efforts were often a year long or several years long, had little to show along the way. And honestly, my success rate was about 50% when that stung. And in the literature, it said that project management overall probably had a 50% success rate using the so-called waterfall method. About 15 years ago, I became aware of other methodologies that started with time first related to agile and related to release. And now, as opposed to saying, what is the full set of things that need to be done? We start our projects by saying, by setting the first time target of typically eight to 10 to 12 weeks and put the teams together to focus on delivering meaningful stuff in that period of time as a release and then following it with another release and then following it with another release. It's a way to tackle big problems in chunks.

[52:18](https://youtu.be/8BTzvJH9psI?t=3138) It's a way to have incredible focused energy on particular items, address them and move on. And I have further come to appreciate the power of so-called Agile Scrum in this release-based concept. So, this slide is really a one slide cartoon of Agile and Scrum, Agile Scrum. I'm going to talk about the items on the bottom just as an introduction. So, I just spoke a lot about speed and focus and getting things done. And yet, the first words you see here on the lower left-hand side are product backlog. Now, backlog in my terminology typically is a bad thing. It means that things are stuck. But with Agile Scrum, backlog is referring to the list of the key deliverables that you're trying to get done. The sprint backlog refers to a selection from the product backlog of those items that you are tackling in the sprint. Moving slightly to the right, the sprint is typically two weeks, 14 days, including weekends. And there's often a 24-hour productivity cycle. And then the very last box on the right is that at the end of the sprint is hopefully the accomplishment of some of the key deliverables. And it's also a chance to do a sprint retrospective, a look back at what worked and didn't work as you were about to plan your next sprint. So, in pulling together the concepts of the release in Agile Scrum, I put together these steps to help make the concept more

[54:21](https://youtu.be/8BTzvJH9psI?t=3261) real. And I'm going to go through a case study which will hopefully make it even more real. But releases in Agile Scrum leverage the power of time and project management. It's this really neat set of things that goes together. First, you're setting your overall time frame, your release time frame. I suggest 8 to 12 weeks is a good amount of time. It's not so short, and it's not so long. You're determining what I'm referring to as core work streams. These are thematic topics that include the critical problems to address and the essential deliverables. And you'll see some examples coming up. But you might have on really big efforts, you might have 2, 3, 4, 5 of these work streams. Sometimes you only have one or two. And each work stream has its own group, its own team, its own energy, its own scrum. And the third item here is that there's the staff assigned to support the work streams, ideally a dedicated team to the work streams and to the overall effort. You're breaking the overall work that you've set out in this 8 to 12-week period into two-week sprints, the two-week agile sprints. You may be holding a daily scrum, a short meeting where everybody in the group answers these three questions. What did you do yesterday? What do you expect to get accomplished by tomorrow? And what are the impediments for preventing this to happen? There's a lexicon and a scrum leader that could be involved in the agencies that I've done, the public agencies that I've done agile scrum with.

[56:16](https://youtu.be/8BTzvJH9psI?t=3376) We did not meet every day. The work streams met usually at least two times a week, more often three times a week, sometimes four. Holding a leadership review is really important where you're continually having the work at the beginning and the end of the sprints and at certain key points along the way, reviewed, problems raised, hopefully resolved, and celebrate the accomplishments and begin the next release. Here's an example of, in Massachusetts, of the work of the turnaround at the Health Connector. The Health Connector is the entity in Massachusetts which provides Obamacare in Massachusetts to those individuals who don't have either private insurance or Medicaid or other related public insurance.

[57:21](https://youtu.be/8BTzvJH9psI?t=3441) It had a disastrous two years preceding this in its open enrollment. It had many problems. This is after Massachusetts being hailed as one of the founders, really, of the concepts of Obamacare. Here is a one-page snapshot. The line going across the middle, April to November, represents time. The turnaround effort began either in March or in April. Here's a picture of releases. What you see here is release five, above the line, not quite two months, release six, a little more than two months, release seven, about two and a half months. Within each of these releases, the Health Connector, its issues were hugely about the technical system and hugely about its operations. It was both. The words that you see here probably don't mean anything to anybody except for perhaps Jeremiah, but they are the list of the key things that were going to be accomplished technically and operationally in a release.

[58:44](https://youtu.be/8BTzvJH9psI?t=3524) When you roll out a release, one reason I like to show this slide is that stuff happens. Stuff doesn't work out the way that you anticipated. The Health Connector team developed what they called a fix-it release, a 5.1 release or a 6.1 release. It was the acknowledgement that you roll something out and stuff didn't work the way that you thought or there were some other improvements. You have a small release, just a few items, as a catch to fix those while not interrupting the integrity of the releases that follow and the time that follows. Here's a graphic picture of what a release looked like for an enrollment. This is one of the case studies in our results book. So I guess I'll do this. I'll ask you guys, and if you could put this in the chat, Aguedo, but I'm interested in what you guys think about leveraging time in the release concept because it is, some of you are already using it.

[1:00:00](https://youtu.be/8BTzvJH9psI?t=3600) Some of you are probably thinking, I don't know what to think about this, so I'm interested in your thoughts and questions. So if you put those in the chat. That's really interesting. Yeah. Other comments, you guys?

[1:01:08](https://youtu.be/8BTzvJH9psI?t=3668) Somebody had to get it in there, though. It's I think it's an advance on that Jenny, your strategic doing approach, but it's some similar similarities. Okay, any other comments. I'm going to go back to sharing my screen. Thank you. Thank you for your comments.

[1:02:09](https://youtu.be/8BTzvJH9psI?t=3729) Oops. So results framework, people are policy, follow the facts, focus on how in the final element of the final. The fourth element is push for results. And you could have great people, great data, strong methods, but here you're actually measuring. Did you get it done? Are you making a change? Is it really happening? And I love this quote from John Wooden, never mistake activity for achievement. So three things. Measure what matters. I like to look at performance metrics. And with project status is looking at your red, yellow, green to see really about the how are the elements coming together in order to succeed on the succeed on impact. It's so important to discuss these performance metrics and the project statuses in in meetings of leadership of project leadership.

[1:03:18](https://youtu.be/8BTzvJH9psI?t=3798) For learning, for problem solving, for doing. And if there's a mantra here, it's you're measuring on this measurement. You're doing an evaluation about what's working and not working. And then you're doing some of the real hard stuff and doing some adjustments. You may be adjusting people, you may be adjusting policies, you may be adjusting tech stuff, etc. And then you're measuring again. So it's this idea of continuous, continuous measurement, evaluation, adjust is super important in leadership reviews. You are either doing this, you are either in a position to do this now or as you grow as leaders. I have found it incredibly important that as leaders, you model and create the environment where it's comfortable to talk about the uncomfortable bad stuff.

[1:04:26](https://youtu.be/8BTzvJH9psI?t=3866) And that's really hard to do to folks you're reporting to. And so therefore, it's really a almost a responsibility of you guys as leaders and managers to help create that for your teams and your staff. So the two things that I think are important to talk about in leadership reviews of project management, whether something is meeting its deliverable goal and performance metrics, the project management. Green means I love green, yellow, red loves probably too strong of a word. I really think that green, yellow, red is a very effective, simple way to understand what's happening. Green means the deliverables going to happen on time. Yellow means the team or the individual responsible for the deliverable believes it's going to happen on time, but they're waving the yellow flag, so to speak, to say, we've got some issues.

[1:05:27](https://youtu.be/8BTzvJH9psI?t=3927) We think we can resolve them, and we're just letting you know. Red is really, really hard for people to report to their bosses about. It's really hard for people who are used to making things work, figuring out how do things work to indicate. And red is a call for help. It is not a signal of failure. One reason I like performance metrics is they create the opportunity when you're looking at the data, the numbers and the narrative related to the numbers, to move away from the individual that's doing the presentation and focus on the information that's being shared. Again, another way to help your teams be comfortable talking about hard stuff. The Secretary of Human Services, Mary Lou Sutters, who some of you worked with, was maybe the best so what person that I've worked with, I think maybe besides Governor Baker.

[1:06:42](https://youtu.be/8BTzvJH9psI?t=4002) But in looking at the data, in looking at the narrative, it wasn't enough just to look at it. It was the so what. So why are we looking at this? What is there to understand about what's happening? And what do we do to need to consider change? So I said I wanted you to always ask why, and I always want you to ask so what in a very polite way. So what? So what's the point? What's the point of us spending time looking here and so what can we do about it? Here's an example again from the Health Connector of a simple one pager. I've used this a lot for complex efforts is to have on a single top level executive dashboard, so to speak. How is the project doing on its deliverables? And there are some maybe 12 items here. It's less important that you understand what each of the items are. They were super important for the Health Connector turnaround effort.

[1:07:51](https://youtu.be/8BTzvJH9psI?t=4071) But what's powerful about this, and the reason I wanted to share this with you is that there was only one item that was green. Only the call center was on track to hit its deliverable due date. Five yellows, five reds. It took a lot of guts for the teams to be reporting to their executive director and to the governor's office that this is where things stood, but very powerful way to create the honesty about where things were at. And then in turn, where we needed to really focus performance metrics. I'm going to blast through some examples. I believe there are two different ways, and I'm not talking about individual HR kind of performance or management reviews here.

[1:08:51](https://youtu.be/8BTzvJH9psI?t=4131) I'm really talking about customer impact performance, and I think about that both as the service performance, the output, and then some of the critical inputs that enable success, like staffing technology, et cetera. So before I show you a few examples, I want to stress that please take great care when somebody presents an average, because averages hide what's going on, and there are examples over and over again of this. When you are reading a newspaper article, or you're listening to somebody talk, or you're looking at the materials involved with the effort that you're involved with, really ask people to dig into what's behind the average and chart out the actual individual customer experience. This simple example of if you have a bunch of 90-degree days and a bunch of 40-degree days, the average is a very comfortable 65-degree days. No, it is not. It's either way too cold or way too hot. But on average, it looks okay.

[1:10:03](https://youtu.be/8BTzvJH9psI?t=4203) Back to the health connector. This was six weeks into the turnaround effort. These were a couple of slides that were presented to the board, some metric slides. Here's an example of call center performance. The orange towers represent January, February, and March. On the left-hand side, it was how long it took to answer calls. The vertical axis, in February, for example, the tallest item in the average speed to answer calls was 1,600 seconds. That was over 20, 25 minutes just to answer the call. No kidding, one of the results was that when you looked at the abandoned call rate in February, 40%, four out of every 10 calls were abandoned.

[1:11:12](https://youtu.be/8BTzvJH9psI?t=4272) A whole lot of work went into both some systems work, more staffing, training, et cetera, to tackle this to make sure that when people called in to the health connector, they were getting their critical answers resolved. Another example of a kind of a metric, a way to track what's happening with urgent issues, and many of you have to do this, here's a spot in time from April to June in a two-month period. The blue line shows that the number of urgent cases was still increasing by 600, but almost two to one, those numbers were being resolved and were being on track to being eliminated or handled in a reasonable amount of time. I show you this as an example of you do a major release and what are your performance metrics to see whether or not the release was successful.

[1:12:13](https://youtu.be/8BTzvJH9psI?t=4333) Here, as a result of release five that I showed you, a huge change started to occur in the amount of issues that were related to eligibility verification. It was a proof, so to speak, that the changes that were made in release five worked. Then in the bottom here, there's actually a target to get rid of all of the aged eligibility verification items. Another example of presentation of metrics, almost a dashboard, this was also during COVID, and this was related to the quick ramp up of rental assistance to prevent evictions in Massachusetts.

[1:13:16](https://youtu.be/8BTzvJH9psI?t=4396) Here is a weekly chart that we reviewed in our leadership teams. It shows the 11 different agencies that are doing the application reviews. It's a process review, and two things. The first column, after the name of the agency, average age of the applications approved, you get an idea that which of the agencies are performing pretty well and which are taking a long time, but that's average. One of the things that I insisted upon is that we knew how long the oldest application was that was at that agency, and we're continually trying to work that tail end, so to speak. Finally, this is related, also during COVID, to the contact tracing initiative in Massachusetts, a dashboard that we looked at every single day.

[1:14:17](https://youtu.be/8BTzvJH9psI?t=4457) The two colorful graphics really show the extent of the number of contacts, a number of cases and contacts that the contact tracing center needed to reach out to. The graph at the bottom talks about the effectiveness of the calling operation in terms of reaching people, and on the right-hand side, you have outcomes, and then below that, you have CTC staffing. The outcomes our target was trying to reach, 95% of those who tested positive for COVID and 95% of those contacts within a 24-hour period. Our period, our success rate at this point was 84 percent and that's largely due that we didn't have enough staff at that time so we were tracking both. So that's a ton on performance metrics and I show you these and you'll have them by way of example.

[1:15:22](https://youtu.be/8BTzvJH9psI?t=4522) They almost all have a graphic and a narrative which I think is important. And so I show you the performance metrics because this is how you can tell whether or not your service is improving or your new program is rolling out with success. In the next six or seven minutes I'm going to blast through this case study because I think it will help pull a variety of things together and I'll make sure you're done by 3.30, 3.29. So I'd like to start here, Baby Doe, and this is another case in the results book. On Thursday, June 25, 2015, a woman in Winthrop, a coastal town just north of Boston, made a horrid discovery. She was walking her dog along the shoreline of Deer Island when she found an object that concerned her and called 911. The Massachusetts State Police uncovered the remains of a girl of toddler age. This was a child found mutilated in plastic bags. The news was stunning. As a community we had not yet recovered from the loss of Jeremiah Oliver one year earlier. This little boy was found also mutilated in a suitcase along the side of the road. Now this hunt for more information about Baby Doe drove national news stories throughout the summer. What had happened to this innocent child would hang like a dark cloud over all of us. Baby Doe. Who was Baby Doe? Who was Baby Doe? Her name was Bella. This is the Boston Globe front page, September 19. The picture on the left is the police caricature of what they thought she would look like based upon the information that they had. The picture on the right was an earlier picture of Bella. By now,

[1:17:23](https://youtu.be/8BTzvJH9psI?t=4643) by this time, the turnaround effort had started at the Department of Children and Families. There were two other incidents previous to this that were tragic. Governor Baker worked with Secretary Sutters to work with Commissioner Spears at the head of the agency to begin the turnaround. The turnaround, this is, I'm trying to put some of the elements on one page, but the first two items were people items. Getting the teams, getting the leadership team in place, getting the dedicated project team together. The next two items were around facts. We spent a lot of time around points of pain. We did multiple sets of roundtables with different groups. The team did a very deep dive into data, a ton of data, trying to make sense out of the data, and also got out of the tower to look to see what other agencies were doing and what needed to be mimicked and improved upon. Two on people, two on facts, then items five, six, seven, eight, nine. The lion's share of the effort was how. This is where Commissioner Spears, the head of the agency, said we are going to do release-based work. She created the five or six different work streams that you see here, policy and practice, people, management structure, and so forth.

[1:18:50](https://youtu.be/8BTzvJH9psI?t=4730) I'm going to show you something, a tool that we used. I call it a checklist to green. That tip is included in the book. I'm going to show you an example of it. Commissioner Spears embraced Agile Scrum and the release concept. We used project management for just about everything, red, yellow, greening things, and did leadership reviews twice a week. As a part of almost every single leadership review was around the table at the end to uncover items. DCF took the Scrum terminology and created their own vocabulary terminology sheet. Here's an example of that. They ended up doing very short. They were like an hour trainings to about 200 people to get social workers to use Agile Scrum. Agile Scrum is what technology people and engineers use, not social workers, but here social workers did. Here's a picture of what the releases look like from a time point of view, the dark blue lines dropping down. Release one was when Commissioner Spears set for just around Thanksgiving for the first set of activities to be done. Then around March 18th, St. Patrick's Day for release two. A further deeper dive into the sprint calendars is the activity that occurred during the two-week sprint. If there are five sprints in release one, this is the activity that was happening in weeks one and two repeated five different times. If you look at the Monday in week one, there's a planning piece. If it's the second release or third release, there's a retrospective. Twice a week, the SOS sprint review. SOS meant Scrum of Scrums. It was

[1:20:57](https://youtu.be/8BTzvJH9psI?t=4857) the agency leadership team meeting twice a week to review what was happening in each one of the work streams to keep progress going. There's further examples of the language and the roles of people down on this page. Here's an example of what I call getting to green. I've been involved with a lot of efforts to launch large initiatives. This is a tool that helps get things off to a strong start. It gives you a better shot at succeeding. They call it a checklist green. As I said, there's a checklist green in more detail in the book. Here, DCF had a shortened version of one. Be clear about the objective. Be clear about the scope of work, what you're trying to get done. Be clear about what the work was going to be done in this sprint. Be clear about all the activities that were going to be done in the full product backlog release. They wanted to make sure that they had strong communications and engagement for everything they did and metrics and then overall status. Here's the people work stream example of the checklist green. Their objective, first and foremost, identify, post, and fill critical hires. This was an agency that had a hard time making a single hire within months. They were now being asked to hire hundreds of people as quickly as possible.

[1:22:42](https://youtu.be/8BTzvJH9psI?t=4962) This whole effort, when this was shown to the leadership team, and if you look at the status notes, and just two weeks after the start, and they had 83 positions posted, the room literally broke into applause. Here's a quick summary of some of the major deliverables for the two releases that we were just referring to. Two major policies. Again, this was an agency that took years to create a policy. They did two foundational policies in this 10-week period, posted positions, brought back a key management function. For the next release, more positions, more policies. DCF hires its first medical director that became a model for the nation. I'm almost through, you guys. I'll talk about this slide for just a second. This is an example of where averages mask reality. We were hearing, even with this great progress, that there were still issues with social worker caseload. Governor Baker insisted that DCF break down the data from, you can see in the lower right-hand side, the mean and the median, 18 and 19, break it down into what was happening with social workers by each social worker. When you're looking at this chart, the 186, the tall column, meant that there were 186 social workers who had a caseload of 19, 164 that had a caseload of 20, and so forth. All in all, 43% of the social workers had a caseload of 20 or more. The target was 16. This is just dangerous. This is what people were saying.

[1:24:35](https://youtu.be/8BTzvJH9psI?t=5075) A big effort went into more policies, bringing on, focusing on foster care, and focusing on the caseload. The result, some year and a half later, was a drop from 43% to only 11%, still too many with caseloads over 20, but that huge tail was gone. A summary of some of the major accomplishments that were done at DCF, and you guys could read that when you get a chance, but I want to end with these two slides. There was a tragic incident during COVID, two or three years after these major improvements were made at DCF, and people were saying, so has anything really changed? Is anything really any better? This is a quote from Mary McGowan, and she was the executive director of the Massachusetts Society for the Prevention of Cruelty to Children, one of the most important child welfare organizations in Massachusetts. She said, if I could change this one thing, I think we should stop referring to it as reforming the department. That suggests there's going to be an end. Our child welfare system needs to be continually enhanced. I think that's true for all of our work, that accomplishment is really good, but you have to keep on going. This quote from Nelson Mandela, who I think is one of the greatest humans to walk the earth in the last 100 or so years, for the work that he did in South Africa, after climbing a great hill, one only finds that there are many more to climb. The results framework works.

[1:26:23](https://youtu.be/8BTzvJH9psI?t=5183) How can it work for you is what I hope you found useful. I end with this slide. There's my contact information, my email and my phone number. If you'd like to reach out to me, please do so. Please tell me who you are, that you've come through the Innovate U.S. effort, whether it relates to this presentation or it relates to your work. Then we dedicated this book to you, to those who believe in public service, in the good it must do, and constantly strive to deliver on its promise. Thank you. I kept you a couple minutes over, but I hope this was useful. I can stay for a couple of questions if you'd like.

* * *

Original page: https://innovate-us.org/results-framework-executing-change-and-delivering-results · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org