---
id: "workshop:executing-change-part-1-introduction-to-the-results-framework"
type: "InnovateUS workshop"
title: "Executing Change Part 1: Introduction to the Results Framework"
url: "https://innovate-us.org/executing-change-part-1-introduction-to-the-results-framework"
date: "2025-02-11"
duration: "90 minutes"
language: "English"
instructors: ["Steve Kadish"]
recording: "https://www.youtube.com/watch?v=cwGJ6Mts6C4"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Executing Change Part 1: Introduction to the Results Framework

InnovateUS workshop · 11 February 2025 · 90 minutes

Join us for a 90-minute, interactive, virtual training with Steve Kadish, former Chief of Staff to former Massachusetts Governor Charlie Baker. Together, Kadish (Democrat) and Governor Baker (Republican) developed the Results Framework to focus on the necessary but often overlooked parts of good governing: successful execution and implementation in the delivery of public services. Their bipartisan collaboration demonstrates how effective governance transcends political boundaries when focused on practical results.

## Learning goals

- Learn how to improve government services with a comprehensive approach, from problem definition to practical results, using proven methodologies that have succeeded across various government departments and agencies
- Understand the 4 key concepts of the Results Framework—People are Policy, Follow the Facts, Focus on How, and Push for Results—and how these principles create a foundation for successful public sector initiatives
- Take-away immediately applicable tips and tools, including templates, checklists, and practical strategies that can be implemented in your current projects

## Speakers

- **Steve Kadish**, Former Chief of Staff, Governor Charlie Baker

Recording: https://www.youtube.com/watch?v=cwGJ6Mts6C4

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/executing-change-part-1-introduction-to-the-results-framework.txt (plain text) and https://innovate-us.org/executing-change-part-1-introduction-to-the-results-framework.json (every segment with its start and end second).

**Steve Kadish**

[0:00](https://youtu.be/cwGJ6Mts6C4?t=0) I'm very appreciative of you guys taking the time to do this. I'm super aware of what difficult and tenuous circumstances everyone in the public sector is in right now. Federal government is spotlighted, but our work in state and local governments are also even highlighted, perhaps even more. So, I have an extensive presentation for you. I will go through it quickly. The presentation will be made available to you after this afternoon, so that you don't have to worry about frantically taking notes. I'm going to jump in. The one thing that I will say is, several times throughout this presentation, this workshop, I'm going to be asking you guys for your input.

[1:14](https://youtu.be/cwGJ6Mts6C4?t=74) I have found that the input that is provided is really good, really helpful, really current, and that it's an opportunity for you to learn from your colleagues. So, I am now going to share my deck, and we're going to rock and roll for the next almost hour and a half, hour and 25 minutes. So, here we go. I'm going to share the deck. Can folks see it? If you want to give me a thumbs up or a hey, I can see it in the chat. I just want to make triple sure. Great. Thank you, Edwin. That's great. Okay. So, this is the first of a two-part workshop. The next workshop is held a week from today, February 18th.

[2:24](https://youtu.be/cwGJ6Mts6C4?t=144) Today, I'm introducing you to the results framework, the elements of the results framework. And next week, I'm going to do a deeper dive into some elements. And at the end of this, I may ask you, time permitting, if there are particular things that you'd like to dive into. So, here we go. So, my hope for you in this workshop is that you get three things out of it. You get a core understanding of what the four elements of the results framework are, and we'll go through them one by one. I'm going to be presenting you some tips and tools that you can use immediately, or you might use later on this year or later on in your career. The way we've structured the book is that there are tips and tools throughout. And I really hope you find this informative, interesting, and especially a little bit inspiring, given the circumstances of today. So, that's why I'm here. Why are you here? And I'm going to just read these two quotes to you. This first quote is from our book, Results Getting Beyond Politics to Get Important Things Done.

[3:37](https://youtu.be/cwGJ6Mts6C4?t=217) But I thought it was especially poignant now. A core mission of government is to provide strong public services, and results are inherently rewarding. They fulfill the purpose of government itself, they make lives better, and they help restore our faith in democracy. Which made me think about a quote from U.S. Supreme Court Justice Louis Brandeis, who we also quote in our book. Brandeis is, you know, one among the things that he's known for is talking about states as laboratories of democracy. And this quote from him in a particular case, a single courageous state may, if its citizens choose, serve as a laboratory for economic and social innovation. So, here we go. This is, I'm on the right, Charlie Baker, Governor Baker is on the left. This is 10 years ago now in January 2015 in the Massachusetts Statehouse.

[4:50](https://youtu.be/cwGJ6Mts6C4?t=290) This is my office. It's bare bones. I think what I had installed after this were a couple of whiteboards and maybe one other painting, but the governor's office was on the left, lieutenant governor's office was on the right. Why am I showing this to you? Charlie Baker is a lifelong Republican. I am a lifelong Democrat. And though he and I had worked together, I think three or four times prior to him asking me to be his chief of staff, the idea that he would pick somebody from the opposite party really had people wondering what was going on. And also that he picked somebody that wasn't part of his political apparatus. I'm a CFO, a COO, a chief operating officer. I've done a lot of big project work, and he and I have worked together.

[5:51](https://youtu.be/cwGJ6Mts6C4?t=351) But I think of myself as a problem solver and helping people get complicated things done and help to figure out how to untangle complicated problems in bureaucracies, in the public sector, in healthcare, and in higher education. So when Charlie, Governor Baker, asked me if I wanted to be his chief of staff, I said to him, are you sure? And I asked him that three different times. And at the end of the day, I think a couple of days later, he said, okay. And it really occurred to me that what he was looking for was somebody that he had experience with on problem solving and in complex areas. And that's our approach. And that was the approach that Baker, the Republican, and I, as his chief of staff, took to the issues in Massachusetts. So this is a quote in the introduction of our book, and it's a kind of a rudder for us. And it goes back to John F. Kennedy, President Kennedy from Massachusetts. And it just seemed very poignant now.

[7:06](https://youtu.be/cwGJ6Mts6C4?t=426) Let us not seek the Republican answer or the Democratic answer, but the right answer. Let us not seek to fix blame for the past. Let us accept our own responsibility for the future. And this is a core rudder for me and for the people that I work with, and I know for Governor Baker as well. So we, and I'm going to refer to the book a lot because the results framework is described in detail in the book. But we wrote this book because for over 25, 30 years, we were hungry for how to do things. And for the most part, the information that we found was business literature, business articles. There wasn't a lot in the public sector or the public administration world about problem solving and how to do things in complex situations.

[8:12](https://youtu.be/cwGJ6Mts6C4?t=492) So we wrote this for you guys for leaders and managers in the public sector. It's by folks who have had experience in both the public and private sector. And the idea was to really provide a window that did not need translation for you in terms of how you can begin to tackle complex problem and improve services. The book is in two parts. The first part is the results framework, and I'm going to spend most of the time today on that. And I'm going to go through each of the four parts. And the second part are four case studies on health care, transportation, social welfare, and COVID. At one point, we were thinking about calling this the book of how or a book of how, because that was something that we really felt was missing in the literature and the thinking about taking something from a policy idea to delivery. And then January 6th happened, and we really thought that the idea of a functioning democracy was super important. And I think that's even as true, if not more true today. So the book is really a message about how to do things and about hope.

[9:30](https://youtu.be/cwGJ6Mts6C4?t=570) Here's a keystone in our approach. We really believe in talented people, and we really believe in the importance of data and information and facts and fact, but that talented people and data by themselves are not sufficient to tackle difficult things. To get results, it really requires an understanding of how things work, a method, a change method, and then the know-how to make it happen. Programs and policies are important. Our approach focuses on their necessary but sometimes overlooked companions, successful execution and implementation. So here's the four parts of the results framework. People are policy, follow the facts, focus on how, and push for results. And I'm going to go through each of these, a few slides on each, a tip or two on each, and then I'm going to end this workshop today with a case study about how these things come together.

[10:37](https://youtu.be/cwGJ6Mts6C4?t=637) We always start with people, and this may sound naive. It may sound like, well, of course, but it's something that I have learned to appreciate that people don't appreciate the importance of making sure you have the capacity to act and not just continually assume that the team that you have is the right team to do the change. That's there. So we start with people to assure that you have the capacity to act for change, to roll out a new program, to improve things. Building capacity, particularly on a new initiative, particularly on a project where you are trying to launch something new or make changes or make significant improvement. It's assigning accountability to leaders and building those teams.

[11:41](https://youtu.be/cwGJ6Mts6C4?t=701) This next point, constructive tension between new people and current staff, I'm particularly struck by the actions of the federal government right now, or some parts of the federal government, where they're just wiping out whole agencies. I have worked in the public sector and in the private sector, and I used to really hate it when new leadership came in, and they thought that we, the folks that were there, were the problem, didn't know what we were doing, and that they were trying to figure out how to deal with it. They were trying to figure out what to do with us in the worst cases. There are always good and great people in organizations that know how to do things, that have incredible knowledge, yet when there are problems or there's a new program to roll out, they may not have the experience or the capability to make those changes, to roll out those new programs.

[12:55](https://youtu.be/cwGJ6Mts6C4?t=775) I have always appreciated, especially in my last 10 or 15 years of my career, the importance of having and respecting the current staff, and at the same time, when you're trying to make change, roll out new programs, bringing in new people, but the idea is creating a constructive tension, not a destruction. I'm really bothered isn't a strong enough word by what I see at the federal level, yet I know that this idea of constructive tension with new people and current staff is super important at the state and local levels. So there are people that are subject matter experts, and then there are people that know how to get things done, and sometimes there are people that know how to do both of those things, but they're different. And when you are trying to implement something and not just make an announcement about a policy change, it's important that you are seeking out both knowledgeable people, your subject matter people, and the people that have the know-how to actually make the change, roll out the program, do the IT changes, make the purchases, hire the people, do whatever is needed in the facilities and the project management support to do that.

[14:18](https://youtu.be/cwGJ6Mts6C4?t=858) The public sector work is all about working together. It is not about my way or the highway. I'm going to say it's especially important in all parts of the public sector, but I'm going to focus on state and local government right now. And we talk a lot about trust. We talk about the humility of knowing that you do not have all the answers, the importance of respect, and the essentiality of compromise because there isn't a single right way to do things. I believe and we believe and we talk about it in the book. The importance of what we call the collaborative gene and the strength of people with diverse experience in everything that they bring to help make, to help the people that they're trying to serve and to work with their colleagues. We start with people, our policy, as the first tenant of the results framework and when we have focused here we have found the greatest success.

[15:30](https://youtu.be/cwGJ6Mts6C4?t=930) I'm going to talk about two tips right now. This first is the dedicated team and I really became aware of this through a book by Vijay Govindajaran and Chris Trimble called The Other Side of Innovation. It was a business book and it was a business book about how to do innovation, but the idea was and then I brought it into my work into the non-profit and the public sector was to when you're tackling something that requires a lot of change or you're rolling out something new, simply asking everyone who's currently doing their work to change what they're doing or to figure out what needs to be done might result in that successful change, but it generally doesn't work and so this idea of the dedicated team and it's more than one person, it could just be two or three people and sometimes it's a much larger team who is focused as the central part of the change and then pulling in people from different parts of the enterprise is really important. I acknowledge that it's really hard to create a dedicated team of people, but it's possible to do and I've had some very successful strategies about doing it even when budgets are tight about how you can create a dedicated team. It's like lightning in a bottle when you are able to do this on a project, you are able to move things so much more quickly, more effectively, learn what works, learn what doesn't and so I start with this tip as part of people because it's something that isn't done enough and if you do have the opportunity to do this, it's really, it's a true powerful catalyst. If you already haven't worked with

[17:40](https://youtu.be/cwGJ6Mts6C4?t=1060) a consultant in your public sector career, you will. I think very highly of consultants, I think they can bring immediate knowledge, they can bring immediate resource, they can bring a whole bunch of how-to, often technical experience, but they're different than working with staff and through my experience, I've had good experiences working with consultants and not so good experiences and I realized that the consultant engagement is something that you need to pay a super amount of attention to. Just when somebody says, let's bring on a consultant to help us figure this out, it requires a tremendous amount of your best people and your time and energy to make it work. So these are some of the tips that I have for when you're working with a consultant and in our book, we talk about this more, but before you just say, let's bring on a consultant to help us figure this out, really take the time to understand what it is that you're trying to do. Is it an operations fix? Is it more resources? Is it the development of a policy? And then look to those consultants who most appropriately match that. As you're putting together, working with the consultant, be super, super clear about what the scope is and the timeframe and what you're looking as an outcome and what the deliverables are. When you are hiring a consultant, and this is, we always try to do this with hiring staff, but if they're offering a team, you should be insisting on their A team and actually asking for their very best people.

[19:38](https://youtu.be/cwGJ6Mts6C4?t=1178) And if you are not getting there, somebody that's not quite working out for whatever reason, it is okay. And it's actually your responsibility to ask them to put, make a change there. And they, they know they can do that and they will. And that's one of the opportunities of the fluidity of having a consultant. When you're managing a consultant contract, it's not the same as managing your staff. They are, they have different principles, rules that they operate to, even if they are saying they're aligned with you and your work, you really have to work at helping your team and the consultant team work as one. It is absolutely okay to be demanding to consultants. They typically are getting paid more than we are in the public sector. Always be fair and always try to be there as a problem solver for your staff, as well as for the consultants. So I'm going to ask you guys to go into chat now, and I'm going to come up, come off sharing the screen so we can, we can all see the chat. But I'd like you to put into chat, what do you think is important for selecting teams and building leaders?

[21:03](https://youtu.be/cwGJ6Mts6C4?t=1263) These were some of the things that I talked about, but you have experience in terms of what's worked and what didn't work. So what do you think is important for selecting lead, leaders and building teams? And I'm going to stop sharing my screen. So please jump into chat. Vickie, thank you for being the first to jump in. These are great. Great.

[22:13](https://youtu.be/cwGJ6Mts6C4?t=1333) Thank you. Rachel Wassell, it may not be pronouncing your, your name correctly, but you mentioned project management and communication. Would you speak to that for a second? And if you don't want to, that's okay.

**Audience**

[22:49](https://youtu.be/cwGJ6Mts6C4?t=1369) Yeah, no, that's, that's fine. Hello there. Yes. And Rachel Wassell from Vermont Agency of Transportation. And, and to be fair, I am a project manager. So I think it comes a little bit from that, but being able to set timelines and help people stick to timelines, understand the importance of task delegation and providing milestones so that people can celebrate as changes happen. I think all of that helps leaders help their teams move forward and understand what is needed when, and then also being able to adapt as many of the folks in the chat have said. So if you have a timeline and things need to shift or variables change, being able to really look down the line and see how that's going to affect the overall timeline. I think those things, and then being able to communicate that out clearly and with the diversity of thought and needs that

**Steve Kadish**

[23:51](https://youtu.be/cwGJ6Mts6C4?t=1431) your team might have. Thank you. Amina, you wrote with leaders, it's integrity with teams, it's trust and competence. Would you speak to that for a minute?

**Audience**

[24:03](https://youtu.be/cwGJ6Mts6C4?t=1443) Sure. Thanks. I'm Amina Foster. I work in Oregon DEQ. I think that one thing that I've noticed in working in different teams and in different organizations is that people really do follow the leader. Leaders do set the tone and the people take their cues and clues from the leader. What can I, what's expected of me? How am I supposed to show up? Even for some people, what am I allowed to get away with? So it's really the leader sets the tone and leadership truly matters. It helps with people feeling safe on a team. Will their ideas be accepted? Are they allowed to disagree? Are they allowed to have an opinion? And then with the team, you have to have that trust and if you have that leader that builds that safe space and that welcomes people in and that has integrity and demonstrates and models the right values and intangibles, emotional intelligence, et cetera, then you have people that feel like they can show up as their whole selves and they can fully participate. They can truly give their best and give their all and know that it will be accepted, recognized, and valued. And so then that's where, and then of course, you want people to be competent, but that's also how you get competence out of people.

[25:41](https://youtu.be/cwGJ6Mts6C4?t=1541) If people feel like they are able to participate fully, then I think they usually show up and they do and they give their best and they give their all. So yeah, thank you.

**Steve Kadish**

[25:51](https://youtu.be/cwGJ6Mts6C4?t=1551) Yeah. Stephen Buckley, genuine curiosity requires humility. Would you speak to that for a minute?

**Audience**

[26:03](https://youtu.be/cwGJ6Mts6C4?t=1563) Yes, thanks. I'm a recovering bureaucrat from the federal bureaucracy, environmental and impact statements and so forth. So it does have to do with public planning and so forth. But the idea being that that's what environmental impact statements are supposed to be. It's like, wait a minute, before we build this bridge, what is it that we don't know, that we should know, and maybe we don't want to know because it might change what we want to do. So it's the, you said my way or the highway, sometimes a lot of transportation agencies, it's my way for the highway. And they don't want to hear that. So yes, it requires genuine curiosity so that people will say, I'm so glad you told me that. I did not know that. So thank you.

**Steve Kadish**

[27:00](https://youtu.be/cwGJ6Mts6C4?t=1620) Samson, you wrote for selecting team, the main criteria should be talent and skills necessary and relevant for the project at hand. How do you do that?

**Audience**

[27:14](https://youtu.be/cwGJ6Mts6C4?t=1634) All right. Good evening, everyone. So what I actually meant about selecting teams, I think when you have a project, first of all, you need to, like someone as earlier said, of course, the leader needs to have a project management experience. So it means that for the project you're about to execute, you should know the skillsets or the talents that are needed. Right. So let me just give a particular example. Let's say you're about to, you know, construct like a public healthcare facilities, right? So as a leader for that project, you should be able to understand or know in details the exact skillsets you need. So it means that when you're building a team, you need to build it of diverse talent and skills, right? You could need a UI designer, you could need a web designer, you could need a programmer, right? So you need to identify those skills that you need to make you be able to build a project or is to like a very good, like very perfect is to actually what I was trying to make. Thank you, sir.

**Steve Kadish**

[28:21](https://youtu.be/cwGJ6Mts6C4?t=1701) Thank you. The, if folks want to indicate a thumbs up, or if they like one of the comments, they should, I'm going to go back. I really appreciate the comments in the chat. I picked up some things in there. I think you might too. And I'm going to go back to, back to sharing my screen. Okay. Okay. Okay. You guys should see my screen. And so this, so the first part of the results framework is people are policy. The second part is all the facts. And this, some of this just may sound obvious, but first you have the people capacity, to do things. And then you are focusing on really trying to understand what the problem is.

[29:43](https://youtu.be/cwGJ6Mts6C4?t=1783) And therefore what the potential points of intervention might be. There are two kinds of facts that that I have learned are really important. So I am a day. I used to exclusively focus on what I could read, the budget information, reports, expert stuff, and I thought if I understood that and really spent time on these things, that I fully understood what the problem was, and then I also had to a very good understanding of where the points were to intervene to make changes. I have to say it was later in my career where I began to really appreciate that that is really important, and that is really critical for a baseline, but it's incomplete, and that it is really important to take the time to talk to the people who are actually being affected by the program, by the service, talk to the staff and providers that are providing the service, look at the media reports, and that the two of them together really help explain not just what a report might say, but how it feels in reality to the staff delivering the program and also to the people who are recipients of the service. So both data evidence and points of pain, and I think we as humans tend to be good at one or the other.

[31:41](https://youtu.be/cwGJ6Mts6C4?t=1901) And so I mentioned both of these now because for me it was a learned skill, and I can see it with other people. They are really good at talking to people and getting the stories, the points of pain, and maybe not so strong on backing it up with data, and other people are data people and aren't so strong about what are some of the impacts. So following the facts, it's also important don't just jump to conclusions about what should be done. When you are following the facts and beginning to define what the problem is, you're looking at both the data evidence and the points of pain. You should be either writing down or mentally or orally articulating what is happening with the service, the program, what is it you are trying to do, and then writing down, beginning to enumerate the problems and the opportunities.

[32:48](https://youtu.be/cwGJ6Mts6C4?t=1968) And with problems, you're asking why. Why is this occurring? Why is this occurring? Why is this occurring? And trying to get deeper and deeper into what the root cause is and not with facts, ideally, and not just assumptions. When you are at the point of beginning to define the problem to solve, you want to have a problem that is solvable, and you want to be able to break down a very large problem into doable pieces. There are often examples of things that are working really well. You might have a team of people. One person is performing at a very high level. Others are performing adequately. And then there are folks that are having problems. What is happening with the person performing at a high level that can be replicated? That could be true if you have 12 different provider organizations that you're working with and on and on.

[33:56](https://youtu.be/cwGJ6Mts6C4?t=2036) You'll spend the time to understand what might be working well and do more of that and then focus on what isn't working and really drive to doing less. I've talked about, just for a minute, getting a problem of the right scale and then looking at what works and then looking at doing more of what works and less of what doesn't. But it almost always is important to take a step back to get a better view of the why, why something is occurring. And that may cause a different intervention point. It may cause a different way to intervene and so forth. I always think it's important to spend time with facts to understand the why before you are starting to describe what you are going to do. I picked this up from Beth Novick's book, Solving Public Problems, and I just thought it was another very nice way of lassoing, lassoing your problem.

[35:12](https://youtu.be/cwGJ6Mts6C4?t=2112) Is it limited in scope? A, is it something that's capable of being acted on? Is it specific? Is it a problem that will be supported by my organization and is there owner? It's just another way of thinking about what is necessary to define a problem. One of the very first things that I think are really important to do when you are defining a problem is also defining what the performance metric is that will say I'm actually creating the change that needs to be done. That the work that we're doing is actually adding value in the way that it should be or reducing the negative. And so as part of following the facts and describing what the fact set is, that's also a time when you can begin to set what your performance metrics are for the project, for the launch.

[36:19](https://youtu.be/cwGJ6Mts6C4?t=2179) Two tips here. This I call round the table. And I use round the table in two different ways. I use round the table often at the beginning of a project with leadership or with the team that's involved with the project or the launch of an initiative. And literally go around the table and ask people to state their point of pain. To state what they think the point of pain is in this effort. And this is a simple, it's something that you could build into one of your agendas now at work. And it's we in hierarchies always look to the person with the highest title and the highest position in meetings. And what this does is breaks down that hierarchy and asks everybody to speak going around the table or around the Zoom without requiring a response from the leader.

[37:38](https://youtu.be/cwGJ6Mts6C4?t=2258) The first time that I do this with a group, sometimes it works really well, but sometimes people are looking at one another and don't believe that they actually have the opportunity to speak. The second or third time that this is done, it's this incredibly powerful table setting, groundbreaking is too big of a term, but level setting for everyone. So I use this often at the beginning of projects to get at points of pain. And I also use this during leadership meetings and project meetings where after the formal presentation of the progress reports and the performance metrics and if there's another particular update, is to ask everybody to go around the room and bring up a point. Ask a question, say thank you or whatever. And this too is something that I have built into agendas and it's been super helpful because there's the formal part of the meeting and then this creates an opportunity for informal but really important questions and issues to be raised.

[39:03](https://youtu.be/cwGJ6Mts6C4?t=2343) This next tip again might be, well, of course, but it really takes effort. Whether you're in the public sector or the private sector in organizations, there's a tendency to believe that within the organization, we know how to do things. We know the best way to get things done. And this getting out of the tower is a mindset and an effort to understand what the expertise and what you're learning from within your organization and the people closest around you, but it's then deliberately going out and talking to other people, reading other things, perhaps asking for a peer review, talking to the customers. Pretending you are a customer and it's super powerful to understand not how you want to believe that your program is working or what the issues are, but for you to really understand better what they are.

[40:06](https://youtu.be/cwGJ6Mts6C4?t=2406) And as I say, it's a mindset, but it takes effort and time to do that. So this first tip is around the table. The second tip is get out of the tower. I'm going to go back to you guys now and ask you to, what are the facts that you seek to understand a problem? Where do you find the data evidence, the hard data? And how do you seek points of pain, qualitative data? If you would go into chat and talk about how do you find the facts to understand the problem? What might be your tips for seeking information on data evidence or points of pain? Oops.

[41:10](https://youtu.be/cwGJ6Mts6C4?t=2470) Okay. Sadie thanks for being the first comment. I'm going to go back to you guys now and ask you to, what are the facts that you seek to understand the problem? Where do you find the data evidence? Great comments you guys.

[42:20](https://youtu.be/cwGJ6Mts6C4?t=2540) Carla, you wrote, I ask frontline workers, stakeholders, and the user for facts. I also look at the data on the program to see where there are trends that indicate issues. Can you speak to both of those?

**Audience**

[42:36](https://youtu.be/cwGJ6Mts6C4?t=2556) Hi, this is Carla Brown. So I am a user experience designer and when we're looking at a program or a service, we always want to start with the user. So that could be through user research. So we'll start with like user research and then look into if there are people that are dealing with the user on a regular basis. I want to hear the problems that they're seeing, complaints that are being made to customer service lines, things of that nature. And then I'll also take a look at the data if it's an app or some sort of program that has been collecting data, whether it's through surveys or some sort of qualitative things that have been done in the past. I'll take a look at that and look for trends to see where there is an indication of an issue or if there's just a gap that we can fix. Because you'll start to see like regular complaints or several people clicking the wrong button or people calling a specific line for a specific thing that does not exist there. So I look for those things.

**Steve Kadish**

[43:47](https://youtu.be/cwGJ6Mts6C4?t=2627) Super. Thank you. Erwin, you wrote, I talk to people who potentially use a solution we are trying to change in some way. Can you speak to that?

**Audience**

[43:56](https://youtu.be/cwGJ6Mts6C4?t=2636) Yeah, no matter what kind of consultant you bring in, they're a stranger to the organization. And that person is coming up with something that's going to affect employees. Employees typically say, another consultant, we've been telling our management what the problems are for a long time, and nothing's gotten done. And so that's why we're bringing in a consultant. But it really takes probing in a friendly way with the people that are affected by what the change that's going to be made to find out like, well, how much do we really need this change? And if we make the change, what would you like to see and see if that aligns with our statement of work and the things we're asking the consultant to do? Because on the on the front end, when you're hiring a consultant, you have to go through a procuring process. And that also changes the statement of work.

**Steve Kadish**

[44:47](https://youtu.be/cwGJ6Mts6C4?t=2687) Right, right.

**Audience**

[45:00](https://youtu.be/cwGJ6Mts6C4?t=2700) work in ways you may not want it to have changed it.

**Steve Kadish**

[45:04](https://youtu.be/cwGJ6Mts6C4?t=2704) Yeah, well, Stephen, and you got at least one heart. The five why's root cause analysis really helpful. Stephen Fry. If you're talking, you're muted.

**Audience**

[45:26](https://youtu.be/cwGJ6Mts6C4?t=2726) Yeah, Steve Fry, state of Vermont. I'm a planning architect and definitely like to ask questions of folks when we're doing projects, which are mostly physical, sort of facility design and planning type projects. Definitely find that that's just a nice scaffold to just keep asking questions. When you think you're done, you're probably not

**Steve Kadish**

[45:54](https://youtu.be/cwGJ6Mts6C4?t=2754) done. There's another level to go down. Yeah. Great, great points. Thank you. Okay. Thanks, everyone. If you like a comment that's on the chat, please give it a a thumbs up or a heart. I'm going to go back to sharing my deck. Okay. So, the results framework, the first element is people or policy. The second element is focus on how. And the third element, I'm sorry, the second element is follow the facts. And the third element is focus on how. And this is all about the structure to move from the fact, problem, problem piece to meaningful, meaningful, meaningful results.

[46:58](https://youtu.be/cwGJ6Mts6C4?t=2818) There are two elements of the how, the what to do and how to do. And I think it's important to break these two things out. The what to do is either what your objectives are or the tactics are in terms of what if you do these things, then your premises is if we do these things, then this will occur. Then this new program will be launched. This element will be fixed. This service will be improved. The how is really ensuring that the work gets done to actually deliver on that promise, on the implicit promise that is part of the what to do.

[47:58](https://youtu.be/cwGJ6Mts6C4?t=2878) I did speak about the importance of putting together the team and the people and the people capacity piece. I'm going to spend a minute or two on project management. I'm such a believer in the work of project management. We'll talk about time and I'm going to talk a little bit about releases and agile scrum, maybe introduce folks to that. So if you have people, our policy, follow the facts, focus on how, and then the fourth element is push for results. It is the focus on how that requires the most time, the most resource, the most intelligence, the most ability to actually assure that the idea is going from an idea to delivering actual results. So this, forgive me for my cartoons, but so often in the public sector, there is an announcement and the announcement could be based on fact, but often it's not.

[49:11](https://youtu.be/cwGJ6Mts6C4?t=2951) And an announcement is not the same thing as delivering the result. I think the media misses out on that at different times, but for us who are actually in the process of making government work, an announcement is not the same as delivering results. And so I have a diamond shape on a precarious point, and then I have a strong base, a triangle base. And I say that the triangle shape is poised on a precarious point because too often the effort is about the announcement. Not enough time and attention is put into the facts and the why, and the idea of how it's actually going to be delivered is an afterthought to leadership that doesn't work. And so a foundation that's built on a stronger understanding of the fact, a good understanding of what the points of intervention are, and some expertise in the what, but all the time and effort, the majority of the time and effort going into the how is what is necessary. So this is my cartoon to try to describe to you guys why you need to focus on both the what and the how and the importance of how to do it. So this first tip is about time. And I want to say for the first 15 or 20 years of my career in doing project management work, doing a chief operating officer type work, when addressing the launch of an initiative or addressing a gnarly problem that we were trying to

[51:12](https://youtu.be/cwGJ6Mts6C4?t=3072) to fix, I would spend time with teams on the scope. And so this tip is flip the work management triangle, time drives results. So the triangle is the relationship between the scope of what you're trying to do, the time that you are is being allowed for the effort, and the resources. And you could imagine that how those three things, scope, time and resources all relate to one another, in terms of the ability to succeed. I always used to start projects, focused on scope. And then it was more typically a waterfall type approach to projects that were very, very sequential, and took weeks, if not months to identify the scope, and would often take a year or two or three to actually get to the point of delivery.

[52:20](https://youtu.be/cwGJ6Mts6C4?t=3140) Frustrating, my success rate was probably about 50%, which was considered good when I read the literature, but that just didn't feel good. And then about 10, 12 years, 15 years ago, I became exposed to the concept of Agile and Agile Scrum. And even more importantly, is that a central concept of, as opposed to starting with scope, starting with time. And what am I saying is, you have a problem, or you are trying to launch something, as opposed to trying to lay out everything that is possible to do with that. Think about the software releases on your iPhone, or on your laptop. The technology does things in releases based upon, at a particular time, there's a certain number of features that can be rolled out. So this idea of starting with time would be, what can we get done in 150 days? What can we get done? Then refining that, in my experience, what can we get done in eight to 10 weeks? And then refining that even more, is what can we get done in two weeks? And the power of flipping something from, of not just trying to completely understand what needed to be done and describe it in the scope, which in hindsight is impossible to do, as things change, and it's always going to be wrong, but instead to focus on time, and what can you get done in a period of time, in a release, quote unquote, that's followed by another release, that's followed by another release, is incredibly powerful, and incredibly powerful way to approach getting work done.

[54:26](https://youtu.be/cwGJ6Mts6C4?t=3266) This is a bigger cartoon, I guess, of Agile Scrum, and I just wanted to introduce Agile Scrum to you folks that don't know it, what some of the terms are, and then I'm going to talk about it more in a couple of minutes in the case study that I go through. But starting from the left-hand side and going to the right-hand side, you see the term product backlog in the lower left-hand corner. Well, backlog in my mind is a negative, but in Scrum, backlog is the list, the backlog of items that you want to address in the project. The sprint backlog is a subset of the product backlog, and those are the tasks, and the products, and the deliverables to get done during the sprint, and a sprint is typically, in the next kind of column, is two weeks, 14 days.

[55:36](https://youtu.be/cwGJ6Mts6C4?t=3336) And so, from the product backlog, there's the sprint backlog, which is a smaller set of items that are meant to be accomplished in the sprint, and then if you go up, there's three blue figures in a daily Scrum, and the idea of a Scrum, whether or not it happens daily, I've experienced doing scrums daily, but I've also experienced doing scrums or huddles, you know, three times a week with project teams, but it's the idea of a intense, short meeting with all the principles there focused on that work, and then at the end of a sprint is the retrospective. This is my attempt to try to introduce some vocabulary to folks that may not know Agile and Agile Scrum, and then this tip is trying to describe to you how you put the release concept or time with Agile Scrum, and I have really believed that a release leverages the strength of project management and time, and these are some of the key steps, but one is to set an overall aggressive release time frame. Determine your core work streams. These could be five, six, seven, eight. They're not 20, but these are the threads of work that include what you're doing to address key problems, key tasks that need to be done, and importantly, what are the deliverables.

[57:22](https://youtu.be/cwGJ6Mts6C4?t=3442) Make sure you're assigning accountability for each work stream, naming a team leader and naming the staff. You're taking the work of the product backlog and then breaking it into two-week sprints. Consider a daily scrum. This concept of a leadership review, I think, is super important in project work, and I've experienced projects, and you guys probably have too, where the project team is doing its thing, but leadership doesn't know what's going on, or the project team is doing its thing and it really needs leadership to know what's going on because there are things that they can't resolve, and I have found it critical to the success of getting things done in a timely way is to build into the project's regular leadership reviews. I say here weekly. Depending upon the intensity of what needs to be done, I've been involved in efforts that had leadership review meetings twice a day, 7 a.m. and at 4 or 5 p.m. or a couple times a week or weekly, but a leadership review isn't something that happens monthly, and I even think that an effective leadership review doesn't. It has to happen more than over a two-week period. A debrief at the end of each of the release and then acknowledge accomplishments and begin the next release. One of the important things about the concept of release, Agile Scrum, using time and project management is the focus on what it takes to deliverable, to accomplish the deliverable in the time frames and being rigorous about that but not being stupid about it, being fair and being innovative. Here's a snapshot of

[59:22](https://youtu.be/cwGJ6Mts6C4?t=3562) a series of releases. The line in the middle is time. It's April to November. In this case, these are releases related to the Health Connector in Massachusetts, which is the Obamacare health insurance product agency in Massachusetts. They had a couple of dramatic failures after being quite successful, and this effort was to get the health Health Connector back on track for successful open enrollment over a very short period of time. Whether or not you can read the detail within the releases, I like to show this because they're short. Release 5 is April to the end of May. Release 6 below is mid-April to July. Release 7 is August to October. They're eight weeks, they're ten weeks, and they include within them both the technical IT systems changes and operational changes, but they were wise enough to recognize that when you're making a lot of change, and I say especially in a short period of time, and even though you have done the changes through a lot of quality control, that there are things that happen that need to be developed and focused on right away. And so the Health Connector leadership said, we're going to have a Release 5, we're going to have a Release 6, but we're also going to have a Fix-It Release, Release 5.1, Release 6.1. And the idea of these was to then quickly park items that were broken or were absolutely necessary to be done quickly

[1:01:29](https://youtu.be/cwGJ6Mts6C4?t=3689) without upsetting the rhythm of the overall development of the releases. So a good example. I want to open it back up to you guys, and I'd like for you to put into chat what you think about the concept of leveraging time and the release concept, and if you've had experience with this, can you talk about it, but how is this different or the same as to how you and your agency are now approaching problems? How do you think about leveraging time, and how do you think about the release concept? And then we'll do the fourth element and then wrap things up. So in chat, if you guys would put your thoughts.

[1:03:28](https://youtu.be/cwGJ6Mts6C4?t=3808) Jim Acker, would you speak to your comment? I think of it in terms of minimal viable product, a good term, and how to get that out to people we serve as quickly as possible.

**Audience**

[1:03:55](https://youtu.be/cwGJ6Mts6C4?t=3835) Yeah, so I'm a project manager from Oregon, and even in waterfall type projects, you have these milestones and stages, and what I try to do is set those up so that we have something that can be released out to the people that we serve that's going to help make their lives better. That's the way I like to set up those milestones so that even down the road, if the project, for whatever reason, would get canceled, we've been able to deliver something, and I try to set those up kind of in an order of priority so that our top priority deliverables gets put out first, if we can at all manage that. So it's just maybe a different way of looking at waterfall through kind of an agile lens, if you will, but to focus on let's produce stuff that's usable and useful quickly and kind of build on those as precursors to the big lift that's supposed to come at the very end.

**Steve Kadish**

[1:04:59](https://youtu.be/cwGJ6Mts6C4?t=3899) That's great. Thank you. Zadie, and I apologize if I'm mispronouncing your name, timeboxing helps test new things and iterate. If it works, build on it. If not, pivot.

**Audience**

[1:05:15](https://youtu.be/cwGJ6Mts6C4?t=3915) Yeah, Zadie Ortiz from North Carolina, and when we are trying new things that we don't know if that would be useful or not, having a specific time frame, it's only two weeks or it's only one month of effort, help us focus and help us build and then try with the people that would be affected to help us understand. Yeah, that works. Let's continue down that path.

**Steve Kadish**

[1:05:47](https://youtu.be/cwGJ6Mts6C4?t=3947) Yeah. Kristen, you had a really nice answer to a question. Do you have any tips on how to get the team to align around priorities for a time period sprint? Would you talk briefly about what you put down here? And if you are speaking, we can't hear you. Maybe I got it wrong. Maybe I got it wrong.

**Audience**

[1:06:47](https://youtu.be/cwGJ6Mts6C4?t=4007) Steve, I don't actually see Kristen and the participants. She might have had to drop off.

**Steve Kadish**

[1:06:52](https://youtu.be/cwGJ6Mts6C4?t=4012) Thank you. Thank you. I appreciate that. That's very helpful for me, so thank you. I'm going to go back to the deck and do the final piece of the results framework and then end up with the case study, so just bear with me. Very good. Okay. Results framework. People are policy, building capacity, following facts, understanding the data evidence and the points of pain and the points of intervention, beginning to develop your metrics, focus on how is both what you are trying to achieve and then all the effort that it takes to deliver, and then push for results. This is where you are actually determining performance as to whether or not things worked out. I love this quote from John Wooten, a famous college basketball coach from the 60s and 70s, never mistake activity for achievement, and we're all busy with email, texts, writing, meetings, et cetera. That isn't the same thing as actually getting things done, although they're important, and results. So, three things here. Measure what matters, and I would suggest you focus on your project status and performance metrics. Discuss results and leadership reviews. They build common understanding.

[1:08:32](https://youtu.be/cwGJ6Mts6C4?t=4112) It allows for rapid problem solving, and it overcomes bureaucratic silos, and then measure your performance metrics, your measure, evaluate, adjust. So, in leadership reviews, in my mind, these are sessions, particularly if you are in a leadership role, where you are trying to get folks to be uncomfortable about race, comfortable about racing the uncomfortable, and I'm a big believer in red, yellow, green, or green, yellow, red. Green means a deliverable or a metric is on time, or the deliverable is on track and on time. Yellow means that the team or the people responsible for the deliverable believe that they're going to hit the goal, but there are issues that they're aware of, and they're flagging that they're issues, and red is one of the hardest things for people to put down, because red needs to be seen as a call for help and not as a signal of failure, and in your leadership reviews, whether you're looking at project status or performance metrics, when you are looking at red, yellow, green, or you're looking at the particular numbers, numbers, performance metrics, they do provide an opportunity to move away from the individual that's presenting to the data that's there and having a discussion about what needs to be done or how to continue to do what needs to be done.

[1:10:32](https://youtu.be/cwGJ6Mts6C4?t=4232) It's not enough to track data and to track information. You should be continually asking, so what is it telling me, and that in your presentation on a project status or performance metric, you should be saying, what is it telling me, even if it's green or if it's red, what is it you're telling me? What is it that's required? There are a bunch of different things that I think are important to look at for performance metrics, and I captured a couple of different categories, and I'm going to be talking about these more next week, but there are the categories that measure the service performance, and then there are the categories that measure the ability to actually enable or deliver the service, and they're listed here. You could add or expand on these, but depending upon what you're trying to do, the type of metric is, you know, will make sense and will fit. So here's another example from the Health Connector, and this is a project status report, and the reason I'm showing you this, one is because it's simple. It's a one-pager of where they're at on a, I think it was a 45-day release.

[1:12:08](https://youtu.be/cwGJ6Mts6C4?t=4328) There's only one item that's green. There are five items that are yellow, and there are five items that are red, and the honesty and the humility for staff to indicate to their leadership, and in this case, I was participating from the governor's office. The actual status of where things were was, I give huge credit to Louis Gutierrez, the man who was leading this effort, is that he created a dynamic where staff felt it was possible to do so, and in a leadership review meeting, we then spent our time and attention, a quick congratulations on the green, but then spent time and attention on the reds and the yellows, and so this is an example of how a one-pager can help in a leadership review, explain what's happening, and then move things along. So here's, I had a lot of, I have lots of performance metrics, and I was trying to figure out what might be the best ones to show. So this one isn't red, yellow, green, or a green, yellow, red. This one is green, gray, yellow, and I think the folks at the Massachusetts Registry of Motor Vehicles didn't want to say that something was red, but instead, they wanted to highlight when something was green, highlight when something was gray, the performance was just okay, and if something was yellow, that's how they wanted to show that they needed an attention, but here's an example of customer service wait times in six different offices in the northeast part of Massachusetts, including the Boston is the big one on the left, Haverhill, Lawrence,

[1:14:09](https://youtu.be/cwGJ6Mts6C4?t=4449) Revere, and so forth, and you can see these metrics that the green is this interesting concept of serving a person in less than 30 minutes, and then serving, the yellow is serving a person in in 30 to 60 minutes, and the, I'm sorry, the gray is serving a person in 30 to 60 minutes, and the yellow is if somebody took over 60 minutes, and you can also see here in the very light green, the bar, the bar in the middle, that's the staffing availability, so this is an example of a metric showing the performance to a customer and also a key input, a key enabler, the staff, and we would look at these metrics on a daily basis. We would actually have a brief discussion about these metrics on a daily basis, and then a more intense discussion a couple times a week.

[1:15:12](https://youtu.be/cwGJ6Mts6C4?t=4512) We launched an effort to really improve the services at Registered Motor Vehicles. So here you could see in May of 2018 there were only 38% green, and then six months later we were at 97%. A whole lot of work went into that. A tip here on metrics is really beware of averages. They mask things. Whenever I see that something is an average, I think that's possibly an indicator, but that it often is hiding the tails in terms of how long something is taking and what that means. So I'm encouraged to chart out the actual experience of the customer and then focus on the actions that improve the customer. So I'm going to race through in the next eight or nine minutes this case study showing how different things come together at the Massachusetts Department of Children and Families and start with this story, Baby Doe.

[1:16:18](https://youtu.be/cwGJ6Mts6C4?t=4578) Baby Doe, on Thursday, June 25, 2015, a woman in Winthrop, a coastal town just north of Boston, made a horror discovery. She was walking her dog along the shoreline of Deer Island, where she found an object that concerned her and called 911. The Massachusetts State Police uncovered the remains of a girl of toddler age. The baby was found in a plastic bag. The news was stunning. As a community, we had not yet recovered from the loss of Jeremiah Oliver one year earlier. Jeremiah Oliver was another child found mutilated in a suitcase by the side of the road. Now this. The hunt for more information about Baby Doe drove national news throughout the summer. What had happened to this innocent child would hang like a dark cloud over all of us. There were three terrible incidents that happened in 2015. And after this incident, after we learned about this incident, Governor Baker launched a turnaround effort.

[1:17:20](https://youtu.be/cwGJ6Mts6C4?t=4640) So Baby Doe, Baby Doe, her name was Bella, Bella Bond. The picture on the left-hand side is the police caricature, and the picture on the right-hand side is a picture that they found of the child. So this was in September. By then, the turnaround effort was organized, and this was a case example of using the results framework. You can see that I'm highlighting ten items. Two are people. There was an executive leadership team and an agency improvement leadership team. That the commissioner, Commissioner Spears, implemented. One of the things that was really important about the agency improvement leadership team is that it included people from the central office and the field from the first time. You had not the central office dictating to the field what to do.

[1:18:22](https://youtu.be/cwGJ6Mts6C4?t=4702) You had the central office people working with people in the field on what could actually work. They created a dedicated project team that included people from the agency and some consultants and some folks from the governor's office. Two items of fact, points of pain, I remember doing around the table with the commissioner leadership team and continually doing that with other leaders and people to really understand what the impact was on families and children, as well as staff. And we spent a lot of time on the data evidence. So two people item, two fact items, and then items 5, 6, 7, 8, 9 are all how. It was a release-based effort, 10 to 12 weeks, defining using work streams to define what needed to be done. These are the names of the work streams. And importantly, there was a chair from both the central office and the field.

[1:19:22](https://youtu.be/cwGJ6Mts6C4?t=4762) We used a checklist to green that I'm going to show you to launch the work streams, used Agile and business process analysis, and were heavy, heavy users of project management in terms of focusing on getting things done, along with Agile and the releases. The leadership reviews happened twice a week. This was pre-COVID. So people drove in around from across the state for hours, for meetings that were several hours long, to review metrics, to review project status, and to do almost every single meeting around the table at the end of the meeting where new items were uncovered. The agency, the Department of Children and Families, DCF, took Scrum and developed their own terminology or translated their own terminology. They did trainings for the staff that were involved. The trainings were as short as an hour for some people and might have been three or four hours for some others, but it was possible to train social workers to use a technology like Agile Scrum.

[1:20:31](https://youtu.be/cwGJ6Mts6C4?t=4831) Here's an example of how the releases and sprints worked. The efforts started in mid-September, around September 21st, and the first release was November 25th, around Thanksgiving. The second release is the dark blue line dropping down March 18th, and you can see the first release had five sprints and the second release had seven sprints. But they're short, and if you spend time on this, you get an idea of what was happening within each one of the sprints. So the getting to green is something, a checklist to green, is something that I describe in the book and it's something that I've used a bunch of different times, including at the World Bank and on projects, both public and private sector. It's to help launch an effort, and the idea of this is you pick the five or six, seven items that are critical to making sure that you have a clear understanding of what the problem is, what you are trying to achieve, and how you're going to measure it.

[1:21:47](https://youtu.be/cwGJ6Mts6C4?t=4907) And in this case, we had, is the objective clear? One or two sentences describing it. Is the scope of what you're doing, what's included, what is not? The sprint backlog, what's included in the two-week sprint versus what we're trying to do for the overall release effort. DCF, Department of Children and Families, wanted to have a strong communication and engagement piece. We wanted metrics and so forth. And so the teams were asked to put these together. This is the PeopleStream. It took the agency months to do a single hire. Here, in a matter of just a couple of weeks, they had already posted 60% of the positions that they were trying to achieve. So this was an example of the success. Here's a sort of an example of what they were doing in terms of hiring using business process analysis.

[1:22:50](https://youtu.be/cwGJ6Mts6C4?t=4970) They dove into the existing process, took it apart, rebuilt it, and rapidly expanded, accelerated the way that they were able to do hiring. It was absolutely transformational and was launched in a matter of weeks. So here are some of the accomplishments of the releases in Release 1 and Release 2. This is an agency that would take years to do a major policy. Here they did two foundational policies in a matter of weeks, posted positions. Release 2, more policies. We're bringing people on. Hired the first-ever medical director. And yet there were still issues. And I am running out of time, and I'm aware of that. So I'm going to talk about this slide and then jump to the end. Why are we aware of, so afraid of looking at averages?

[1:23:53](https://youtu.be/cwGJ6Mts6C4?t=5033) We kept hearing, Governor Baker and I, that the mean or the median for the caseload for a social worker was 18 or 19, and knowing that the target was 16. I thought, what's the big deal? You're only off by two or three. But until we broke this information down by social worker, could you see the tall line, the 186, or the number of social workers who had 19 in their caseload, the 164, those that had 20, and so forth? So over 42% of the social workers had more than 20 in their cases. A lot of work was done. A year later, there were only 11% with that. Then and now, great success. And then there was a problem during COVID, a really significant issue during COVID, and a lot of people began to, some people began to question whether or not there had been any improvement at the Department of Children and Families.

[1:24:56](https://youtu.be/cwGJ6Mts6C4?t=5096) And this woman, Mary McGowan, who was the Executive Director of the Massachusetts Society for the Prevention of Cruelty to Children, said, if I could change this one thing, I think we should stop referring to it as reforming the department. That suggests there's going to be an end. Our child welfare system needs to be continually enhanced. And another way of saying this is a Nelson Mandela quote, who I think is one of the greatest humans in the last hundred years. After climbing a great hill, one only finds there are many more to climb. So thank you. It's 3.30. I'm going to leave the book for you guys, to those who believe in public service and the good it must do, and who constantly strive to deliver on its promise. I'm going to stop sharing and turn it back to Agada, and thank you guys for hanging in there.

* * *

Original page: https://innovate-us.org/executing-change-part-1-introduction-to-the-results-framework · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org