---
id: "workshop:executing-change-part-2-applying-the-results-framework"
type: "InnovateUS workshop"
title: "Executing Change Part 2: Applying the Results Framework"
url: "https://innovate-us.org/executing-change-part-2-applying-the-results-framework"
date: "2025-02-18"
duration: "90 minutes"
language: "English"
instructors: ["Steve Kadish"]
recording: "https://www.youtube.com/watch?v=wi3Ps-ioC70"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Executing Change Part 2: Applying the Results Framework

InnovateUS workshop · 18 February 2025 · 90 minutes

Join us for a 90-minute, interactive, virtual training with Steve Kadish, former Chief of Staff to former Massachusetts Governor Charlie Baker. Together, Kadish (Democrat) and Governor Baker (Republican) developed the Results Framework to focus on the necessary but often overlooked parts of good governing: successful execution and implementation in the delivery of public services. Their bipartisan collaboration demonstrates how effective governance transcends political boundaries when focused on practical results.

## Learning goals

- Explore real situation case studies across diverse sectors—health care, social welfare, transportation—demonstrating how the bipartisan-developed Results Framework adapts to different contexts and challenges
- Learn how to apply powerful implementation methods like agile scrum and business process analysis in public sector settings, with specific attention to government constraints and opportunities
- Gain practical knowledge on achieving strong project results, from initial setup to performance measurement, including establishing meaningful metrics, tracking progress, and making data-driven adjustments
- Sign up for part 1 [here](https://innovate-us.org/executing-change-part-1-introduction-to-the-results-framework)

## Speakers

- **Steve Kadish**, Former Chief of Staff, Governor Charlie Baker

Recording: https://www.youtube.com/watch?v=wi3Ps-ioC70

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/executing-change-part-2-applying-the-results-framework.txt (plain text) and https://innovate-us.org/executing-change-part-2-applying-the-results-framework.json (every segment with its start and end second).

**Steve Kadish**

[0:00](https://youtu.be/wi3Ps-ioC70?t=0) Let's go. Hey, well, welcome everyone, and if you had a chance to participate a week ago, welcome again. If you hadn't, the recording is available. I'll do a short recap of what we covered last week and then roll into some new things. I have an extensive presentation deck that Agedat will be making available to you all afterwards, and so you don't have to frantically take notes. I have too much information, and even the information that I'm presenting you is a selection of all the things that I would love to talk with you about. Just briefly about me, and I keep reading things and listening to folks who are working in the public sector and what I'm reading, but it seems increasingly important, and you know this, folks, in terms of what's happening in Washington, D.C., in terms of the ability for, in particular, states and local governments to deliver, and I've always felt that the importance of government to deliver, that it wasn't just about the announcement of something, it was the actual delivery of something that really matters and, in the end of the day, builds trust for folks in government.

[1:42](https://youtu.be/wi3Ps-ioC70?t=102) All right, so it's all to e-mails from you, okay, and when you say Google Meets, do I need to download any specific? Chike, oh there you go, okay, and I, there's a great piece in the New York Times yesterday or the day before about this, you know, what we're talking about today, and so my passion is to help people like you all with helping to make government work better. I'm gonna flip it very quickly to Chike right now. He'll be coming in and out at a couple of different times into this presentation, particularly when I'm talking about Agile Scrum and then also at the end, but Chike, do you want to introduce yourself?

**Unknown speaker**

[2:39](https://youtu.be/wi3Ps-ioC70?t=159) You're muted.

**Chike Ayogu**

[2:45](https://youtu.be/wi3Ps-ioC70?t=165) No. Can folks hear me? Now, yes. Perfect, sorry about that. Steve, great to see you and thank you for having me, and good afternoon to everyone. My name is Chike Ayogu. I also serve as a senior fellow here at Northeastern with Innovate US. Previously served as the chief innovation officer at the U.S. Department of Labor in the prior administration in D.C., and way before that, beyond the private sector as well as the non-profit space, I also served in the public sector, actually for the predecessor to Governor Baker being Governor Patrick, and I cannot just, cannot overstate the importance of what Steve said, which is delivery. Someone who's been in government at the local level in New York City and Massachusetts as well as the federal government, it's easy to believe the announcement is the end. What Steve knows is that the announcement is the beginning, and that end is a long tail that goes through weeks, months, and years in getting that right, and the only other thing that I'll say, and I'm sure we'll talk about this a little bit, is one, something that exists but that cannot be accessed in a way that citizens find easy and dignified doesn't exist. That's a thing that we have learned so many times, and then also whatever you think the thing is at the beginning, you are wrong. It's kind of the first thing you learn when I was in the private sector. Whatever you think of the first time is wrong, and this belief that you have to iterate consistently to get to the right thing, or at least closer to the right thing on behalf of the citizens that you serve is so important. It is, in some ways, unusual for us in government. We believe that at the beginning, we can spend 18 months and make a strategy. Of course, by the time you've made the strategy, the problem's changed, and whatever you were thinking of no longer works. So knowing that you have to iterate and do so in a way that the citizens understand and feel like and interact with is just so important. Steve, I'll throw it back to you, and I'm really excited to be here

**Steve Kadish**

[4:45](https://youtu.be/wi3Ps-ioC70?t=285) today. Great. Thank you. I'm going to toggle between my presentation deck and then asking you all, actually, for your comments and chat. Folks that are just joining us, if you would like to tell us what agency and where you're working, that's great. It's great to see folks from literally all over the country and a few folks from the outside. So I'm going to share my deck now. Okay. All right. If folks can see that, thumbs up. Great. Okay. Super. Okay. So I'm just going to do a short recap of the hour and a half that we spent together last week, and this is sort of a recap of the recap. I started off talking about a bit what Chike spoke about and what I just spoke about, but the importance of how and the importance of delivery and how, at one point, Governor Baker and I were calling this the book of how, because there was so little out there in the literature about how to actually deliver something. Two, we spent a lot of time on the results framework, and I'm going to do that again now, go through each piece. There were probably eight different tips and tools that I presented to you guys while talking about the results framework, and I'm just highlighting four here.

[6:50](https://youtu.be/wi3Ps-ioC70?t=410) One was a tip on how to manage consultants. The second is a tip that I call round the table that I use in uncovering points of pain related to facts and also on very frank discussion about project progress. The third is about the importance of time and agile scrum, and today I'm going to do a deeper dive into that along with Chike. The fourth was the importance of leadership reviews. It's not enough to do the work and report on it and effectively talk among yourselves, but the importance of actually having you guys as leaders or helping your leaders in leadership reviews. We closed with an extensive case study on the turnaround at the Department of Children and Families, a child welfare case in Massachusetts. When you're binging a TV series, they go back and show you pictures of who some of the actors were. Well, this is a picture that I showed you last week and again today. Governor Baker, lifetime Republican, asked me to be his chief of staff, a lifetime Democrat. I'm not a super political person, but Baker was looking for somebody that could help him with tackling the delivery issues, the operational issues, the service issues inside the Commonwealth of Massachusetts. We had developed a framework, the results framework, over some 20, 25 years of working with one another. The result was

[8:53](https://youtu.be/wi3Ps-ioC70?t=533) our book, Getting Beyond Politics to Get Important Work Done, and laying out to folks how it worked. In a nutshell, I just wanted to repeat this because I think it is so important that this book that we wrote, the process that we use, it's about delivering results. We emphasize the importance of people, and we emphasize the importance of data, but it's not enough when you're delivering results. The results require an understanding of how things work, a method, and the know-how to make it happen. Programs and policies are often the rudder for what needs to be accomplished. There's a lot of books and literature and activity around programs and policies. Our approach focuses on their necessary, but sometimes overlooked, companions, successful execution and implementation.

[10:05](https://youtu.be/wi3Ps-ioC70?t=605) That's what the results framework is about, these four parts. People are policy, follow the facts, focus on how, and push for results. It's simple. It's not earth-shattering. It's not rocket science. It's diligent, it's deliberate, and it is, as Chike said, it requires repetition of learning and doing. I'm going to go through each of these again quickly. People are policy. We always start here. This may sound naive or too simple, but time and time again, I've been involved in efforts where there was a lot to do, whether it was to fix something, roll out a new program, make major changes, and the existing team was asked to pull it off. And sometimes that is possible to do, but more often than not, when something is broken and needs to be fixed or a new effort needs to be rolled out, you need to make sure that you have the people in place to pull that off. So we start with people at our policy to assure that we have the capacity to act. The third bullet, seek both knowledge and know-how. Again, may sound a little bit simplistic, but there are people that are experts in their subject matter area, but they don't know how to actually deliver. And then there are people, project managers, some budget people, some tech people who actually know how to make things work.

[11:58](https://youtu.be/wi3Ps-ioC70?t=718) You need both. You need both. And in the public sector, I think it is almost never my way or the highway that lasts for very long. It's really about how we work together. And I think the importance of the collaborative team. So we start with people or policy. We then move to follow the facts. And there are two kinds of facts that we look at. Data evidence, the stuff you can read, the stuff you can look at, the so-called hard data, expert input. And then equally as important, points of pain. I'm a data geek person, and I always used to start with trying to understand problems by reading reports, looking at spreadsheets, looking at performance metrics if they existed, looking at what some other entities did.

[12:58](https://youtu.be/wi3Ps-ioC70?t=778) And it was later that I became to fully appreciate that there's only so much you can learn and know by reading and analyzing things. Talking to the people who are actually getting the service and talking to the people who are providing the service is incredibly important to get the full idea of what the fact set is. And so defining the problem points to the interventions. And defining the problem well will also point to your metrics for success. So we start with people or policy. We then move to follow the facts. And the third element is focused on how. And this is the area where you have to spend the most time in terms of the actual implementation, execution, and delivery. It's the structure that moves to results. There are two elements to focus on how. What to do and how to do it. Now what to do, you could consider that the announcement. You could consider that the development of the program paper or the idea of what the new technology would do or what the new service would do.

[14:26](https://youtu.be/wi3Ps-ioC70?t=866) And it could be tactics like more funds, a technical fix, a new law, a new regulation, operation improvements. But those are kind of like the announcement. They're the implicit promise. How to do it is all about what it takes to actually deliver the results. And we spent a lot of time last week and we spent a lot of time in our book talking about various approaches, putting together accountable leads and dedicated The importance of project management, and today I'm going to do a deeper dive into the importance of time and is to talk about a tool, Agile Scrum. So, we know this, and we've experienced this, you've experienced this, perhaps the excitement of actually getting to the point of an announcement, or even a law, but it's not the same thing as actually delivering the change.

[15:34](https://youtu.be/wi3Ps-ioC70?t=934) So, the fourth and final element, which again starts the cycle, but we start with people or policy, we go to follow the facts, we then do a focus on how, and again, this is where most of your time, your emotional energy, your intellectual energy, your resources will be. And then four, it's push for results. And this is where you're actually measuring the performance. I love this quote from John Wooten, a former basketball coach in the 60s and 70s, never mistake activity for achievement, meetings, texts, emails, you name it. Even reports, those are not the same thing as deliver, they could be necessary and helpful in the development of some of the what, some of the tactics, some of the programs, but they're not the same thing as actually delivering the change.

[16:35](https://youtu.be/wi3Ps-ioC70?t=995) So, these three things, measure what matters. I can't stress the enough of both when you're on a project, really adhering to some kind of project status and being objective with how you're doing with that. I like simple red, yellow, green on project status. And then the importance of performance metrics that are actually measuring the improvement or the new program rollout and measuring the impact. I think the most important thing to measure is the fixing, the addressing of the point of pain. Item two, and I mentioned this briefly just a couple minutes ago, but I too often have seen good projects, good efforts only go so far because leadership has not been involved. And some of you are at a leadership level where I implore you to make sure that when you are doing programmatic change or rolling out a new program, that you just don't give it to the people that are working with you and your projects team to deliver, that you are part of the leadership review, as well as your key folks.

[17:58](https://youtu.be/wi3Ps-ioC70?t=1078) It's so important in building common understanding, allowing for rapid problem solving. And it's a really powerful way to overcome bureaucratic silos. And I think bureaucratic silos and the lack of an ability for problem solving are two incredible Achilles heels, particularly in public bureaucracies. Measure, evaluate, adjust, repeat. You're measuring your project status. You're measuring your performance metrics. In the leadership reviews and in your project teams, you're doing an honest evaluation of what's working and what's not, adjusting, repeating, then repeating the cycle, measure, evaluate, adjust. So that's the overview, as opposed to an hour and a half. That's the 10-minute overview of the results framework. People are policy, follow the facts, focus on how, push for results.

[19:00](https://youtu.be/wi3Ps-ioC70?t=1140) So I just want to toggle to you guys now, and if you would go into chat, but what resonates with you? People are policy, follow the facts, focus on how, push for results. If you would put into chat what element resonates to you and maybe a few words as to why. What resonates with you? And I'm going to come off this.

[20:01](https://youtu.be/wi3Ps-ioC70?t=1201) Ruby, thank you for being the first person to put something in. Purvi, thank you. Patrick, I super agree with your comment. Great comments, folks. Samantha, you wrote, all are imperative, but push for results with metrics clarifies throughout the framework cycle. Would you speak to that for a minute?

**Audience**

[21:01](https://youtu.be/wi3Ps-ioC70?t=1261) Hi. Having lunch. But, yeah, I was just sharing that all the areas are important. But pushing for results, advocating for results with metrics throughout, I should have put throughout the cycle of the framework. Always be metrics, not driven, but metrics, like identifying the metrics necessary at each stage in the framework cycle that shows and reflects the results or the objective will help to keep clarity as the cycle, as the project cycle rolls. These are the objectives, these are the associated metrics throughout the cycle. It helps to keep everything clear as to why we're doing it and the degree to which we can accomplish it because we've identified tangible metrics.

[22:01](https://youtu.be/wi3Ps-ioC70?t=1321) And it just helps with clarity to have a number next to that objective.

**Steve Kadish**

[22:06](https://youtu.be/wi3Ps-ioC70?t=1326) And it's hard to often get at the right metrics, but I so agree with your point. So, so thank you. Leslyn, you wrote, measure what matters and follow the facts because aspect because it enables sustainable change and process management. Would you speak to that?

**Audience**

[22:26](https://youtu.be/wi3Ps-ioC70?t=1346) Yes, I can. Hi. Oh, can you hear me?

**Steve Kadish**

[22:30](https://youtu.be/wi3Ps-ioC70?t=1350) Yes.

**Audience**

[22:31](https://youtu.be/wi3Ps-ioC70?t=1351) Yes. I think throughout the four different aspects that you spoke about, measure what matters really sort of struck out to me because I feel like in a lot of change management and process management, we get lost in what's being done, but we don't really evaluate and don't really take feedback. And I really think that plays a big role in building sustainability in the work that we're putting in.

**Steve Kadish**

[23:02](https://youtu.be/wi3Ps-ioC70?t=1382) Thank you. Princess Asita, you wrote, people are policy. It's the heart of the effort. It determines how we follow the facts. Focus on how and push for results. Would you speak to that for a minute?

**Audience**

[23:15](https://youtu.be/wi3Ps-ioC70?t=1395) Yeah, thank you. So my view is that, essentially, it is people, whatever it is we want to do, they're not going to work themselves. It needs people. And so, if the right, I mean, the hiring is intentional, the building of the team, you know, guiding the focus on what matters. That's how we're going to look out for what we want to follow, what we want to focus on, how we want to measure if we're moving forward. Whatever it is that we think the beautiful vision or goal we're working towards is, in the end, it's people who are going to do it.

**Steve Kadish**

[24:00](https://youtu.be/wi3Ps-ioC70?t=1440) Thank you. There are great comments here. I'm not going to get to them all. I encourage you to look at them all and throw a thumbs up or a heart for the ones that you like. Patrick, you wrote, how, how, how, how to get it done. I'll let you be the last one.

**Audience**

[24:19](https://youtu.be/wi3Ps-ioC70?t=1459) Just never short on ideas, right? It's just figuring out the particular levers that makes the most sense for where you sit. I think that's something I'm always trying to do.

**Steve Kadish**

[24:43](https://youtu.be/wi3Ps-ioC70?t=1483) Well said. Thank you. Okay. I'm going to go back, back to the deck. Great comments. And I appreciate for the folks that I cold called that you, you were awesome. So, let me go back to sharing the deck. Okay. Yeah. One second. So, now for the next hour, 58 minutes, something like that. I'll end this on time. I want to focus on two tools, Agile Scrum and also communication.

[25:52](https://youtu.be/wi3Ps-ioC70?t=1552) And then, then take you through a case study of an agency that was called the worst in the nation. And in a very short period of time was, was called the best in the nation. And so, here we go. I do not know how many of you use Agile Scrum. Have you Agile Scrum? For those of you that do, I am really interested in your comments and feedback and your, your tips that you can give folks at the end. Here, what I did is pull together a slide with, with an audience in mind that may not know what Agile Scrum is and, and try to describe and present some, some tools. And some of these materials were materials that I had used in, in a training on Agile Scrum for the Department of Children and Families and other places. So, here we go.

[26:59](https://youtu.be/wi3Ps-ioC70?t=1619) So, I start here. And so, why are we talking about Agile Scrum? And I had this slide in the last deck. And I just emphasize the importance of how time works to drive results. And I believe that there is this physical thing, a thing of physics. It probably needs a name, but there is a work management triangle that has scope, what it is you are trying to accomplish, time, the second element in terms of how much time you have available, and third, the resources that are available. And I think for, I know for a long, long period of time, projects and efforts always started with scope. And a good, good efforts might, good project teams might have a 40% or 50% success record. I feel like that was about mine.

[28:03](https://youtu.be/wi3Ps-ioC70?t=1683) But it was when I learned about Agile some dozen years ago, and began to apply it, did I fully appreciate the idea of using time as the driver in, in, in, in getting real meaningful work done in chunks, and then moving, moving it along. So, I'm a huge believer in the concept of releases that are time boxed, and in, in the ability for Agile Scrum or a version of Agile Scrum that works for you in terms of moving things along. So, in the next half dozen slides, I'm going to go through Agile Scrum or a version of Agile Scrum. And so, this is the basics. This slide too was in the, in the previous deck, and just the core elements are all here in this one picture.

[29:07](https://youtu.be/wi3Ps-ioC70?t=1747) From the left, you have the product backlog, and the, the backlog is a reference to the key deliverables and artifacts that are the result of the effort. And so, it's not a bad term. It's a, it's a way to catalog all the items that you are trying to address. The sprint backlog, moving one effectively column to the right, is for the sprint, for the two-week period, what is it that you are trying to get accomplished? The sprint is here, shown in orange, a two-week, 14-day period with a 24-hour cycle. That's hard to do in the work that I do. had done and do in the public sector. We have two-week sprints, but usually our scrum cycle is is two days or three days. The daily scrum at the top right is something that I'll spend more time on. And then something that I really appreciate about the Agile scrum is that it builds in a retrospective. It builds in the time for the team to look back and say what didn't work, what did work, and how to plan for going forward. So this comes from the Department of Children and Families in Massachusetts and their intense turnaround effort. You can see I have a tip in here is to include project management support as an entity, as an effort while you are doing Agile scrum. And your project people can take some of these roles. But it requires first your

[31:12](https://youtu.be/wi3Ps-ioC70?t=1872) executive leadership determining the goals, the time frames, the work streams. They're designating the project owners. And in leadership reviews, they're addressing the cross-team problems. The second person identified as a project owner, and there'll be some six, eight, dozen of people who are identified as project owners for a work stream. And these are accountable leaders. They're often senior people in an organization or leaders of units in an organization. And they're the ones who are accountable for the thinking about the delivery from design, delivery, and implementation. The captains or the scrum masters are the folks who are the captain of the effort of the work team to deliver in a work stream. Sometimes you may be involved in an effort that has a separate scrum master from the captain. More often than not in the public sector, the captain has acted as the scrum master. So what is the scrum master? And I'll talk about it a little bit more. But the scrum master is the person who is skilled in the process of the Agile scrum and keeping things running. The captain is responsible for making sure that the work is getting delivered. And of course, and in some ways I wish this was flipped, the whole chart was flipped, but the work team and the work team members are the folks that are actually doing the work to make sure that things happen.

[33:16](https://youtu.be/wi3Ps-ioC70?t=1996) So the captain's role is outlined here in more detail. The captain can act as the scrum master. Sometimes the scrum master, as I said, is a different individual. But the purpose of the captain's role is to make sure that things are getting delivered within the sprint. And they're hyper-focused to keep the team on track. They're constantly looking at the backlog of the sprint and making adjustments with the team. They're the one who are identifying if additional resources are needed. They're running, perhaps with the scrum master, the planning meetings and the retrospective meetings and any other key meetings, including brainstorming sessions and solution workshops. I have often found also that folks involved in the project support can help with these meetings. But the captain is the one who's reporting at a leadership meeting, at a progress meeting in terms of what's going on. And the captain has the responsibility of escalating problems that cannot be resolved by the team. A really important role, a really important role for the overall effort, and a really important role that the work team members feel is that they have confidence in the captain being an advocate for them. And that the leadership has confidence that the captain is being honest and objective with them about what's working and what's not and what's needed. The work team member, I have this slide here up because I so believe in the importance of acknowledging that the work gets done by individuals or small groups, putting it together. And the idea that in Agile, the idea is that the work team members need to be both intensely

[35:20](https://youtu.be/wi3Ps-ioC70?t=2120) collaborative and that they're the ones responsible for self-organizing and self-managing their work in terms of achieving the deliverable. So there's an intentional, supportive, collaborative sharing of responsibility of the work team members. And the work team members has to continually provide updates to the captain or the project manager on the progress. Daily scrum, I have daily in quotes, as I said earlier. In some cases, I've been on teams and experienced the daily scrum, but more often than not, it's happened two or three times a week in public sector and nonprofit work, and that works. And the idea of the scrum is to, it's information sharing. It's a short meeting. It's fast, it's facilitated, and it's focused.

[36:26](https://youtu.be/wi3Ps-ioC70?t=2186) All team members are equal, meaning all team members can raise issue. And there's literally around the table that should occur at a scrum where each individual is reporting on what they did since the last scrum, what their work is going to be going forward, and do they need help? Are there any impediments that need to be addressed? The scrum master or the captain or sometimes the project manager will be the person who is leading the scrum, the regular scrum. Fast, facilitated, focused, honest, collaborative, bringing up the issues that need to be addressed, indicating what has been accomplished and still what needs to be done. So the backlog is something that the work team will do on a work stream, usually at the very beginning of a release, and it's an ongoing task. And so building the backlog is really important in terms of identifying the specificity of the work that needs to be accomplished to deliver on the overall release and then as a sprint. And you can see here I noted that the backlog will expand. You'll have items that will be completed, but as work gets done, there'll be new items that the team will be wanting to add. Here's just a picture of how the current sprint is a subset of the larger product backlog, if you think about the first picture that I showed you. And so this is just an example that we used in terms of trying to articulate the work in the current sprint and in the prioritized backlog as a subset of the larger of the larger product backlog for the release. I like this chart because it shows

[38:35](https://youtu.be/wi3Ps-ioC70?t=2315) the first two releases for the Department of Children and Families, the DCF turnaround. And the effort started, as you can see, in September, September 21. And the first release was targeted for November 25, the dark blue line going down. And the second release was March 18. And you can see that there were five releases in the in the first five sprints, sorry, two week sprints in the first release, and then seven sprints in the second release. And the heavy focus on team deliverables, this is a chart is intended to show what the idea that some deliverables will be accomplished in one sprint, and some deliverables may go on to other sprints. But it's intended to reinforce the importance of time, what can get done in two weeks, what might can get done in two weeks, what might get done in four weeks, and what is supposed to get done by the by the release date, different types of meetings. This was very helpful for the Department of Children and Family turnaround, department turnaround, in identifying what the different meetings are, or, and, and then this, this is a picture showing those meetings on a on a two week sprint calendar. If you go to the left, you'll see, you start off with a sprint planning session. And then also, if this is happening after the first sprint, there's a retrospective of the previous sprint, the blue box, the SOS sprint review, the Department of Children and Families called their leadership review meetings, SOS, which was,

[40:36](https://youtu.be/wi3Ps-ioC70?t=2436) which was scrum of scrums, they were where all of this, all of the work streams, all of the scrums were reporting into one place. And you could see what they were trying to do in each one of these, these four meetings twice a week, over the two week period. This is a terrific chart, I think, because it describes what, what the key goal was of the, of the, of the SOS sprint review meeting. What was the captain's role, in terms of what was the expectation to present, and then if there was a template that the leadership team was expecting. So, let's try to think about what I learned from Agile and, and what were some of the important things for groups that hadn't used it yet. And so, I put these notes together, and I'm very interested in hearing from others who have used Agile, but it, it's not an abstract idea. It's something that I think there needs to be a commitment to doing. And it's, you need to be, you need to have clarity about what the problem is you're trying to define, the specific improvements or solutions or new program you're trying to roll out in a time frame. And if you haven't used Agile before, I would, I would get some help. Now, there are people on your teams that are, that are probably Agile trained. They could be Scrum Masters. There are all kinds of consulting groups that can come in and help you, either for, to support the whole effort to help you get started, but get help. I've also found that

[42:38](https://youtu.be/wi3Ps-ioC70?t=2558) I've never used pure Agile Scrum in, in all the efforts. And I'm thinking of efforts that involved some World Health Organization work, World Bank work, work in several agencies in Massachusetts. We always took the basics of Agile Scrum and then adapted it to our meeting structure and our, our language. It's important to invest in training when you're starting with Agile Scrum. And this isn't like going off for two weeks. We've done trainings of where they've been 45 minute training so that people understand the basics for folks that are participants in the Agile Scrum to, to trainings that might be a couple of hours or a little bit longer for folks, either on leadership teams to know what's happening or folks who, it's more extensive training that's available for folks that want to be Scrum Masters. But invest in some training and make sure you have the support for as questions come up, but don't let the training, the investment in training be a reason not to do it. Leverage the release concept, concept and Agile Scrum together. That may sound obvious, but you set your timeframe for the release and then break the work up into two, into a two week sprint. So you'll be amazed if you haven't used this, how fast you will get tangible work done. And the, the important thing about this is you are not going to get done in a, in a 10 or 12 week release, what you would get done on a project that is a year and a half, but you'll get done some really important things and you'll learn and you'll roll it out and it will begin to make improvements or begin to roll out,

[44:43](https://youtu.be/wi3Ps-ioC70?t=2683) you know, a minimum, a minimum viable project effort, and then another release, and then another release, and you will be making progress all along the way. So I want to I want to go to you guys in one second and ask you, if you haven't used Scrum, Agile Scrum, what are the possibilities you see with this concept? And if you have used it, if you would know what some of your experiences and some tips and what your questions are. So your thoughts, what are your thoughts about Agile Scrum? If you've experienced it, would you jot down why it's worked for you? And if you are interested in it, what possibilities do you see for this concept?

[45:47](https://youtu.be/wi3Ps-ioC70?t=2747) Erwin, thank you for your question and let's get more of them out and then a combination of Chike and I will answer them. Comments, guys, in chat.

**Chike Ayogu**

[46:01](https://youtu.be/wi3Ps-ioC70?t=2761) Steve, I see we have two questions. We have two questions already that I actually think are really good. I have a thought on the first one from Erwin. If you want to take the one from Stephen.

**Steve Kadish**

[46:33](https://youtu.be/wi3Ps-ioC70?t=2793) What's up? Let's get a few more Chike and then I'll let you jump in, but thank you. These are great questions. Samantha, you are so right about check-ins. Any other comments from folks that have used Scrum?

**Chike Ayogu**

[47:11](https://youtu.be/wi3Ps-ioC70?t=2831) Chike, take it away. Take any of these questions. I have thoughts on two of these. One is the question from Erwin around time commitment and then the other one is around involving comms and legal. And so I have thoughts on those two and I'll take the comms and legal first. As somebody who has worked on this federal level and then in variance on the local and state level, one of the things I have found, and I worked at the Department of Labor on the national level, which I think after the Department of Justice and Department of Homeland Security has the most lawyers in the federal government. We are an enforcement agency and so we have lots of lawyers who have lots of thoughts. One of the things that I found was involving our counsel as early as possible. Now, the challenge is, the instinct is at times you want to involve counsel as late as possible because you're afraid they're going to say no. I have generally found, this is just my point of view, obviously make sure you have an idea of what you're doing, but bring them in as early as practicable. Because one, what I have found is at times counsel will want to say no, not because necessarily they believe what you're doing is against rules of protocol, but they just don't understand.

[48:27](https://youtu.be/wi3Ps-ioC70?t=2907) To the extent that you can bring them in early so that they understand, you can kind of bring that in. I would say the same thing on the comms side. Me personally, I've had less of an issue on the comms side than on the legal side, but I think it is important to bring in legal to the extent that you need to bring it in so that they understand what you are doing. The other thing I'll say is, as I always say, pick your lawyer carefully. I think at times we've all gone to two different lawyers in our legal department and gotten two different answers. You probably already know who are the lawyers who potentially will look at a way to get you to yes, of course, within the bounds of the laws and protocols. That's one piece of feedback that I've lived. Secondly, the first, which is what is the typical time limit for Scrum members? Yes, they're going to be formal Scrum members, but I think, and it actually echoes something that I think Samantha R said, which is your organization has to see Scrum and Agile, and just that way of thinking, take the terms off it, as just a way of doing business, not as a thing that sits over here and the core work is over here. Ideally, you've done it right when Scrum and Agile are just your way of doing work and you don't think of it as an additional time commitment. I do things this way because it's the most effective way to get things done on behalf of the citizen. Initially, I think the biggest time difference and commitment, and Steve, I'm curious on your thought, is the difference in the way in which you communicate. You're now going to be communicating on a daily basis, at times double daily basis, for that big transparency, and you're working on much tighter time frames, which forces a lot more activity. I think initially that is the biggest difference, but I will, if Steve, if I can tell one story, and it's a Boston story, I was fortunate to work at the McChrystal Group during the beginning of COVID, and we supported the city of Boston in responding to COVID. We created basically a 24-hour cycle of structure and activity to get a bunch of things done, and this was in probably April of 2020 when we were all trying to figure out what was going on. It was a daily cadence, and literally whatever was said

[50:28](https://youtu.be/wi3Ps-ioC70?t=3028) 24 hours earlier, we expected a progress update on 24 hours later. I remember, it was actually before I went into government, and Secretary Walsh was my secretary, a member of his team said to us, so beyond the pandemic, this is just the way that the government should run, right? We kind of were like, yeah, that's ideally how you want to run the government, at that level of speed, because of the problems that we all are seeing. I think, but those were different. Those meetings that we had, the way they were run, the preparation of those meetings, that was the initial kind of time difference, and then from there, it just became the way that they did business. Steve, I'm curious on your thoughts on those two questions. I also see some feedback

**Steve Kadish**

[51:15](https://youtu.be/wi3Ps-ioC70?t=3075) in the chat. I have been involved in efforts that have used agile scrum and the cadence of meeting, sometimes leadership meetings, meeting twice a day or daily and so forth, but I've also been involved where the leadership said, okay, and you could smell that it just wasn't working, and so the question about how do you get leadership to believe in this, you know, that Stephen's comment, is there a good way to sell this concept to management to establish the need? It's worked for me, and I've seen it work when when particularly things are in really rough shape, and there's a realization that doing things the way things have been done are just going to create the same non-accomplishment, non-result, and so will you give it a try? And so I've seen it, so when people have been almost desperate to try something new, and then I've seen it when people have seen its success that others, they have followed a project or somebody who they respect has used agile, and so Stephen,

[53:15](https://youtu.be/wi3Ps-ioC70?t=3195) I would say if it's a situation that is really tough, then this is a really good alternative to doing things the way that things have been done. If you are trying to sell or advocate to a leader or leadership team that this may be a way of doing things, think about who they might listen to, who's experienced agile scrum, and have them do a heart-to-heart about how it worked and all. And Samantha's point is that this really needs leadership. I saw that Mary Joyce was on here, and I don't know if she's able to speak, but I'd be interested, Mary, in your thoughts about how do you start? What were some of the key tips in starting with agile?

**Audience**

[54:29](https://youtu.be/wi3Ps-ioC70?t=3269) Hi. One thing for buy-in that I found is prove that it works. So make sure that leadership, when you promise something, that you deliver. So I worked with someone at MassDOT, so it's the Department of Transportation in the state of Massachusetts, and we had a leader who saw that it worked, and he showed up to every scrum, he showed up to every retrospective meeting, and he always stuck to his promises. We always said that if an issue comes up on the scrum, we will have an answer for you on the next day, and he always stuck to it. And that was where people bought in. Leadership was bought in. They saw that he kept his promises. If he couldn't make a decision, he came back to the team looking for recommendations. But having that senior leadership buy-in was key to moving forward and being successful. And again, when you make promises to say we'll get things resolved within 24 hours, you get things resolved within 24 hours. And that was important, and that got the buy-in. What I found when we rolled it out in MassDOT also is that some of the hardest nuts to crack who were part of the team were, I don't want to meet every day for 15 minutes. Once they saw the power of that, and that if it's structured and that you don't go past the 15 minutes to do your daily scrum, and you're focused, and you get the answers to the key questions, they started using it in their daily ops. So now folks that were bought in, I call them the hardest nuts to crack, now were using it and continue to use it in their daily operational meetings for running their day-to-day units. So that was important too.

[56:22](https://youtu.be/wi3Ps-ioC70?t=3382) So I think having senior leadership buy-in was important. One of the senior leaders said, give me two weeks. I'll show you that it works. Come back to me if you can't see the results, and then we can work through it, or you can help adjust and pivot where it doesn't work for you.

**Steve Kadish**

[56:39](https://youtu.be/wi3Ps-ioC70?t=3399) So that was important. So thanks, Steve. Thank you, Mary. Matt, you wrote, I found that the sprint length is really key. Telling someone their thing won't be in a two-month cycle can be hard for them to accept, but telling someone we can't do this for the next two weeks and on. Would you talk about what you wrote there for a minute? If you're talking, you're on mute, and if you're not talking, that's okay too, but you got two thumbs up on the comments. So you ended, not that you should lie if it's really a year out, but it makes it easier to separate the immediate tasks from the short-term to-do list, and I think that's important. Chike, anything you want to add before we leave Agile Scrum and go on?

**Chike Ayogu**

[57:38](https://youtu.be/wi3Ps-ioC70?t=3458) I think, I would just reiterate, I think, and Steve, you kind of said it, and I think Mary also said it, there's a ramp up, but the ideal is it proves itself so effective that it just becomes the way that one does business. So it's not this other thing that you're doing, it is just the way that we do business here, and I would say, if you're not getting to that point, you're probably doing it wrong. So again, I think the key to success is making this a better business model versus an additional thing that people have to do.

**Steve Kadish**

[58:11](https://youtu.be/wi3Ps-ioC70?t=3491) Yeah, well said, and hard to do, so, but so powerful, it is so powerful. I'm going to go back to sharing my deck. Thank you, great comments, and thank you guys for jumping in with your oral comments. Okay, okay, okay, so this is the, so, so I just did a quick review of the results framework, and then we did a deeper dive into one of the two tools that I wanted to talk about today, and that was Agile, Agile Scrum. The second is communication, and this is so basic, and so, so abused as to not the right word, but it's, it's, communication is rarely done well, and if you think about in your work, when there, when is there good communication, when is there not good communication, and all the positives and negatives that come from that, and particularly when they're rolling out change, whether that's fixing something, or rolling out a new program, or trying to make improvements, a change is really hard for people, even small changes. It means doing something different. It may not be clear why the change is being made, and what exactly the change is. We all have experienced that. Communication can be the difference maker.

[1:00:00](https://youtu.be/wi3Ps-ioC70?t=3600) but it's too often overlooked, undervalued, or just an afterthought. Communication is not only about how to transmit information, it's how the information is received and how we learn and we all learn in different ways. Communication is also about receiving feedback and making adjustments. So I'm gonna, you know, start here. I will continually refer to Nelson Mandela as I think he's really been one of the most important people in the last hundred years on this planet in terms of advocating the right kinds of changes. And know your audience, right? The most effective communication means presenting things not as you see them, but as to how your audience is able to see and understand. And I really like this quote from Nelson Mandela. If you had talked to a man in the language that he understands, that goes to his head.

[1:01:03](https://youtu.be/wi3Ps-ioC70?t=3663) If you talk to him in his language, that goes to his heart. And so in a way, this is a little bit like follow the facts, that there's the hard data and then the points of pain. And that's in a way what Nelson Mandela is referring to. But you guys are both recipients of communications and you are deliverers or creators of communications. We all learn in different ways. I really want to stress back to the, you know, know your audience is what works for you and what works for various audience. This slide is sort of the common basic stuff, the common internal things that are all tools, a meeting, an email, a call, a Zoom, project-oriented communication, status reports, performance metrics, targeted workshops, things that are related to staff development or rolling out new tools and processes.

[1:02:17](https://youtu.be/wi3Ps-ioC70?t=3737) The in-person training or the online modules or the subject matter experts, these are all things that can be done well or badly or not at all. And then public-facing social media and websites. And there's a ton that could go into there, but I'm trying to, that's not the right word, touch upon, put out there the importance of communication and all the different ways that occurs, because I think it's really important to focus on that. When you are making a change, make a communication plan. And these are some of the key elements of the communication plan. This is one of the tips that we have in our book. And you could literally do this in a format where you have a column of who's impacted, a column next of what is the information to be transmitted, what is the best way to transmit it, a column of when it's needed, how frequently, who's responsible for delivering it and so forth.

[1:03:31](https://youtu.be/wi3Ps-ioC70?t=3811) So this works. This does not take a lot of time to do. It's rarely done and it's rarely done well. But if you do it and you do it as part of your work on your work team or for your overall release, it will give you a much better chance to make sure that you are communicating with and getting information from the people that you need to get stuff to and hear from on a regular basis, because you have a plan to communicate. And I think in the public sector, we're lousy communicators. We're lousy communicators to our staff. We're not good communicators to often our providers and to the public that we serve. And we can be better without a whole lot of extra effort.

[1:04:31](https://youtu.be/wi3Ps-ioC70?t=3871) And so I stress this tool, this idea of making a communication plan of knowing your audience and speaking to that audience in both the language and in the ways that they'll hear it and in the forums, in the means that they'll hear it. And part of a plan is assigning responsibility for who is going to do the email, who's gonna hold the meeting, who is going to reach out to so-and-so, who is going to design the training session and on. There's this rule seven that I have been aware of for quite a long time in the communication literature. And the rule seven is it takes seven times, it takes seven times before a message takes hold.

[1:05:36](https://youtu.be/wi3Ps-ioC70?t=3936) It takes seven times before that a message is delivered for the majority of people before it sticks in any kind of way. So if you have sent the single email or if you have held the single meeting or if you have held the single training and on, that could, for some people, be enough. But for the vast majority of people, that is only the beginning of the kind of training and communication that you need for when you're trying to roll out change significantly. So tip one was about Agile Scrum. This tip is about a communication plan. And so here are five ideas that I wanna share with you that are like, I wanna think kind of beyond the obvious, but they've worked for me in a whole variety of different ways. Consider a weekly blog.

[1:06:39](https://youtu.be/wi3Ps-ioC70?t=3999) There are ways to make these fun, newsy anecdotes, stories, personal stuff. These, when they're done well, become the must-read things for an organization or your team. And I've experienced and done these in organizations from the World Bank to nonprofit organizations to a couple of different ways in the public sector. All-staff meetings take a ton of time, but there's nothing like pulling people together. And I don't mean all project meetings. I mean, if you're in a leadership role or if you're in a position of advising somebody in a leadership role, all-staff meetings are, there's nothing like them. And you should take the time to make them interesting, informative, not to miss.

[1:07:44](https://youtu.be/wi3Ps-ioC70?t=4064) If you can hold them in person, that is super. If they need to be virtual, that's okay too, but they're cool tools and we don't use them enough. This item three is go have coffee. This is a way for, I have it in communication, but it's really about networking and learning. Go have coffee with somebody and without an agenda, there may be something you wanna bring up or ask and they may have something that they wanna ask you, but there's something about taking that time in a more casual way that builds a relationship and a chance to learn how a person's think, learn about their life a little bit that can help with when things get really difficult going forward. During COVID, I used to schedule virtual coffees. Now I try to do that in person, but I suggest you make a list of five people today that you wanna go have coffee with and go have coffee.

[1:08:50](https://youtu.be/wi3Ps-ioC70?t=4130) This item four, we, is a way to, I wanna say, formally create informal advisory groups to help you on efforts. Over and over again, when I have done project work, change work and went out to roll it out with what I, with the project team and a great project team, but didn't include folks from the outside in terms of what this would look like, what would work, what didn't work. In informal, I'm saying informal ways, not formal advisory groups, but reaching out to them. I, every time I did it and I did not create the informal advisory group, I kicked myself for not doing so. So I pass this along to you so you don't have to kick yourself for not doing something like that.

[1:09:51](https://youtu.be/wi3Ps-ioC70?t=4191) And then item five is kind of like a coffee, but this is for you, this is for your work and it's for both learning and communication, but make a list of 10 people that you should be in regular touch with. And these could be people who are work colleagues outside of work colleagues, but I'm referring to the work world, not your personal life. And make an effort to stay in touch with them, a call, an email, a coffee. And so these ideas are not strictly quote unquote communications, but these are very powerful ideas for you, either in a leadership role or in your careers how you guys can perform even better. I guess I'd like you to go into chat for just a minute and do you have good examples of communication tips or learning tips or network tips that you would like to share with your colleagues here?

[1:11:03](https://youtu.be/wi3Ps-ioC70?t=4263) And then we'll end with a case study. Steven, that's great. The rule for seven, it's been attributed and I'd have to go back to actually find the source, but it looked to me like soft research, but it was research-based. It looked to me like soft research, but it was research-based, but I'm not gonna pound the sand on it.

[1:12:09](https://youtu.be/wi3Ps-ioC70?t=4329) Although it made intuitive sense to me that doing something once doesn't make sense. Pervy, impromptu lunch meetings work really well in building one-on-one relationships. That's great. I could have just said to everybody the deck will be shared tomorrow or soon. Okay. Sheba, write to Mary in your chat, just to Mary and why don't you guys exchange an email or something and then you guys can talk a bit more. Okay, I want to, I'm gonna go to the final part of the presentation now. So thank you guys. I'm gonna go back to sharing the deck.

[1:13:14](https://youtu.be/wi3Ps-ioC70?t=4394) So our book, Getting Beyond Politics to Get Important Work Done, the first part of the book has a chapter on each one of the four parts of the results framework. The second part of the book has four case studies. I presented the Department of Children and Families case study last week. There's a case study on COVID. There's a case study on transportation and there's a case study on the health connector where it went from being called by the New York Times the worst in the nation to the first in the nation. And so we're gonna go through these slides quickly. So the health connector, what is the predecessor to Obama Healthcare came from Massachusetts and there was a health connector entity before Obamacare that had rolled out an insurance product to individuals and others at a good cost who otherwise were not able to access health insurance at a reasonable price.

[1:14:31](https://youtu.be/wi3Ps-ioC70?t=4471) Obamacare has passed and it's a huge positive what and when it was rolled out, it actually rolled out to some failure where people had lost the health insurance and then they did a great deep how and created a team to focus on to make sure that Obamacare, the actual systems pieces of it would actually work. And Massachusetts struggled with the. and that were coming from the federal government on its original program. And as opposed to being a national leader, it struggled with those changes and went to actually being the worst performer, according to the New York Times. So the turnaround with the Health Connector began in January-February of 2015 with Governor Baker's election. And these elements, people, facts, how, and results.

[1:15:33](https://youtu.be/wi3Ps-ioC70?t=4533) The new leaders were put in. We recruited an experienced chief operating officer in a new role and made board changes. But we started with people and got very experienced people with systems changes in place, created a COO role and some critical board changes and got that first element of the results framework in place. Second, and I'll show you this, the facts were both points of pain and data evidence. A ton of stuff was coming in from customer experience in terms of what was not working. It actually was super sad to read some of the emails and phone calls and listen to the phone calls. And I'll share one of them with you. And then the leadership team did a deep data evidence mapping of the systems and the operations.

[1:16:40](https://youtu.be/wi3Ps-ioC70?t=4600) And I'm going to show you that. The how, heavy focus on project management, a simple project management methods, red, yellow, green. They created a dedicated team of folks that included Health Connector staff, some consultants, and some folks from the governor's office. Leadership reviews were held daily. And the leadership reviews included staff and consultants. The releases were eight to 12 weeks, but it was a release-based effort. And while they didn't use agile scrum per se, it was the scrum type of principles of short timeframes, accountability, daily conversation about what was happening. The results, the Health Connector was very deliberate in creating a toolkit of performance metrics, operation status board, and project status reports. And we're very deliberate about that right from the get-go of the turnaround effort. So here's a, and I'll just read this quickly, but this is Lewis Gutierrez, who was the newly appointed CEO at the Health Connector, wanted to make sure that when he began the full-on turnaround effort, that folks were focused on real-life people and real-life points of pain. So he put this up in his PowerPoint. My husband was turned away from his appointment this month because he thought we had coverage, but we don't. We should have as we paid and were promised. I have inhalers that are too costly for me to pay out of pocket and an appointment with my doctor on April 2nd.

[1:18:34](https://youtu.be/wi3Ps-ioC70?t=4714) Yet last I heard, I'm not enrolled. Beyond frustrated as this is illegal and no one seems to care enough about the hundreds of families in Massachusetts that are experiencing the same issues with the Health Connector. We need our health insurance that we paid for. Please help. So there's the plea. So data evidence, frustration grew. Among the things that just jumped out was the inability of the call center to function. The call center had a 40% abandonment rate. Some work was begun immediately to focus on the call center, and along with that was the first combination of operational and technical release began in April of 2015. Now, Luis Gutierrez, as opposed to talking about objectives and deliverables, as you might in Agile Scrum, spoke about outcomes and artifacts, and I thought it was a very precise way to talk about what he wanted his leadership team to do and what he wanted the project team to do. And here are some of the examples of what he was looking for in outcomes in this, he set up 45-day release effort. A steep and permanent decline in cases and case escalation were problems. And on a contractual SLA, service level agreements for call center performance. Artifacts, he wanted a clear process analysis and a control point documentation in terms of what was going on. Artifact, he wanted a comprehensive list of operational deficiencies. Artifact, he wanted a plan for immediate, near, and medium-term

[1:20:35](https://youtu.be/wi3Ps-ioC70?t=4835) interventions that had high impact on the customer. Artifacts, so he was talking about outcomes and artifacts. In Agile, we might talk about objectives and deliverables. This is a picture of the six customer-oriented processes that the Health Connector identified as being essential to deliver on in terms of a satisfactory enrollment experience for the customer. And you can see what they are. But this is the basic structure for how the team approached the work. For each one of these processes, the artifact was looking to map the customer experience, identify supporting systems, identify the handoffs, pain points, and so forth. But for each one of these processes is to identify these and then to create a roadmap that was actionable in terms of what to do to address these deficiencies and an operational dashboard for ongoing management so that folks knew what was happening when. Here's a snapshot of the first presentation of anticipated deliverables in the first 45-day release, ending with the last week of a readout to project leadership, an operational roadmap to succeed to the open enrollment, and a toolkit in terms of what to look at. This project governance is another important Achilles heel in many efforts, and there is a lack of clarity in terms of who's making decisions, who can actually help with

[1:22:39](https://youtu.be/wi3Ps-ioC70?t=4959) problem resolution. And it's one of the things that I think is really important in major projects and major rollouts. And so this project governance that we structured, I was involved, but more importantly, the executive director of the Health Connector was involved, the chief operating officer, and then this leadership team included people from the CCA, is the Health Connector, the governor's office, the consulting firms, and so forth. But the idea was that it was fast problem resolution, honest, objective conversation. Daily meetings of this, of two different groups, the Commonwealth Leadership Project team did not include the consultants, and then later in the day was the cross entity Leadership Project team, the group that was noted earlier, plus all the key personnel that were working on the projects. But this is the idea of leadership review. Here's a snapshot of the releases. I think I might have had this in the presentation last week. You could see the releases are relatively short, and they also included a fix it release. So you have release five, and then you have release 5.1 as a fix it release. I know I showed this last time, because I found it so striking. This is an early example of a senior level of executive level project status report. And only one item was green when it was first rolled out, and it was presented to the executive director, to folks in the governor's office. And I just so appreciated the honesty of where things were at.

[1:24:30](https://youtu.be/wi3Ps-ioC70?t=5070) And when you see something that where a project is all green, beware. That there are a variety of reasons why something might be all green, but it's super unusual. And if a project status is all green, maybe that is towards the end, or the goals aren't aggressive enough. I'm just going to run through examples of performance metrics. Here's an incredible improvement in the call center after an intense set of interventions and actions were taken. The picture on the left is the average speed to answer calls. And after the effort began in March to tackle this, it quickly dropped down into much more reasonable levels, as did the abandoned call rate. Here's an example of a way to show progress on dealing with cases, problem cases. And this was an interesting way to demonstrate the jaw, so to speak, of the total number of cases that came in, and then the total number that were still in play. But the total number resolved was over double what had come in during this period of time. Here's a picture of an example of did the release work? Well, here's release five, and that on May 6, the release was put into place for this particular fix. And from there on, this problem of verifying eligibility began this rapid de-escalation. And so I'm going to end in just a second, but a big turnaround. And the Globe noted it, and it was super important.

[1:26:29](https://youtu.be/wi3Ps-ioC70?t=5189) So I close with this slide. The framework works. People are policy. Follow the facts. Focus on how. Push for results. I ask you to think about how it can work for you and some of the tips that we used and ask you to think about what you would do differently. And with that, I'm going to close. I went one minute over, and thank you very, very much.

**Unknown speaker**

[1:26:59](https://youtu.be/wi3Ps-ioC70?t=5219) Thank you, Steve and Jessica. I mean, really, really impressive, amazing. And people, I think, will be enjoying a lot all the presentation and so much, I mean, interesting content and so actual. So thank you, everybody, for joining us. Yes, I mean, there are some comments on the chat that maybe it's always thank you. And we will be sharing tomorrow the recording and the Slack, I mean, the deck and the certificate of attendance. And please, I hope to see you next week, but especially thank you again. Yes, I mean, you can read Steve on the chat, all the comments. Thank you very much, everybody. I don't want to take more time. Thank you very much.

**Steve Kadish**

[1:27:48](https://youtu.be/wi3Ps-ioC70?t=5268) I just want to say one thing. My email is on the presentation deck that you guys will all get. So if anybody wants to reach out, I'd love to talk with you about how to help you with your work.

**Unknown speaker**

[1:28:01](https://youtu.be/wi3Ps-ioC70?t=5281) So thank you. Thank you very much, Steve. Chique, thank you. And Agata, thank you. Very welcome. See you soon. Ending the meeting now.

* * *

Original page: https://innovate-us.org/executing-change-part-2-applying-the-results-framework · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org