---
id: "workshop:principles-for-public-sector-ai-evaluation"
type: "InnovateUS workshop"
title: "Principles for Public-Sector AI Evaluation"
url: "https://innovate-us.org/principles-for-public-sector-ai-evaluation"
date: "2026-10-06"
duration: "60 minutes"
series: "Practical Approaches to Evaluating AI for Public Benefit"
instructors: ["Dan Chenok","Michael Chen"]
moderators: ["Deborah Stine"]
recording: "https://www.youtube.com/watch?v=M6w0EZfdS6E"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Principles for Public-Sector AI Evaluation

InnovateUS workshop · 6 October 2026 · 60 minutes

Drawing on lessons from research and practice, this session presents a practical framework for evaluating AI in government. Participants will leave with a set of principles, methods, and questions for assessing AI effectiveness, measuring public value, monitoring performance over time, and making informed decisions about adoption, scaling, redesign, or retirement.

## Learning goals

By the end of this workshop, participants will be able to:

- **Apply** a practical framework for evaluating AI effectiveness and public value in government settings.
- **Use** evidence from testing and ongoing monitoring to assess whether AI systems continue to meet agency needs and performance expectations.
- **Make** informed decisions about when to adopt, scale, redesign, or retire an AI system based on performance, risks, and public outcomes.

## Speakers

- **Dan Chenok**, Executive Director, IBM Center for The Business of Government
- **Michael Chen**, Partnerships and Evaluation Lead, Nava Labs

## Moderators

- **Deborah Stine**, Senior Fellow, Innovate US

Part of the series [Practical Approaches to Evaluating AI for Public Benefit](https://innovate-us.org/workshop-series/evaluating-ai-in-practice-a-benchmarking-series-for-government).

Recording: https://www.youtube.com/watch?v=M6w0EZfdS6E

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/principles-for-public-sector-ai-evaluation.txt (plain text) and https://innovate-us.org/principles-for-public-sector-ai-evaluation.json (every segment with its start and end second).

**Dan Chenok**

[0:00](https://youtu.be/M6w0EZfdS6E?t=0) I see the poll questions coming up as we talk. Really appreciate coming on here. Thanks for the introduction. Honored to be speaking with Michael. Our names are very similar, as you can see from the spelling of the last name. We don't think we're necessarily related, but you never know. I'm also happy to be here on the fifth and final of this series in InnovateUS. I know you've had some fantastic sessions that have come before us. This may be my fifth time on InnovateUS, so I told Beth Novak that next time I see her, I'll get the t-shirt so I can wear that. That'll be terrific. Really looking forward to this discussion. Michael and I are going to start out with a couple of frameworks just to orient people around the concept of evaluation. I know Debbie's going to throw some questions at us, and we hope this can be very interactive with you all putting questions in the chat as well. I think, Debbie, we're going to make the slides available to the participants afterwards, so we're not going to read every word. Don't worry, you don't need to take pictures of every screen as we go. I'll talk about this concept of a framework across the AI life cycle.

[1:15](https://youtu.be/M6w0EZfdS6E?t=75) Just a quick word. I know you put my bio in the intro, but I was in the government for a long time, actually over 15 years, both on Capitol Hill in the Congressional Office of Technology Assessment, and then at the U.S. Office of Management and Budget, where I ended my career as the head of technology and budget policy at OMB. It's sort of the portfolio that's now led by the federal CIO, and I did it before that office was created and actually helped stand up that office. I worked actually with Beth in the Obama transition, running the technology and government reform team for that transition, and then since 2012, I've run the IBM Center for the Business of Government, which is a research team at IBM, and we work with university experts. We've had a university expert from Northeastern University or two as authors over time who write on topics of interest to government leaders, and AI is certainly among those topics, and I'll be drawing on our research for a lot of this framework, so really looking forward to the session.

[2:21](https://youtu.be/M6w0EZfdS6E?t=141) All right, so next slide, please. So, I just wanted to kind of do a bottom line that one of my former bosses used to say that, you know, the ethical business means doing the right thing even when nobody's looking. So, my version of that here is think about responsible AI as the default, and we'll talk about what responsibility means throughout my and Michael's presentation and the discussion, even when you don't have to do it, because, you know, at a time when people, when there's discussion and debate about, you know, how do we regulate AI, how do we set standards, et cetera, if organizations, especially governments, and I know there's great leaders from state governments and some from the federal government and the city level across the country here, if they kind of ask what's the right thing to do, how do we apply this in a way that puts people first, that thinks about not just the technology, but how does it achieve the outcomes that we're trying to achieve, and then not just ask that once, but ask that continuously as you apply this technology to the operation of programs, that that's going to go very far in building trust and responsibility and how we use AI and in helping you to evaluate the use of that and developing that structure. Next slide. And I'll talk through these pretty quickly. So when we talk about evaluating AI, there's actually two levels of this. One is evaluating the tool itself. There's also, and we're not going to talk a lot about this today, but happy to get into this in Q&A, a lot of program evaluators are using AI to do evaluation, so to take the tool and to evaluate programs itself. But in either case, this concept of human first, I was at a conference of the UN General Assembly, sort of a digital track side meeting at that session a couple of weeks ago, and one of the leaders there, one of the government leaders who's from one of the state governments, said we've taken it using the term human in the

[4:25](https://youtu.be/M6w0EZfdS6E?t=265) lead. So the concept of people really should own rather than human in the loop, which implies, you know, somewhere there's a person in there. And I thought that was an interesting concept. So real quick, the five elements of evaluation that we're going to talk about today, standards, principles, methods, measures, and evidence. So I'll just fly through these slides and then turn it over to Michael. So next slide. So at the federal level, there's a statute that I know is mirrored to some extent in state and local government called the Evidence Act. It's actually called the Evidence-Based Policymaking Act. It is a long title. But the idea is let's think of some standards for responsible data use and think about how does that apply to applying artificial intelligence in that usefulness. So think about is something relevant, you know, as you're applying AI, are you doing it just for fun or is it something that's really needed in terms of, and I'm not going to, again, read every word, but looking down to ethics, right? How are we thinking about responsible application, thinking about public protection, and thinking about how would it look if we use this AI application on a benefit system?

[5:45](https://youtu.be/M6w0EZfdS6E?t=345) You know, could we do that in a way that builds trust and doesn't just detract from trust? So just a few thoughts there on these standards. I think next we go to principles on the next slide. So from our research, some of the things that we think about are don't start with the technology, start with the outcome that the program is trying to achieve. Making sure that you're holding people accountable to how those, especially in government staff, you know, how are you applying AI? How are you making sure that you keep people first? Making sure that there's an audit framework around models so that it doesn't become such a black box and that that auditability is explainable and transparent in itself. Building in data protections into the algorithms from the front and then auditing for anomalies, auditing for problems and patterns and the ways that data can be misused and developed by us. And then finally, making sure you don't get lost in a lot of algorithmic gobbledygook, that's a technical term, but really to try to think about how do you explain what's going on to the public that we serve, to our employees, et cetera, in plain language. So next slide. So having thought about standards and principles, there are various different methods that can be used to analyze and evaluate AI. You know, obviously you want to think about what performance measures you're using to evaluate the model. How do you actually sample? Let's say you've got a complex agentic AI framework that's going on. You want to pull out samples of that to audit that and to see, make sure that the model is functioning as it's intended. Test the model, right? Are the adversaries of government and adversaries all over the world are trying to misuse this. So try to think about like what's often in the cybersecurity world called like red teaming or black hatting. And how do you test that model? There are various different toolkits that can be used that are available in the marketplace

[7:48](https://youtu.be/M6w0EZfdS6E?t=468) around bias detection, especially in assessing whether the model is learning from and training from information that reflects the state of the world that the model is supposed to be supporting and isn't using data that doesn't reflect that. So some of the other methods you can see here. Next slide. And then how do you measure, right? Obviously when you're evaluating something, you want to measure it against particular elements. These are fairly straightforward, not a lot of rocket science in these measures, but it's a good framework for is, you know, does the application of AI actually have an effect on program outcomes? Does it serve all people fairly? Does it enable staff and auditors and legislators to understand what's going on? And is it being modeled in a way that builds trust and in a cost-effective way? So some categories of measurement that exist. And so next slide. And then as you think about sort of now, once you've evaluated, at what stage are you at the AI sort of migration frame?

[8:58](https://youtu.be/M6w0EZfdS6E?t=538) You know, are you at an initial adoption? Have you already piloted it and you want to scale? Have you already scaled and now you want to learn from it and redesign? Or is it time to move on? Is it time to say, all right, this system has sort of served its useful purpose and we want to move forward? So you see here some of the criteria that can be used for decision frameworks once the evaluation is done. And then I think I have one more. So anytime you are working with one of your teams, those of you that may not be algorithmic professionals or prompt auditors or some of the new terms, just ask some basic questions. You know, how are we thinking about what the evaluation is going to assess in terms of public outcome? Is the model transparent? Can we see it? Can we explain it? And are there sensitive data, personally identifiable information? And have we secured that data effectively? We built cybersecurity in upfront as a design parameter for the model.

[10:05](https://youtu.be/M6w0EZfdS6E?t=605) We talked a little bit about fairness across different groups. So making sure that the model is serving the communities as the community's composition is reflected in real life. And then the final step is like, you know, typical government decision-making. Is the benefit worth the cost of this? Are we getting the most out of AI? And AI isn't always the automatic answer to every problem. So think about, you know, am I going to get something that, you know, could I use something more lightweight? Like, you know, could I just use a simple text mining protocol to achieve this, which would cost a lot less and serve the taxpayers just as well? So think about those types of questions. So a quick framework, and I look forward to hearing from Michael and then the discussion. Back over to you, Debbie. Great. Thanks, Dan. And just to

**Deborah Stine**

[10:57](https://youtu.be/M6w0EZfdS6E?t=657) remind everybody, if you have questions, please put them into the Q&A box. If you want them directed to a specific speaker, please put their first name at the beginning of your question. So with

**Michael Chen**

[11:08](https://youtu.be/M6w0EZfdS6E?t=668) that, let us go to Michael. Looking good. Okay. Hi, everybody. Yeah. Thank you, Debbie. And thank you, Dan, for that insightful opener. My name is Michael Chen. Super happy to be here with you all. Coming to you from Rochester, New York, where it feels like fall. I see that folks are coming from all over the place, different parts of the country, different parts of the world. I hope wherever you're at, things are good at your side. So let me tell you a little bit about where I'm coming from. I work for a company called Nava PBC, Public Benefit Corporation. Specifically, I'm in a group called Nava Labs, where we try to use philanthropic funding to develop, to experiment with AI tools in service of public benefit access and service delivery. Before all this, I was in academia for a couple of years. And my first job for a couple of years, I was actually a community organizer, so working in affordable housing advocacy.

[12:13](https://youtu.be/M6w0EZfdS6E?t=733) And I say this to say, I think much of what I'll tell you today and the examples I'll use are sort of a reflection of our attempts to leverage AI for, quote, unquote, the good, right? We're building tools for human service caseworkers with the goal of reducing their burden, improving access to public benefits, improving service delivery. And ultimately, I think the North Star for us is always helping to alleviate some of the hardship and difficulties that individuals and families and communities are navigating and get the help that they need. And so I have a brief slide deck, partly to keep myself on track, but also part of our work, it's always in collaboration with government partners and nonprofit organizations. And a lot of what we hear is, you know, government teams are resource constrained or personnel constrained, or just there's not enough bandwidth to think about evaluation in sort of the typical academic sense, right? And maybe the word evaluation in the context of AI might even be sort of triggering, or it might induce some stress and panic, right? Sort of like sweating.

[13:28](https://youtu.be/M6w0EZfdS6E?t=808) And so the deck here is really meant to offer some of our thinking and our lessons learned from working particularly with local county and state government partners about how to design a really, what we hope are practical evaluation studies with the intention of helping to inform quality improvement and storytelling, right? And I think oftentimes we sort of go into this, really setting the stage that this isn't about finding fault or blame or shortcomings, but instead this is sort of a learning opportunity through the lens of evaluation. So the underlying question that we as evaluators or as people learning about AI and thinking about responsible use and ethical use of this emerging technology is what works for whom and under what conditions, right? So I mentioned to you in the beginning that all of our, so far, all the tools that Nava Labs has been building is designed for caseworkers and benefit seeking or help seeking clients. And so really sort of our driving question is like, does the tool work? And if so, who does it work for? You know, what kinds of caseworkers and what sorts of delivery settings, or maybe even navigating specific kinds of benefit programs. And then also under what conditions, right? Thinking about, you know, something as pragmatic as like working on a laptop versus a tablet, or maybe, or does the AI do that?

[15:00](https://youtu.be/M6w0EZfdS6E?t=900) I work when there's not reliable internet access or Wi-Fi connection. But again, as you listen to, you know, as we go through this deck, I want you to think about, my encouragement to you is to think about what might be feasible for you, particularly if you, I would love to hear from you afterwards, if there's one idea that resonated with you that you are curious to try and take back to your team or your organization. So one, sorry, if you heard a dog barking, so one of the, one of the lessons that we learned early on is that oftentimes evaluation is sort of treated as an afterthought. And so what I want to encourage you to do is to think about how might you be able to plan for evaluation before launching a project, right? So here's sort of our seven suggested steps, right? Right. The first thing is you want to begin identifying what outcome you're interested in, and then you want to figure out how you're going to collect data to inform that outcome, right?

[16:02](https://youtu.be/M6w0EZfdS6E?t=962) You want to think about a timeline, what it looks like, who's responsible for what. Oftentimes in project management, you think about who's owning, who's driving, who's the decision maker. You want to make sure you have the capacities, whether that's technical capacity or human capacity to actually collect those data. You want to verify the quality and completeness of that data. Getting down to step five. And then finally, you want to run your analysis and share out those results. And I just want to echo briefly Dan's really important insight earlier, which is too often, I think we forget about the importance of translating and interpreting results and findings to sort of a broad audience, and so wanted to sort of double down, double click, reemphasize sort of the value importance of like making sure that, you know, whatever our evaluation findings may be to put them in plain language. So one of our, another sort of our practice principles or ideas or things that we'd really try to do is to be clear about what we mean by impact or outcome or change, right?

[17:05](https://youtu.be/M6w0EZfdS6E?t=1025) In other words, like what, you know, as we deploy this AI tool, or even as we're designing this AI tool and brainstorming about an AI tool, what is the outcome that we're trying to change or what is the outcome we're trying to drive and for whom? So if you have like any friends who are economists, economists often talk about this idea of a counterfactual, right, which is sort of this thought experiment where, you know, Michael Chen in the present universe here and versus Michael Chen in a parallel universe and in the present universe, I start using the AI tool and in the parallel universe where everything is identical to the current universe, I don't, the other Michael Chen doesn't use the AI tool. And the idea of a counterfactual is we then compare these two Michaels over time and see what difference emerge between the one who uses AI tool versus the one who doesn't, right? Obviously this is a thought experiment. It's not actually possible because as far as I know, there's only one universe, right?

[18:05](https://youtu.be/M6w0EZfdS6E?t=1085) And maybe if there's physicists in the audience, you know, call me out and we'd love to have that separate conversation about parallel universes or multiverses. But I think the practical wisdom here is being as clear and as specific as you can about what is the outcome that you're hoping to change with your AI tool deployment. And part of the reason why that's important is because your definition of impact or outcome will define, will influence how you conduct your measurement, what data you collect, what questions you ask in an interview and so forth. And then the third bullet here, I know the idea of using a logic model, or maybe you might've heard it as a theory of change, might make some of us feel itchy, right? It sort of feels like a classroom exercise or maybe something that you do in school and, you know, that it sort of loses its relevance or you never want to see it again after your, you know, final senior seminar or whatever it is.

[19:06](https://youtu.be/M6w0EZfdS6E?t=1146) But I think we want to submit to you all that there is a value for using logic model in a practical setting beyond just compliance or like making, you know, checking a box. And if you, if you are less familiar with the logic model or a theory of change, it's essentially a cascading series of if then statements, right? So sort of the most simple example you can see here on the slide is that our logic model says if we build an AI chat or if we deploy, if we put an AI chat bot into production, right? If we do that, then it will improve the quality of decision support that caseworkers experience. And then the hope is then that, you know, caseworker burnout, their administrative burden would decrease and improve their overall wellbeing would improve, right? So thinking through this, this series of changes or sometimes you might hear it as like, begin with the end in mind, right? Will help you frame really clearly and specifically about how that process, how that mechanism of change might potentially occur.

[20:11](https://youtu.be/M6w0EZfdS6E?t=1211) And so this is a really busy slide. I apologize, but we will share the deck afterwards and feel free to take a look at what a full sort of more flushed out logic model or theory of change might look like. And I think what's, what's potentially a value to you and your team here, whatever setting you're located at, each of these box could become a measurement that you collect data on, right? And what we have heard or what we have learned from working with county governments in particular is that oftentimes these boxes are already in there. There's already data being collected on these things. And so what this means is that when it comes to the purpose of doing an evaluation, we don't have to build from scratch. Instead, what we can do is sort of reuse some of those metrics that are already being collected to help us gain insight and visibility into what's happening, how we can improve, and ultimately what sorts of outcomes and impact are being, are being realized or accomplished by the AI tool.

[21:12](https://youtu.be/M6w0EZfdS6E?t=1272) A quick note here about measurement, right? And like, I have this conversation with my, with my, with my mom a lot, like if we want to measure weight, right? The, the, the thing to use is I want to step on a scale, right? If we want to measure height, I know that the doctor's office, there's a thing that I can stand up against and it'll tell me what my height is, right? The last thing, like, I don't think most, I would say, I would contend that most of us would reasonably say, it doesn't make sense for me to estimate your weight by just looking at you, right? Nor does it make sense for me to try to measure your weight or your height, whichever one it is, by sort of just, just, you know, taking a guess or putting my, pulling a number out of a hat, right? And we want to use validated measures to measure things that we're interested in understanding. And so as you deploy these AI tools, particularly when it comes to sort of the human impact, right, we want to encourage you to think about using both validated measures.

[22:12](https://youtu.be/M6w0EZfdS6E?t=1332) These are essentially measures that have been studied that have been published on, as well, you can also mix it with developing your own measures, which means creating your own survey questions, testing them with an audience, and making sure that, you know, that you're able to measure something that's really, really tailored and specific to your context. And so I want to share with you sort of this example that we use throughout our AI pilots for caseworkers is something we're interested in understanding is how does the AI tool help, what's the effect of our AI tool on caseworkers' experience of administrative burden, right? And we're borrowing this concept from our partners and friends at Georgetown University's Better Government Lab, where they have hypothesized administrative burden to include these three components. There's learning cost, compliance cost, and psychological cost, right? And so without getting to the weeds, I think the potential suggestion here for you is if you're interested in this thing, right, whatever this thing may be, rather than having to create your own measurement, your own survey questions, your own interview questions, try to see if there's already existing ways of measuring it that are out there, right?

[23:23](https://youtu.be/M6w0EZfdS6E?t=1403) Maybe deploy a CLAWD agent to do that search for you, and the advantage of that could be potentially, I mean, cost savings, time savings, and it also allows you to benchmark your evaluation efforts against what's been studied out there. And this one, just to say, you know, because evaluation is oftentimes about storytelling, where can we get data from? If you think about your program or your department, your team, your division, chances are there's a good wealth of different data sources out there. There's quantitative data that come from surveys. Our AI tools oftentimes have product log data, where we try to build visibility into what's going on with a product so that we can see how the human users are interacting with the tool. If you work for a government agency or if you partner with a government agency, you might be able to get access to their administrative records, right? Enrollment, application, requests for information, all of those can be interesting data points to help inform and help you better understand the impact of your AI tool.

[24:26](https://youtu.be/M6w0EZfdS6E?t=1466) You can also think about doing time studies. So part of what our partnering caseworkers have generously helped us to do is they actually track for about a week's worth of time how they're spending their time, because ultimately what we want to understand is sort of, you know, what is the impact on efficiency on how caseworkers are spending their time as a result of using the AI tool? And I know I mentioned interviews and focus groups and direct observations. And just to say that these qualitative methods are just as valuable, right? You might have heard that, you know, quantitative methods tell you what's happening and qualitative methods or qualitative data tells you why it's happening. And I think our encouragement or our lesson learned is that it's oftentimes much more powerful to sort of blend these two data sources together to really help us understand what's going on with the AI tool. How is it affecting the users or humans that they're intending to help? OK, just a quick note here about how do we know? So we deploy this tool, something's happening.

[25:26](https://youtu.be/M6w0EZfdS6E?t=1526) How do we know if it's related to a tool or if it's related to something else? And so think about this example here on the left, right? So we have a chatbot that's in production, caseworkers are using it. We also ask, how often do you use the chatbot and how much sort of administrative burden do you feel? And we put them on an XY plot and we see that those who say they always use the chatbot seem to have lower burden than those who say they never use the chatbot, right? In other words, the more you use the AI tool, the lower your burden goes, right? Which, again, initially is sort of like, yeah, that's very exciting. It affirms exactly what we had hoped to see. But I wanted to caution you and sort of say that remember that old saying that correlation doesn't necessarily mean causation, right? There's oftentimes potential alternative explanations to consider, one of them being reverse causality, right? So imagine if a caseworker who is sort of has a low case load, has a good amount of free time, is able to sit around and do lots of training or things like that, right?

[26:27](https://youtu.be/M6w0EZfdS6E?t=1587) And maybe they also use that time to use the chatbot just to test, just to see what it's like, right? So in that scenario, it's not actually the chatbot that's reducing the burden, but rather it's the low burden to begin with that's causing the usage, right? And so sort of these scenarios where reverse causality could account for the same pattern that you see are important to think about. And then the other thing is about concurrent factors, right? So we we know, for instance, from our past three pilots at Novel Labs across the US, there's sort of growing sentiments about hesitation or reluctance, or maybe worries about the speed of artificial intelligence, the environmental tolls of building data centers, right? And so there's sort of this general sentiment to work, let's perhaps like shying away from AI tools. Or maybe there's also what's called a public charge sort of chilling effect, right? Where applying for a public benefit program might potentially jeopardize your immigration status, right?

[27:27](https://youtu.be/M6w0EZfdS6E?t=1647) And what that means for us as a technology team trying to build an AI tool to help people apply for benefits, right? And how that could show up in terms of low adoption, low usage over time. It might not be about the tool at all. It might be about sort of the broader things that the society, the social forces, the other larger events, the macro policy things that are changing that's outside of our control. So what do you do, right? Some practical advice for you, what we try to encourage, what we work with our government partners on together is whenever possible, whenever we're testing a new AI product, we want to take measurements before we do anything and after it, right? In other words, sort of this pre post comparison. I have this chart here that just shows the change in administrative burden from a particular pilot we did. And you can see that from before the implementation to after the implementation, right, compliance costs went down, psychological costs went down, and yet learning costs actually increased by a little bit.

[28:30](https://youtu.be/M6w0EZfdS6E?t=1710) So again, this is where you're fairly simplistic, kind of straightforward, where you have everybody essentially take a quick survey and then they do the training, they go through the onboarding, they use the tool. And then after a certain period of time, you ask the same questions again. And then you sort of link them and sort of say, oh, you know, Michael's score was a five at the beginning and ended up at three. So we know it went down by two. Dan's score was at a one in the beginning and became a five at the end. So you know that his score actually went up, right? And depending on what measure you use, right, those numbers could signify different things. If you want to level it up a little bit or add a little bit more rigor, a little bit more oomph into your eval, something to consider is to identify a control group, right? So remember that example I was saying earlier where in a parallel universe, Michael Chen exists, right, but he doesn't use the AI tool. So if there's a way that you can split your participants into two buckets, one bucket of participants will use the tool, will use the AI tool, the other bucket of participants will not get access to the AI tool, right?

[29:39](https://youtu.be/M6w0EZfdS6E?t=1779) And you follow them over time. You take the same before and after measurements, right? And so it gives you a sense of like, what does it mean potentially when a certain subset of your of your of your participants are not getting or not receiving the AI intervention? And so if you look at this graph again, you see that for the control group, which is the red line, there's no, you know, so. In our example, caseworkers don't have access to the AI tool. At the start of a study, their burden score was 3.59, and by the end of the study, their burden score was 3.1. For whatever reason, their burden went down by about .4 points, right? Maybe it's because of timing. It was tailing towards Christmas, it was end of the year, so their caseload lightened, people don't need as much help. Whatever's happening, their burden's improving without doing anything, without receiving our intervention. But if you look at the blue line, those are the caseworkers who get the AI tool. And you see that their burden goes from 3.54 in the beginning, before the tool gets rolled out, and at the end of the pilot, right, their tool drops down to two.

[30:41](https://youtu.be/M6w0EZfdS6E?t=1841) So we can sort of compare the magnitude of change between those who are using our tool versus those who aren't. And that gives us a little bit more of a sense of like, oh, potentially this is how the tool is helping. But if you're skeptical like me, you're probably wondering, okay, what if you put all the super experienced, the super chill caseworkers in the treatment group and they get the AI tool, and we put like all the beginners, and what if there's some sort of selection that we're kind of steering caseworkers to one group versus the other? And so what, I don't have a slide for this, but just to say that if you want to level that up even one step further, you might wanna consider designing a randomized experiment. So that means you have a pre-post, a before and after measurement, you have a treatment and a control group, meaning you have people who use the AI tool and those who don't, and you make that assignment to the treatment or control group randomly, right? And so-

**Deborah Stine**

[31:41](https://youtu.be/M6w0EZfdS6E?t=1901) And so, Mike, we need to finish up pretty quickly here, just so-

**Michael Chen**

[31:44](https://youtu.be/M6w0EZfdS6E?t=1904) Okay, thank you.

**Deborah Stine**

[31:45](https://youtu.be/M6w0EZfdS6E?t=1905) Yeah, thank you, yeah.

**Michael Chen**

[31:46](https://youtu.be/M6w0EZfdS6E?t=1906) So the idea is you randomize and you sort of develop an experiment that gives you a sense of how your AI tool might be working. Okay, last slide here. Happy to connect with you afterwards, questions, you know, working together. This is all our resources, our main website. If you wanted to try our tools, caseworker.navapbc.com, our GitHub, our sources, our tools are open source. We also publish in Substack of stuff that we're learning, mistakes we're making, lessons we're learning. And also if you want, I think this is a 15 page toolkit that expands on what we talked about here today. So thank you.

**Deborah Stine**

[32:26](https://youtu.be/M6w0EZfdS6E?t=1946) Hey, thanks. Yeah, we have a whole slew of questions. We have over 300 people, lots of questions. I wanna make sure that we get to them. So I think given the time, I'm just gonna dive into the audience questions and skip the ones that I have as sort of background. So Dan, let's just start with you. So we have a couple of questions about auditing. You know, like how to do auditing, you know, some of the bias auditing tools, you know, what has been your experience with some of them? So how would you do auditing? Can you give us a little bit more information on that? Oh, you're muted, Dan.

**Dan Chenok**

[33:11](https://youtu.be/M6w0EZfdS6E?t=1991) Apologies. It's an important question. And in fact, I come from an auditor father. So it's a question that's near and dear to my heart. And we have a couple of studies that our center's issued. One pretty recently on sort of using AI for, and auditing AI in the evaluation setting. It's by an author named Daniel Foner. I'll try to put these in the chat when Michael's answering another question. And then actually we had a number of years ago, a paper written for us on algorithmic auditing as a tool that was actually written by a Harvard Kennedy School person who was doing research in that space. And I think that the key is to set up the audit trail so that you can follow the information through its algorithmic progression. That sounds like a fancy sentence, but basically set up the, you know, as the data that goes through an algorithm into some sort of an AI-based tool set, whether that's sort of a traditional analytic tool, a generative tool that's sort of creating a chatbot session or an agentic tool that's performing a number of tasks.

[34:27](https://youtu.be/M6w0EZfdS6E?t=2067) That you have outputs that an auditor can see and access. And you can't do the whole thing all the time, right? There's too much happening in the model for any one person to possibly do it. It would probably take them a lifetime to try to catch every permutation. So think about like, how do you sample, you know, sort of to borrow Michael's term, how do you do a random sample of sort of data access points through the algorithmic progression of that data into the model? And then in the output of that model in how it's being used to provide a service or help a government official make a decision or design a policy. I know a lot of governments are using AI for research into policymaking. So I think, you know, it sounds fairly simple, but make sure that the audit trail can get to the information so that it doesn't get lost in the black box of a model. And make sure you're providing kind of points where the data interfaces so that auditors, whether they're auditors with a capital A and have a certification, or whether it's just your teams who are trying to check to make sure the model's working as intended can get to that point.

**Deborah Stine**

[35:40](https://youtu.be/M6w0EZfdS6E?t=2140) Great, thanks. And Michael, you know, what if, you know, like what you have is sort of like the high-end evaluation, right, that not everybody can do. So can you give some thoughts in terms of like, what's the minimum viable kind of evaluation? And also thinking about like, if you're an independent civil society, you know, how can you conduct an evaluation?

**Michael Chen**

[36:07](https://youtu.be/M6w0EZfdS6E?t=2167) Yeah, thank you, Debbie. You know, I think I'm trying to reflect on all of our pilots and I'll say my honest confession is that I, you might be able to tell from sort of my persona coming across the screen, I get really excited thinking about this stuff. And I, oftentimes my first shot out of the bat is to propose this like, you know, fancy thing that is, turns out very quickly that it's not feasible. And I think for myself, like part of the lesson I've sort of been learning over the past couple of years is that that conversation about what the evaluation looks like has to be built on dialogue with the implementation partner and also like trust building, right? And what, you know, what's meaningful and rigorous for us as a product organization may not be meaningful or relevant to sort of the government agency that we're working with. And so I think what's important is to come to a middle ground, right?

[37:07](https://youtu.be/M6w0EZfdS6E?t=2227) And so for us, you know, I'm thinking about the three pilots that we completed. The flavor, the range of flavor spans from small qualitative studies where we interviewed 20 caseworkers through these one-on-one sessions, right? And we try to draw inference, you know, thematic analysis and things like that from those interviews to the actual actual results and things like that from those interviews to large randomized studies with like 120, 150 caseworkers. And so part of what, I think part of the opportunity and part of the navigation is to figure out like what might be interesting and useful and also evidence building, right, for all of this. I would say, sorry, just to answer your question about what might be sort of a good way to go. I think the two things would be, one, pick measures, select, like defining your outcome and pick measures that align with those outcomes.

[38:08](https://youtu.be/M6w0EZfdS6E?t=2288) And two, if possible, do a baseline and then, you know, have that pre-post so that you can at least know what happened as a result of your intervention. So you can start to get some signal on the direction and potential size of your effect.

**Deborah Stine**

[38:25](https://youtu.be/M6w0EZfdS6E?t=2305) Okay, great. No, thank you. So Dan, one of these questions is asking about sort of measuring the ROI, you know, the return on investment for different AI solutions, balancing benefit risk, licensing, all the different things and also how to screen out vaporware versus solutions that are really needed and effective. I don't even know what is meant by vaporware. So maybe you can explain that one to me. Oh, you're muted, Dan.

**Dan Chenok**

[38:54](https://youtu.be/M6w0EZfdS6E?t=2334) I keep muting myself when Michael's talking. So that background noise gets, so I'll be better, I promise. Vaporware is a longstanding technical term for a technology solution that doesn't really have much behind it other than the marketing shtick. So, you know, beware of something where you can't see under the curtain as to how things really work is that. So the question I think was sort of, how do you measure ROI?

**Deborah Stine**

[39:20](https://youtu.be/M6w0EZfdS6E?t=2360) AI, yeah.

**Dan Chenok**

[39:22](https://youtu.be/M6w0EZfdS6E?t=2362) And so I think that, again, it's about applying the model to, and I'll get a better link to the comment on the public link, applying the model to the outcome that you're trying to achieve. Michael talked about this a little bit as well. So rather like, it's important for technology teams to understand the inner workings of the model. How much throughput is happening at a particular moment in time? How much faster is the compute result happening in a particular application? But in terms of a public sector application of ROI, you wanna think about, is the person getting a benefit faster? And are we getting more benefits at the right amount of money to the right people? And are we catching the bad guys more quickly who are trying to come in and scam the system? Are you using the AI to achieve that? So in a benefit program, you can almost use a little bit of a pre-post, a little bit of a type model to say, all right, once we apply the AI, are we getting better at being, increasing the specificity and sensitivity of the model so that you can reduce errors and you can get more of the correct benefits to the right people.

[40:43](https://youtu.be/M6w0EZfdS6E?t=2443) In a setting like you're applying AI to a transportation agency, do you have less time of people trying to figure out where planes are flying around in the sky because they're sort of the FAA auditing team for safety and can use AI to improve their ability to catch when there might be a risky activity somewhere in the airspace and sort of do the analysis or has been used in the auto industry to increase the ability of a transportation safety engineer to understand what is there a common element to a particular set of accidents happening to a particular vehicle. And maybe that common element is traceable through an AI model back to a plant that a human could not have seen because he or she did not have the analytics power to be able to identify that correlation. So I think that the thing to do is to understand what the public sector application is that you're applying it to and then to think about the almost Michael's point about if you didn't use AI, what would the result be and if you did?

[41:51](https://youtu.be/M6w0EZfdS6E?t=2511) And that can kind of give you a sense of measuring ROI.

**Deborah Stine**

[41:56](https://youtu.be/M6w0EZfdS6E?t=2516) Okay. So Michael, this question is asking about RCT which is randomized control trials. We did have one of our earlier speakers in the series talk about that, but it'd be good to have your thoughts on it as well in terms of measuring for key impact. And then I'm gonna also add another question just for time considerations. Like, what do you think are the key considerations for integrating evaluation into cities policies and procurement frameworks when internal evaluation expertise and review capacity are limited? What are the most feasible and resource efficient approaches?

**Michael Chen**

[42:34](https://youtu.be/M6w0EZfdS6E?t=2554) Yeah. Thank you, Debbie, for that. I think that question came from Aisha. I apologize if I'm saying your name wrong, a fellow Rochester person. So, I think what we've observed in our work with sort of resource constrained teams is that sometimes it's really helpful to bring different partners together to do the work, right? To design the evaluation, to go through data collection, analyze data and so forth. And so let me give you a super specific example for the NAVA AI pilots. So we work, we are a technology company. We work with county government agencies that the caseworkers who are using the AI tool work for the Riverside County Children and Family Commission. And so as part of the evaluation work, we collaborate with academic partners at Georgetown University and Cornell University.

[43:39](https://youtu.be/M6w0EZfdS6E?t=2619) So we have these essentially outside academics who are not invested, they're fairly objective and not really invested in wanting to ensure that the tool is successful and impactful, right? In the same way that, let's say a technology company might be. And so I think part of our success or part of our learning has been like, it's really helpful to bring in an external partner who doesn't have a horse in the race, so to speak, right? Because they can point out the weaknesses, identify opportunities for improvement. And in some ways, I think this is sort of a win-win-win scenario because we can also delegate, we can also kind of shift tasks around, right? So a very clear example is that, RCT data can be messy to clean and to analyze. And so what we're able to do is to essentially farm out or not to farm out, but to share that workload with our academic partners who have access to graduate students, who have computing clusters and can generate fancy graphs that we as a technology company might not be able to, nor would our county government partners would have the bandwidth to pull that off.

[44:51](https://youtu.be/M6w0EZfdS6E?t=2691) And so I think maybe one piece of the puzzle is identifying those like values and capacity aligned partners. Look at your local higher ed institutions who may be looking for these sort of experiential learning or service learning opportunities. And I feel like that might be a good place to start.

**Deborah Stine**

[45:14](https://youtu.be/M6w0EZfdS6E?t=2714) And do you have any thoughts on the RCT part, the randomized control trial?

**Michael Chen**

[45:19](https://youtu.be/M6w0EZfdS6E?t=2719) Oh, sorry, can you repeat? Yeah, what was the RCT question?

**Deborah Stine**

[45:23](https://youtu.be/M6w0EZfdS6E?t=2723) Yeah, yeah, basically, what do you think of it as a method?

**Michael Chen**

[45:26](https://youtu.be/M6w0EZfdS6E?t=2726) Yeah, so, I mean, people talk about RCT as the gold standard for evidence building for causal inference, but I think there are ethical concerns. There's a whole, you know, if you just run a quick Google search, in part, because I'm not the expert on RCT designs, but it's not always feasible, right? And so I think part of the consideration is sort of like, do you really need that level of randomization to answer the question that you want to answer, right? I wish there was a clear cut answer. I would say use RCT when appropriate, but it may not, you know, I wanna echo what Dan's like broader messaging earlier is, like in the same sense that AI may not be the ideal solution to the problem, RCT may not be sort of the go-to to answer your question about cause and effect. Maybe you can replace, you can get even more by conducting in-depth interviews and doing direct observations than thinking about, you know, designing a rigorous, I don't know, step wedge, randomized control trial or something like that.

**Deborah Stine**

[46:38](https://youtu.be/M6w0EZfdS6E?t=2798) Okay. Okay, now we're sort of in the speed round where we got 10 minutes left and we're gonna try to get through everything and give you guys a chance for final thoughts. So Dan, this is a question about that you indicated that AI models should be explainable. However, many current models, particularly complex models are not fully explainable. Given this limitation, is it practical to require explainability and what level of explainability should organizations realistically expect? I would also say explain what you mean by explainability.

**Dan Chenok**

[47:11](https://youtu.be/M6w0EZfdS6E?t=2831) Yeah, so there's different levels of explainability depending on the audience and it's sort of in ascending order of, or descending order of technical, sort of technical level. One, so obviously the code itself, right? The algorithms, the code that's creating the model, the parameters of the LLM, your technology team, CIO, CTO, the people that are responsible for that, they need to understand that and they can see it in a way that, you know, technologists can relate to in terms of code reading. And in fact, there's a lot of new work that's being done on AI as code and sort of AI being able to help people who are in the technology industry be better at their job as coders and sort of accessing that. So sort of one level out from that is the, what I'll call the, if most people here are from state government or city government, the programmatic staff who's actually applying the AI to their program.

[48:15](https://youtu.be/M6w0EZfdS6E?t=2895) And there you wanna make sure that there's an integrated project team within the agency that's working with the AI that develops, here are the expectations we have for our officials to be able to understand what's happening in that model in terms of the program outcomes that we are expecting to see and the operations that we're administering every day. So think of it like almost like a manual, an operator's manual in a non-technical version, but a way that kind of gets to the effect of the AI on the operations inside the agency. And then there's the sort of the actual explainability to the general public who's receiving the service. And that's more, it's almost like if any of you are privacy experts, the concept of layered privacy notices where you have a very simple kind of upfront explanation that's very pretty general, but you can click through it to get to that broader level of understanding so that most people, when you get those privacy notice in the mail, most of the time, you either don't look at it or you might look at the first paragraph and say, oh, this looks great.

[49:21](https://youtu.be/M6w0EZfdS6E?t=2961) So making sure that you're sort of meeting people where they are, don't think about explainability as a one size fits all answer.

**Deborah Stine**

[49:28](https://youtu.be/M6w0EZfdS6E?t=2968) Thanks. And then Michael, just real quick question to you, which I see you're kind of typing in now, but I'm gonna just ask you to say it out loud, which is how can we be clear when advocating for AI to bring as much efficiency and access as possible that there's no substitute for proper staffing and that AI does not replace humans or doing the work of oversight? Please make this quick within like two minutes.

**Michael Chen**

[49:51](https://youtu.be/M6w0EZfdS6E?t=2991) Yes, I think two answers. One is, I think being clear about our intentions, it's not, you know, this tool is not being developed to replace you. It is not, it is meant to support the kind of work that you are doing. Think about caseworkers in our case. The second thing is do it together, right? So it's not sort of, we built this product here, now go use it, but rather we try to engage caseworkers along the way in our design prototype testing process so that they, to sort of share that ownership with our partners.

**Deborah Stine**

[50:25](https://youtu.be/M6w0EZfdS6E?t=3025) Great, so thank you so much. So this is the chance I just wanna give you like your one final thing, like what do you want people to remember tomorrow? You've given a lot of great information. What's the thing that they should really think about? So Dan, let's go with you.

**Dan Chenok**

[50:39](https://youtu.be/M6w0EZfdS6E?t=3039) Think about the world that you want to be serving as a government official and how can AI help you make that world better and not in sort of at a net level, right? There's always gonna be benefits to any application technology and there will be risks. How can you maximize the benefits, provide better service, create a better customer experience, get benefits to people faster, design policy with a greater understanding of the nature of the information that's used to develop that policy? How can you do that while designing the model in a way that limits cybersecurity risk, that make sure that the information that you're using to create the model accurately reflects the state of the world, that you have good governance in your AI. So you've got modelers and prompt engineers as well as lawyers and programmers as lawyers and program officials and even users kind of helping to guide how the AI is used.

[51:41](https://youtu.be/M6w0EZfdS6E?t=3101) So it's being used in a responsible way and to the earlier question, is transparent and can be explained to people at different levels. So don't leave it to the AI experts. Make sure that as you're applying AI, there's a team that can understand how to apply this in a way that maximizes benefit, minimizes risk and importantly, a theme we talked about earlier, make sure that the AI is the right answer, that you may not need that elaborate a solution. You may need a different type of technology solution. There may be different types of AI that you wanna apply. An agentic AI use case is not the same as a generative AI use case is not the same as sort of an analytics use case. So think about like, how do you apply that? So look under the hood and think about what you're trying to get to and how different forms of AI can be used responsibly to get there.

**Deborah Stine**

[52:31](https://youtu.be/M6w0EZfdS6E?t=3151) And you, Michael, what's your final word for our participants?

**Michael Chen**

[52:37](https://youtu.be/M6w0EZfdS6E?t=3157) As a social science nerd, I wanna encourage us to be skeptical, right? To not buy into the hype that maybe a vendor is pitching that's using our tool reduces whatever X percent. I think along with that is, be skeptical and think carefully and ask those tough questions about how is the AI tool going to be integrated into the existing workflow of your users or your stakeholders, right? Is it going to be a completely separate website that they have to click into to get to, or does it sort of fit seamlessly into their existing sort of click patterns or day-to-day work? Because we have seen that, the lack of integration, when sort of things don't really fit together, it becomes sort of a unrealized potential, right? Whatever benefit that AI tool can bring, if it's not well integrated into how humans sort of navigate the world, then it would just be that tool there, right?

[53:40](https://youtu.be/M6w0EZfdS6E?t=3220) So it isn't a, if you build it and they will come scenario.

**Deborah Stine**

[53:45](https://youtu.be/M6w0EZfdS6E?t=3225) Okay, great. Thank you so much. And with that, that ends our session, but let's send a virtual round of applause to our speakers. They did a great job today on a rather complicated topic. And also thanks to everybody who joined us today. This has been a great series and I think we've all learned a lot about practical approaches to evaluating AI. So as a reminder, please fill out the post-workshop survey and suggest topics that you'd like us to cover in the future. Like I said, we're working on 2027 right now to plan the future workshops. So our upcoming workshops, the first is on Thursday this week. It's in the AI and Public Health series called the Basics and the Breakthrough Opportunities. That's this Thursday, October 8th at 2 p.m. And the next week we're starting a new series, the Good, the Bad and the Ugly of Predictive AI.

[54:45](https://youtu.be/M6w0EZfdS6E?t=3285) That is on October 13th, also 2 p.m. Eastern. And then the second on our AI and Public Health combating misinformation and reaching the public. That is on Thursday. So we hope to see you at both these sessions. Thanks so much to everybody for attending. Thanks to our speakers. And until next time, have a great day. Bye everybody, thank you.

* * *

Original page: https://innovate-us.org/principles-for-public-sector-ai-evaluation · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org