---
id: "workshop:trends-in-ai-and-the-impact-for-the-public-sector-feb-13"
type: "InnovateUS workshop"
title: "Trends in AI and the Impact for the Public Sector"
url: "https://innovate-us.org/trends-in-ai-and-the-impact-for-the-public-sector-feb-13"
date: "2024-02-13"
duration: "90 minutes"
language: "English"
instructors: ["Cory Ondrejka"]
recording: "https://www.youtube.com/watch?v=0xl_6OaNh4I"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Trends in AI and the Impact for the Public Sector

InnovateUS workshop · 13 February 2024 · 90 minutes

Join us for a 90-minute workshop with Cory Ondrejka, former Tech Advisor to the CEO at Google and VP of Product at the Google Experience. In this workshop, Ondrejka will delve into the cutting-edge developments in artificial intelligence (AI) and their profound implications for government agencies and public services. From the transformative potential of machine learning and automation in streamlining bureaucratic processes to new uses of generative AI to power citizen engagement, this workshop will provide invaluable insights into how the public sector can responsibly harness the power of AI now, and what to expect in the future. Designed for public sector professionals seeking to stay ahead of the curve and interested in tech's role in the future of government, this workshop will provide an insightful exploration of AI's role in shaping our public institutions.

## Learning goals

- Provide invaluable insights into how the public sector can responsibly harness the power of AI now, and what to expect in the future
- Provide an insightful exploration of AI's role in shaping our public institutions

## Speakers

- **Cory Ondrejka**, Chief Technology Officer (CTO), SmartNews, previously VP of Product Management and Technical Advisor to the CEO at Google

Recording: https://www.youtube.com/watch?v=0xl_6OaNh4I

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/trends-in-ai-and-the-impact-for-the-public-sector-feb-13.txt (plain text) and https://innovate-us.org/trends-in-ai-and-the-impact-for-the-public-sector-feb-13.json (every segment with its start and end second).

**Unknown speaker**

[0:01](https://youtu.be/0xl_6OaNh4I?t=1) And then let me just spotlight you. Great. All right, Corey, over to you.

**Cory Ondrejka**

[0:07](https://youtu.be/0xl_6OaNh4I?t=7) Excellent. In theory, I'm sharing a slide deck right now. I am more familiar with Google Meet than I am with Zoom, so hopefully that is up and working. And thank you so much. I am so amazed by how many folks have shown up. I guess perhaps not surprised there is a fair amount going on right now. So thank you. What I'll also say is I'm going to leave a lot of time for Q&A at the end. We're going to go relatively quickly through a lot of stuff. If I say something that is baffling to you, I'm going to try to be as jargon-free as possible. But if I use jargon that's unfamiliar, the hardest thing in the world is to stop and say, whoa, what was that? But if you're thinking that, like 40 other people on the call are thinking that, so please feel free to interrupt. That is never a problem. I would rather we all are able to move along through this understanding as we go. So as I said on the tin, we're going to be talking about AI and the public sector.

[1:10](https://youtu.be/0xl_6OaNh4I?t=70) Obviously, AI is having a moment. You can't open a paper without a bunch of good stories about AI, whether it's teachers decrypting old manuscripts from Vesuvius, changing how we do translation, helping folks with disabilities get around in the world. But of course, there are moments and there are moments. We're also having a lot of these moments, which are germane to many of your worlds, I expect. And part of what makes this so challenging, of course, is we're in one of these glorious moments of transition, sort of interesting times as it were. I'm sure you have all seen or drawn these curves on a whiteboard at some point. Whenever economists talk about transformation, whenever they talk about changes, they'll talk about overlapping curves where something new gets invented.

[2:11](https://youtu.be/0xl_6OaNh4I?t=131) It gets into mass use. It gets this great exponential growth period. And then it starts slowing. And the next thing comes along. And those moments of transition are moments where lots of things change and things get really, really exciting. And for all of you, I suspect you've already been through some of the major changes of our lifetime. Like we can remember when conventional linear television, newspaper, landline phones were dominant communication channels to the people who needed us, who we were serving. Obviously, that transitioned over the last 15 years. And if we look at whether it's public sector, education, technology, folks are kind of starting to catch up to social media, search, smartphones, I'd say with varying degrees of success. But what is exciting is we're now in a moment where all of those technologies, technologies that in many ways we have barely caught up to, are about to go through their own period of disruption.

[3:20](https://youtu.be/0xl_6OaNh4I?t=200) As generic AI, large language models, I will talk about what those words mean to make sure we're all on the same page. And of course, hardware using those AI technologies are coming. They're coming really quickly because for all of those headlines we just, I just put up, we're at the flat part of the AI curve. We're at the part of the curve where stuff isn't happening quickly yet. We're at the, you know, iPhone one era of smartphones where, you know, a few people had adopted and was getting some buzzy press, but really phones meant Nokia, right? And so AI is that place. So as fast as you think it is moving, be aware that the fast change is still to come. So we're going to go 60-ish minutes. But again, lots of time for Q&A, so this will be next 30 minutes, maybe a little less. History lesson, talk about kind of where this came from. Talk about what LLM means, large language model.

[4:24](https://youtu.be/0xl_6OaNh4I?t=264) We'll define a few other three-letter acronyms to make sure we're all on the same page. What makes this transformation different because it has some different elements compared to past technological transformations of the last 25 years. And of course, what's coming quickly and lots of Q&A. I have succeeded if at the end of this, you all have a better high level understanding of what's going on. Hopefully some ideas. Hopefully you walked out of this going, whoa, I can go do these super cool things. And hopefully the most burning questions have answered. I'll also note some non-goals. I will not have all the answers. This is a very fast-moving space. It is extremely complicated. I hope you stump me. Hope you stump me multiple times. The act of being stumped is probably the most helpful thing we can do here. Again, lots of time for Q&A. This isn't going to be me talking for 60 minutes. Why am I talking to you? I already got a lovely intro from Hannah. My name is Corey Andreic.

[5:24](https://youtu.be/0xl_6OaNh4I?t=324) I'm currently the CTO of Smart News. We're a news aggregator. I'm also a senior fellow here. I have, depending upon who you talk to, a varied career or a short attention span. I was an officer in the United States Navy. I built video games. I built the virtual world Second Life. I was a music executive. I worked at Facebook during the mobile transition. Until recently, I was at Google, but now at Smart News. Through this arc, lots of major points of technological transformation, lots of shipping consumer products that were really changed by those periods of transition. History of AI in a nutshell, particularly the period of AI we are in now. If we go back to the 50s, we were having the first moments of real computing models and starting to explore what seemed possible. Soon after that, the notion of playing with human communication started to take off.

[6:24](https://youtu.be/0xl_6OaNh4I?t=384) There was a very early piece of code called Eliza. It was the first chat bot. It pretended to be a therapist. It is pretty remarkable because it is a few lines of code. It could run on hardware from the 1960s. It fooled people. It did a very good job. It was actually super primitive and didn't really understand very much. It mostly just reflected back to you, which is why it worked well as a therapist. Soon after that, we started exploring neural networks. Neural networks modeled very loosely. Don't overthink this one, but modeled loosely on how the human brain operates. That research actually went through a very long period of sort of AI darkness, it is talked about, where things were super exciting in the 1980s and then nothing happened. It turned out what we were waiting for was much, much, much, much, much better computation capabilities, so the ability of computers to do more math, do lots more math.

[7:30](https://youtu.be/0xl_6OaNh4I?t=450) And right about the same time that the iPhone came out, that math got fast enough that something called deep learning happened, which was the same ideas from basically the 80s, but a billion times faster, 100 billion times faster, incomprehensibly faster. And suddenly AI got exciting again. You may recall around a decade ago, things like in Google search and on Facebook, images, and actually in like iPhoto on Apple, you started seeing images getting labeled. Not perfectly, but like, you know, this is, you can search your photos for your family and friends and locations, things like that. So deep learning is what enabled that. And deep learning led very directly to the research paper that really changed all of this and is responsible for our recent history, which is the Transformer paper.

[8:35](https://youtu.be/0xl_6OaNh4I?t=515) You might hear if you're talking to nerds in the space, they'll talk about attention is all you need, which was the name of the paper. And the only thing that is really relevant is the paper allowed these very, very large models to get trained much, much faster. And given the training time and compute costs are kind of the limitation, being able to train millions of times faster, hundreds of millions of times faster led to models that were millions or hundreds of millions of elements larger. And so from there, we started seeing very rapid movement of both generative AI, we'll talk about what that is, and large language models. We'll talk about what those are. The first one you probably heard of was GPT-2. GPT-2 came out of OpenAI a few years ago, and it made some noise because OpenAI initially said, whoa, this model is so capable, we're not sure if we should release it because it's dangerous.

[9:38](https://youtu.be/0xl_6OaNh4I?t=578) It can imitate humans perfectly. Now, it couldn't do all those things, but it was super impressive. And it started changing how large companies thought about these models. Prior to this point, Google, for example, had been demonstrating models, but not releasing them to the public because the reality is nobody was quite sure how people would use this technology and react to it. OpenAI and GPT really changed that. GPT-3 came after that, and GPT-3 is when it all took off because GPT-3 was the first one where if you were chatting with it, it would just blow your mind because it would act so human-like. Let's be clear. It's not a human. We'll talk about exactly what it is. But it's really good at imitating what a human or a group of humans would say in response to something. For when you are playing AI trivia, GPT, which everybody uses as a word now, is generative pre-trained transformer.

[10:42](https://youtu.be/0xl_6OaNh4I?t=642) And what that means is transformer we talked about, that was the core technology that enabled this. Pre-trained means they do a bunch of work ahead of time to train up this model and then hand it to you so it's just ready to go. And generative because it can create things. So that's how we got to where we are today. And when you hear people talking about AI today, they are generally talking about one of two things. They're either talking about generative AI or they're talking about large language models. In both cases, we're talking about taking some input or inputs and then making a prediction. That's what AI is like. This is not magical, super complicated. The math is really complicated. The compute is really complicated. But what it's trying to do is you throw some input at it and something comes out the other side. And both generative AI in general and large language models, that's what they do. With generative AI, we're generally talking about synthesis.

[11:43](https://youtu.be/0xl_6OaNh4I?t=703) So the notion of taking some input and then creating some output. And so again, input to a prediction. So like if we give Dolly, a generative AI that makes images, an input like this, it predicts that this is the output you want. So it's going from this text input because that's what it takes as input. And that's what it is an output. Well, OK, that's kind of cool. What gets perhaps more interesting, for example, if you go to this website, you will see an infinite supply of faces of people that don't exist. And these are using generative techniques to create credible looking people that aren't actually photographs, even though they look very good. This is simultaneously really amazing. And of course, still very early. Remember, we're just the beginning of the curve.

[12:46](https://youtu.be/0xl_6OaNh4I?t=766) So how does this work? Because it looks like magic. And it turns out it's actually not magic. It's strangely enough, it's noise. Alternately, it's math, but math isn't as fun a word to put on a slide as noise. So if you take a bunch of images. And you slowly add noise to them until they just become an almost random noise field. That's the first step. And that step is called diffusion, where you take an image and go to noise. Then each pixel in that noise is associated with a text description of, in this case, like cat. So then if you go the other way, you could say, give me a cat. And you take from that noise and you go all the way back to all these different cat images. That's what's going on. And when people talk about training data, what they're talking about is lots of pictures of cats. Obviously, lots of pictures of everything, but lots of pictures of cats.

[13:49](https://youtu.be/0xl_6OaNh4I?t=829) You'll also notice lots of kind of funny errors on this slide. So the picture here was actually made by Dolly. I asked her to give me a slide showing how stable diffusion decimates images. And you'll see there are all these spelling errors and things like that because there's no knowledge wrapped up in this. This is just at the individual pixel level, what is the most likely pixel that this is associated with. It's even more on this one. Reverse has a subtle spelling error in it. Diffusion is spelled in a delightful way that I might just steal going forward. These kinds of errors are very common in generative AI. And in fact, it's what you tend to go looking for. Historically, fingers and hands were often very bad. Except all of that's getting better because on top of these models, people are starting to put an error check in. But at its core, you take source data, you turn it into noise and then you go, hey, in the noise, give me a cat.

[14:58](https://youtu.be/0xl_6OaNh4I?t=898) That's what's going on. Big, magical, scary, under the hood. It's actually pretty simple math. Lots of data to noise, image comes out. But why this is important is, of course, you can give it nonsensical text, right? You could say, like, let's modify our old prompt here. And like, say, we want the surface of the moon covered in cheese. And we said we want the earth to be super happy. And again, what's interesting is you can see some errors in how this was interpreted, right? It wasn't strictly covered in cheese. It put an astronaut there because it was a little bit easier, more likely in the data probably, for a person to be celebrating. And if you put a person on the moon, it's an astronaut. And so there is a randomness to this, which is both very useful in some cases, like if you're ideating, but very hard to work around if you want some exact output. But the magic here, if there is magic, is even though we never trained it with blocks of cheese sitting on the moon, it knows what cheese is, it knows what the moon is, and it kind of did its best to pull that together.

[16:09](https://youtu.be/0xl_6OaNh4I?t=969) And again, because it's trained on things that look credible to us, the output tends to look quite credible. And so these models are being used in all kinds of different ways. You know, person that does not exist, I already showed you that. People are doing the same thing with cats, of course, but you're also starting to see it in like architectural previews. Works really well if you've got a large data set of rooms. And this space is moving quickly. So the ability to synthesize credible, but not real versions of kind of anything you can think of. It will come as a shock to all of you, of course, that bad actors apparently exist. And so guess what? If you can synthesize anything, you will synthesize audio and try to manipulate either money or power with it. And so we're starting to see events like this, robocalls that used a fake voice.

[17:12](https://youtu.be/0xl_6OaNh4I?t=1032) So same underlying technology that we talked about. In this case, it would be take a bunch of Joe Biden's audio clips. Lots of his audio clips are available, turn them into noise, and then say, I want him to say something else. So anything you have enough data, you can basically synthesize. So generative AI is about synthesis. The most common examples are text to predicted image, but insert anything you want into text and image in the sentence, and it's coming very quickly. So you could go instead of text as your input, you could give it an image and say, give me a different version of this image, or you could say, give me text, but produce audio, audio produce an image, et cetera, et cetera. Best way to understand this, go play. I'll have links at the end when we're doing Q&A, but OpenAI, so ChatGPT and Dolly work very well together. You go to their website and you go play with this.

[18:12](https://youtu.be/0xl_6OaNh4I?t=1092) You can generate audio, you can do all kinds of fun stuff. Google, it was called BARD until like last week, but now it's called Gemini, and Google's image generator is called Imagine. Same thing, you can play with those and start getting, you can start getting intuition around how this works, what kind of errors it has, and it might also inspire you to think of some use case. All right, so speaking of GPT and Gemini, again, we said two things here, generative AI, large language models. We just talked about generative. Let's talk about large language models. So large language models are specifically about predicting the next token. And by token, I mean, whatever the unit of the large language model is, usually that's text. So you can think of it as predicting the next word for the English majors and English scholars in the audience. You'll be like, no, no, no, token isn't actually a word, it's part of words.

[19:15](https://youtu.be/0xl_6OaNh4I?t=1155) That's totally correct. But when you think about like the root of a word that starts getting pretty close to a token, it's about four characters on average. But again, it's about predicting the next token. And of course, we all experienced a really early version of this when the iPhone came out and did predictive text, right? Near the iPhone was like typing out a keyboard, a virtual keyboard with our thumbs is really hard. And so it's gonna make a prediction. And there are entire websites about spelling correct mistakes. DAMU autocorrect is a very popular one for reasons of propriety. I cannot link to any of the pictures on that because it's usually changing things into curses or inappropriate words. That's what's so funny about it. But these models, these early models weren't particularly great at it. But of course, if you mix lots of data, lots and lots and lots and lots and lots of data, you go text and then you predict text.

[20:15](https://youtu.be/0xl_6OaNh4I?t=1215) So just like we were going text to image, large language models, text to text. And so for example, you can go to a large language model and be like, hey, how would you explain what a large language model is? And ChatGPT, not at all ironically, is quite good at this. It brags a little bit about how advanced this is, blah, blah, blah. I'm not going to read it at you, you can go do this yourself. But of course you can then say, hey, how about without any jargon? And it's like, well, now imagine it's a smart robot that's read almost everything on the internet. This is actually a very good description that really captures how large language models work. Think of it as it goes out and reads everything, and then it can kind of make predictions based on everything it read. And like what we were doing with the cheese, we could also be like, well, can we do it in iambic pentameter? And of course it's going to go and try to do that. Also, because again, it's just making the best possible prediction, right?

[21:16](https://youtu.be/0xl_6OaNh4I?t=1276) And so the key piece in all this, and this applies to the images as well, but it is particularly important with large language model, is this all has to do with prompts. Prompts are what you tell the model or ask the model, if we're going to be more precise, to inform its prediction about the next token. And take a step back, remember large language models, huge pile of text, lots of math, but all they're doing is text, make a prediction, next token. So the way to think about it is your prompt becomes the beginning sequence of those tokens. And it's literally just trying to start adding tokens onto your prompt. And after one token, it then goes, well, okay, now I've got this whole line of prompt and a token, I'm going to add another token, another token, another token, another token.

[22:17](https://youtu.be/0xl_6OaNh4I?t=1337) And so this makes this transformation, when we think about technological transformations, really different than the other ones we've experienced in recent years. So if we go back to 2000, right? So in addition to the fun of the internet, the first internet bubble crash, two things were happening. Consumer 3D was becoming widely available and broadband was penetrating. It's when everybody started getting Wi-Fi and laptops because suddenly you didn't just have computers plugged into an internet connection. Taking advantage of that was battery technology advancements, computer graphics advancements, LCD screen displays, because you couldn't really do laptops until you had much better displays that you could make cheaply. For those with long memories, 2000 was when you were still getting cathode ray tube computer monitors.

[23:19](https://youtu.be/0xl_6OaNh4I?t=1399) The big ones, the back, that weigh 50 pounds, doesn't work so great for a laptop. So if you wanna take advantage of those transitions, a lot of the work was this very deeply technical specialized work. iPhone comes along, mobile transition. Suddenly it was, we need to rebuild all of our apps for phones. So now we're like, okay, all the coders have to go learn Objective-C or Java for Android and go rewrite all of their apps and experiences and run them on phones. So again, there's deeply technical knowledge. Even the beginning of deep learning. So I talked about how deep learning was the step before large language models. This is kind of a nice little description of how deep learning works. You're like, you have kind of an outline of an elephant. You can break it into its pieces and break those pieces into pieces and you can start making pretty good predictions. Doing that is a field of mathematics called linear algebra, linear calculus. It's lots of matrix multiplications and really either awesome or terrifying math depending on where you are on the interest in that part of mathematics.

[24:28](https://youtu.be/0xl_6OaNh4I?t=1468) But again, to really jump onto that space, very, very specialized knowledge. With LLMs, it's really about, can you understand how a person would solve a problem? Because going back to what these large language models are trained on, they're trained on human texts. They're trained on what we've written in many, many different contexts and forms. And so they actually work best when they're approached in how would you explain this to a human, not how would you explain this using deep technical jargon. Imagine five, six years ago, if we were trying to like prompt for that moon picture, we would have had some weird structured series of commands. Prompt, JPEG, square, base, blah, blah, blah. And so we just throw some English at it and trust that it more or less works. And that works because of how we think about large language models.

[25:32](https://youtu.be/0xl_6OaNh4I?t=1532) What that means is getting LLMs to do cool stuff. You have to understand the problem as a person would understand the problem. And then you need to describe it as a person would describe the problem. It's a very different challenge. And let's be clear, as much as there are programs spinning up about prompt engineering and prompt expertise, nobody's an expert at this yet. You all on this call are basically as expert at this as anyone in the world. So it is a very different moment for getting better at this, for understanding what's possible. This isn't go back to college and study computer science or do a JavaScript bootcamp. All right, this is, go play, seriously. We'll share the likes, but if you can use Google, you can find these two things. Go play, go understand what's possible here and understand what these models are really good at, especially large language models, is doing human stuffed with text.

[26:40](https://youtu.be/0xl_6OaNh4I?t=1600) So things they are already better than humans at, summarizing, filling out forms, frankly translation between the top thousand human languages. They are superhuman at those things already. And you can just ask them to go do those things. So go play, start building intuition around this because people are gonna come pitch you on this stuff. And you don't necessarily have to be the one who's gonna go build it. But when some nerd like me walks in and starts hand-waving, you're gonna do better if you have some understanding about it. Also, since we're public sector, let's be super clear. What is valued in the work market of the workplace is about to change. And a lot of fields that figured that they were safe forever from robots. So programmers, lawyers, doctors in some cases, right?

[27:44](https://youtu.be/0xl_6OaNh4I?t=1664) Are about to either get a lot more efficient because of AI helping them or have less of certain styles of work to do. The high level, if your job depends on kind of inefficiencies, AI can now wipe out more of those inefficiencies. And net for society, that's probably good in the long run, but it's gonna be hard on you if that's your livelihood. Public sector aside, this would be a good discussion to have at some point. We will not have time for that today, but see previous point about what's valuable in the future. Okay, so I mentioned a lot of data. So when we talk about training large language models, right, we're talking about like the entire web and plus the entire history of the web, right? Like just unimaginable contents of data. USPTO, excellent source of training data, everything else you can find and then all the computing power.

[28:47](https://youtu.be/0xl_6OaNh4I?t=1727) So, interesting public sector question to ponder, like what data do you have? Do you have data? Is it either valuable to somebody? Should you be training using it? Worth pondering, especially as you start understanding the space. But before we get to questions and I'm just about ready to wind up here, I wanna talk about two gotchas because they matter. And then I wanna talk about a quirk that you will have heard of that isn't as bad as you will have been told. So the first one, look, you are what you train and this is particularly important for the public sector. Let's talk about generative AI for images. This will not come as a shock but the internet has biases in it. I know, I know. No one had ever said that to anyone before but there are in fact biases in the internet. And what happens when you train models with bias data is you just fundamentally amplify the biases.

[29:56](https://youtu.be/0xl_6OaNh4I?t=1796) So whether you're in generative AI or large language models, those. So in this particular case, it turns out if you asked early image generation models to make a CEO, it would make a white male. And for a host of reasons, that's a pretty terrible outcome. And fixing that is non-trivial, but it also means if you are using large language models to generate things, be aware that this could be a problem. Sorry, this will be a problem. It may be a problem that you can deal with, but it will definitely be a challenge. So be aware of this one and don't let it skate by because it will embarrass you at best and potentially really harm people who are counting on you. This one's also a pretty big deal. A computer is expensive. So how on earth do you go pay for that? So there are a whole bunch of complex questions there.

[31:03](https://youtu.be/0xl_6OaNh4I?t=1863) Second problem. And this one, this is about as nerdy as this conversation is going to get, but it's important to understand. So data that you train with actually becomes part of the model. Let me say that again. Training data becomes part of the model, permanently encoded into the model. So I am not a lawyer. So not only don't consider this legal advice, be certain that I have no basis to be giving legal advice at all. But just be aware that if your data was used in training, and there are three different words you will hear that all mean used in training, building a model is kind of obvious. If you're building a model, you would probably assume that your data is used in training. But if you hear about retraining, which is where you bias a model towards certain domains, like if you want a model to be really good at medical, for example, you would bias it with training data toward medical data.

[32:09](https://youtu.be/0xl_6OaNh4I?t=1929) Fine tuning is where you try to make certain answers crisper and better. All three of those permanently encode your data into the model. It's part of the model forever. Which means two things that you should be aware of, whether they're scary or not, are up to you. One, your data can be extracted. So, early in large language models, it was assumed that everything just got blown up into little tiny pieces and nothing could possibly be extracted. It turns out people are clever, and so your data can be extracted from the model. And so, isolated, extracted. If you don't want your data extracted from the model, don't put your data into large language models. Also, let's say you, so, super friendly nerd came up to you and pitched you on awesome large language model project, you're like, that sounds sweet. You did a bunch of deals, you gave them a bunch of data, everything is awesome. And then you decide that you don't like them and it's not great.

[33:11](https://youtu.be/0xl_6OaNh4I?t=1991) Guess what? You can't remove your data. Data is in the model forever. So, be aware of that. So you got to check the fine print. This applies to services as well. GPT has moved very aggressively to not train with your data, which is, I think, great. But be aware of this, particularly for personal data. This can easily be quite embarrassing. Final, hallucinations in large language models are in the news all the time, and they sound really big and scary and terrible. Because again, we're saying, look, here's some input, go predict things. And more important, we want credible predictions, which means everything that comes out of large language models and generative AI, whether it is complete nonsense or not, is credible. There was a super cool study by a Boston consulting group that basically looked at using AIs to do BCG style work.

[34:13](https://youtu.be/0xl_6OaNh4I?t=2053) And the cool part was when the large language models were bad at the solution, they made everybody worse. But the answers were still really trusted. So remember, credible doesn't mean accurate. Many of you are in the political domain, so I suspect you are familiar with this sentence. But it means that if correctness matters, there are other techniques you should use than just asking an LLM to answer the question. Because if you just ask an LLM to answer the question, it is very hard to tell whether it's making stuff up or not. There is a technique, it is called retrieval augmented generation, fancy nerd speak, jargon I know. But what it means is rather than asking the large language model to just make stuff up, you actually put known data kind of next to the model. And you use the large language model to help search the data and marshal the data and hand the data to the recipient.

[35:13](https://youtu.be/0xl_6OaNh4I?t=2113) This has two really nice properties. You can control the quality of the data, so you can ensure accuracy, and your data doesn't go into the model, which is great. See previous point about that. There's even a place to go play two things that are using retrieval augmented generation, so you can see how this works. Google has a notebook product, notebook LLM, and then consensus helps you find key points from scientific papers. Super cool. How do you think about what's possible? Because I've now thrown a bunch of stuff at you. So here's how you should think about it today. Do you have a problem that you could teach a high schooler to solve? And we are going to ignore for the moment having multiple high schoolers and multiple teenagers. We are going to ignore the managing the teenager piece of that. Just from a capability perspective, could you teach this problem to a high schooler and could they solve it? Because you now have a nearly infinite number of high schoolers that you can just pay for. That's kind of the capability of large language models today, and it's very quickly going to be graduate students.

[36:16](https://youtu.be/0xl_6OaNh4I?t=2176) So that's how you should think about the class of problems that you can solve. And it's going to keep getting better really quickly because we are here. So wrapping up, AI is big, scary, and fancy, and cool, and math, and nerd speak. We take away the jargon. It's about taking data, making prediction. We used to have kind of cool tools for that, like progression. Now we have super fancy tools like generative AI and large language models. And go play because we are really just beginning. So I'm going to stop talking. I'm going to leave this up because these are links that are useful. And with that, we've got 20 plus minutes to hopefully have one or two or a thousand questions.

**Unknown speaker**

[36:59](https://youtu.be/0xl_6OaNh4I?t=2219) Thank you so much, Corey. Let me just stop recording.

* * *

Original page: https://innovate-us.org/trends-in-ai-and-the-impact-for-the-public-sector-feb-13 · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org