---
id: "workshop:making-the-most-of-new-technologies-while-avoiding-the-risks-and-pitfalls"
type: "InnovateUS workshop"
title: "Making the Most of New Technologies While Avoiding the Risks and Pitfalls"
url: "https://innovate-us.org/making-the-most-of-new-technologies-while-avoiding-the-risks-and-pitfalls"
date: "2023-05-02"
duration: "90 minutes"
language: "English"
instructors: ["Santiago Garces"]
recording: "https://www.youtube.com/watch?v=1MPz913JEZQ"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Making the Most of New Technologies While Avoiding the Risks and Pitfalls

InnovateUS workshop · 2 May 2023 · 90 minutes

Join Santiago Garces, Chief Information Officer for the City of Boston, in this 90-minute virtual training, where participants will learn about the uses and advantages of new technologies - including big data, AI, chatbots, blockchain, IoT - for public sector workers as well as mitigation strategies for risks such as privacy, security, and bias. Recording password: d2GdY.hW

## Learning goals

- Learn about the uses and advantages of new technologies - including big data, AI, chatbots, blockchain, IoT - for public sector workers
- Learn mitigation strategies for risks such as privacy, security, and bias

## Speakers

- **Santiago Garces**, Former Chief Information Officer, City of Boston

Recording: https://www.youtube.com/watch?v=1MPz913JEZQ

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/making-the-most-of-new-technologies-while-avoiding-the-risks-and-pitfalls.txt (plain text) and https://innovate-us.org/making-the-most-of-new-technologies-while-avoiding-the-risks-and-pitfalls.json (every segment with its start and end second).

**Santiago Garces**

[0:01](https://youtu.be/1MPz913JEZQ?t=1) All right, so today I figured we'll just do like a quick overview about like what is it that we're talking about in terms of emerging technologies and the risks and how to make sure that we're using technology effectively. Two, how is it that we can measure the value? Three, what are some of the pitfalls that we should look for? These are not comprehensive, but I think there's some that we probably have heard a lot about. Three, fourth, around procurement and other challenges around how is it that we can get these kinds of technologies in government. Then I think I flipped this a little bit and I was going to give you a counterexample on when there's a usage of machine learning in something that ended up being a little bit adverse.

[1:07](https://youtu.be/1MPz913JEZQ?t=67) There's an example that ProPublica shared in about 2016 about the use of some machine learning models to predict a reoffense when setting bail. Then I'm going to share with you a couple of examples of things that I've been close to in South Bend and in Pittsburgh around smart sewers and around using machine learning for capital investment and improving our streets. And then we'll talk a little bit about ChatGPT, which is the hot thing these days, generative AI. So, just for starters, I think that there's like a little yellow line across the screen. All right. So, again, one of the reasons why it's important, based on the name of the conversation, there are pitfalls.

[2:25](https://youtu.be/1MPz913JEZQ?t=145) There are times that when we implement the technology and we don't fully understand how it's going to operate and just based on the novelty of it, there's times that we might end up having adverse outcomes. There was a system, an algorithm that was used and sold in the criminal justice space to judges who had to set bail. And what the tool did is consume information about the offender and try to determine the risk of whether the offender is going to be involved in another crime to be able to set bail, which is something that judges have to do all the time. It is a difficult exercise to do because even without the support of additional data, a judge needs to consider a number of factors in making this decision. So, generally, we would think it is a good application where we might have data and we might have other information to help make a really complex decision.

[3:26](https://youtu.be/1MPz913JEZQ?t=206) In this particular case, what the researchers from ProPublica realized is that this particular algorithm over-identified the risk of re-offense for Black offenders and it underestimated it for white offenders. So, if you're a Black person, it tended to think that you're much more likely to re-commit the offense, so it set the bail to be higher or that you're at a higher risk. And for white offenders, it actually underestimated the risk, so there's bias in two sides. And as you imagine, obviously, these decisions have an implication on these people. And part of what led to the discovery that there's these biases is the researchers started to use the algorithm against known data. They were able to test the algorithm against data to be able to figure out the base on the predictions. Did an outcome actually happen or didn't happen?

[4:27](https://youtu.be/1MPz913JEZQ?t=267) As we talk a little bit about how machine learning works, this is one of the main things to look out for, especially in anything related to AI or machine learning or big data. You might hear it as well as these are systems that help support come to conclusions based on really large amounts of data. But the data that was used to train the model, the data that was used to generate the intelligence matters profoundly. So, if you have data that tends to be biased, like for instance, we know that with 311, some communities tend to call 311 more likely than other communities. And this is well documented. There's some research that's been done, particularly in Boston, but this applies everywhere. So, if we were to build a machine learning model based on 311 data, we'll expect that that model to be biased. It will tend to over-represent or over-predict certain things.

[5:30](https://youtu.be/1MPz913JEZQ?t=330) So, we either have to account for that and adjust the model, but we should also be testing the models to make sure that they are accurate. And the portability of the models in some of these things are some of the things that we need to be considering. Again, the ability of transferring a model that was trained on one set of circumstances under one context and porting it to a different set of circumstances. So, I just wanted to share one of the things, why thinking about this is so important. Now, broadly, technology is neither good nor bad. There's an article recently from the person who pioneered a lot of the work around neural networks, which is this component, this technology tool, that have enabled a lot of the more interesting applications in machine learning and in AI. And they're talking about how there's some really serious risks around artificial intelligence.

[6:37](https://youtu.be/1MPz913JEZQ?t=397) The reality is, technology is neither good nor bad. Technology is either mathematics or some electronics, some tools, but it is really how we use the technology and the context, how we use it, the types of uses that we give to it that bring both value and risk. But there's also an upswing. Like, from government, we usually play a couple of different roles when it comes to technology. We tend to regulate technology, but we also have an opportunity to become users of the technology. And the reality is that there's an upswing and there's an upside in our use of technology. It can help reduce errors. It can help us do things at a scale or a speed that we wouldn't be able to. It can help us improve efficiency. So, if you think about it, like, for instance, with police officers, I was researching, I was curious about this.

[7:41](https://youtu.be/1MPz913JEZQ?t=461) The city of Akron, Ohio, was the first city to have a police car. Before that, police officers rode horses, walked around, I'd imagine. But Akron was the first city to have a police car. And you can start thinking about, like, there's some things that are unique about police cars that weren't available when there was horses and whatnot. It's easier to transport prisoners. It also probably has some other downsides around, like, you lose a little bit of contact with the police officer if they're in a cruiser instead of walking around. And a lot of initiatives around community policing have tried to reverse it. But again, like, you start thinking there's some really good opportunities to things. We often tend to think that technology and emerging technologies are now a problem. But the reality is that for centuries and really since the Industrial Revolution, we've been dealing with these really disruptive introductions of technology into our lives and into government all the time.

[8:45](https://youtu.be/1MPz913JEZQ?t=525) So, I figured we'd just talk about what are some of these kinds of technologies. So, some of these are related to each other. If you're a technologist, you might recognize it. But, yeah, just figured we'd call them out. One area that is very clear, especially in local government, around cameras or computer vision. So, these are types of implementations of usually machine learning or big data, which give us the ability to detect objects, to identify and count how many objects. Do these things called segmented segmentations to try to help us identify what areas of a picture are occupied by different kinds of objects. So, it is potentially very useful because a lot of what we do in government is we oversee physical spaces or production lines, even if you're doing health inspections or safety inspections in the agricultural space.

[9:46](https://youtu.be/1MPz913JEZQ?t=586) Now, there's some concerns. There's some risks. Privacy. So, how is it that we track and care for the data of other people? Security. There's been examples of people using unsecured cameras, especially early on, where the cameras were not encrypted or they have very basic security protections. And someone without due authority might be able to see what the camera is looking at without being, they shouldn't be able to do that. And also costs and storage. So, just as an indicator, one of the rules of thumbs that I've used, especially around public safety and some other spaces, typically when we budget for a camera, it's almost like an iceberg. More or equal amounts of the cost of the technology of the actual camera should go into the storage and networking. And the longer the retention schedule, and again, sometimes if you're doing machine learning applications or other things, you need more data.

[10:51](https://youtu.be/1MPz913JEZQ?t=651) If you're going to be training the models and whatnot, so storage is also like a big driver there to consider. There's embedded sensors, so this is when we put some piece of electronic in the real world. And again, some of these, like, for instance, vehicles and a lot of tools that we have in government for a long time have become more and more sensed. There's more sensors and there's more telemetrics that we could get. But these embedded sensors might be able to give us environmental variables. It might tell us the concentration of particulate matter, about nitrous oxides and other things that we might be interested. So, some risks. First, accuracy. So, a lot of times, especially vendors tend to say, oh, I have a sensor for this or for that. A lot of times, when you're trying to do environmental sensing, you need to have sensors that are of higher quality. And there might be sensors that are of lower quality that give you an answer, but might not be the right answer.

[11:55](https://youtu.be/1MPz913JEZQ?t=715) You might need to calibrate the sensors. You might need to, like, even in the way that they're operated and provisioned. So, for accuracy, a big piece. Because if it's not accurate, then up to some point, like, why are we collecting it, right? The usefulness, too. So, what are we going to do with the data? Is there something that we're actually going to be able to change in our process or in our decision-making based on the data generated by the sensors? A couple of others. Again, and we'll talk a little bit more about generative AI. So, this is ChatGPT and some of the tools that have become prominent lately. And, again, they're incredibly fun and interesting and also concerning new pieces of technology that are using some of the same principles that these machine learning models that kind of in the first place were using. Where, based on these large volumes of data, they were able to determine whether an image had a particular label that was associated with a person or a car.

[12:57](https://youtu.be/1MPz913JEZQ?t=777) They're actually able to work the reverse. So, if you give it text, it can generate an image. Or if you give it a prompt, it might generate a piece of text. So, if you say, I want the memo to tell the City Council of Boston why they should approve the 2024 budget for the Department of Innovation and Technology, it will generate a memo. Again, but accuracy is a concern. Cost, we're still kind of in the early stages where you can use ChatGPT and some of these tools for free. But if you're going to be putting it in production, a lot of what I've seen around costing is around the size of the response. But if you don't know how long the response is going to be, it's hard to estimate how much it's going to cost to be able to do it, especially at scale. Privacy, so when we put information as a prompt, it might get shared into the model. It might be shared with other users as well.

[13:57](https://youtu.be/1MPz913JEZQ?t=837) So, privacy is a concern. Blockchain, again, there's other technologies. Blockchain uses cryptography and some other tools, too. Interesting, it's like the base of Bitcoin and whatnot. To be honest, I haven't found a lot of really useful applications in my work, but there's some that I start thinking especially around. I grew up in Colombia, and I can see in other parts of the world where you might not be able to intrinsically trust what government said. Like if you have a permit or something that was issued by the government, can you trust it? So, blockchain, one of the things that it allows you to do is this multi-party verification and immutability. So, the ability of not being able to edit one record without having – you literally cannot edit one record alone without breaking the chain. So, some interesting applications.

[14:57](https://youtu.be/1MPz913JEZQ?t=897) Obviously, environmental costs. energy use of blockchain. And I was reading somewhere, I think like the usage of blockchain, the energy usage of blockchain at this point exceeds that of some countries, so very concerning. All right, so I figured talk a little bit about, right, how is it that we measure the success, or how is it that we even think about like how much should we be spending on these things, on this technology. So again, these are some reflections based on my experience. Typically in government, we don't generate new services, or we don't generate new revenue based on those services. Like we tax people, we bring it all together, we mix them, and we divvy it up to figure out what services we produce. Whereas Microsoft, or Google, or some other companies might be thinking about monetizing in government, we usually don't think about like what is the upswing in terms of revenue. So we usually think about like what is the benefit in terms of cost.

[16:08](https://youtu.be/1MPz913JEZQ?t=968) So we tend to be thinking what is the value, what would we be doing without the technology, and this is what I was trying to indicate here with the red line, is like what is that baseline expenditure that we would expect to spend on scanning. You know, we want to scan all of the public documents in Boston, and we know that it costs a few hundred thousand dollars to scan, because we have to pay someone to scan it, and then they need to go and make sure that the right documents are getting scanned in the right place, so that we can get the right metadata. But if we're able to use, let's say machine learning, to do the classification of the documents, so like we actually don't have to be thinking up front about like classifying the documents, it just can be all scanned at the same time, and the computer can use machine learning to classify them correctly, maybe that saves like 20% of the cost of the scanning. Well that 20% margin is now what we can think of, that's like the value of using the machine learning, and it'll come at a cost. So usually for me, like if the cost is lower than the benefit, then maybe it's worth exploring, and sometimes when we will talk a little bit about the risk model, about how we procure some of these things, but that's generally how I think about the cost.

[17:32](https://youtu.be/1MPz913JEZQ?t=1052) One other use case, one other way of pricing this is, there's times that we know that there's going to be an increasing cost, so we'll talk about this with the smart sewer example, but sometimes based on a regulatory right, like maybe there's a consent decree, maybe the city is mandated to ensure that some group of people are getting the same level of access to something, and the fines for not doing that would be really high, you can start thinking about like what that expected value of those fines are, like that expected value, even if it's not realized now, and then the reduction in risk around those fines is part of what helps you understand the financial incentive to implement some of these technologies. Let me just make sure there's no questions, so I think that this is good. All right, so let's talk about some risk and pitfalls. I think foremost, the biggest risk and pitfalls is, does the technology actually get you the outcomes that you want? So a lot of times we implement, you know, and it's easy in government, we can get a little bit overwhelmed, we have a lot of responsibility and a lot of work, but sometimes we might think, you know, I joke sometimes, a fitbit doesn't make you fit. So sometimes buying technology doesn't solve the problem, but technology can help complement what you're trying to do to solve the problem. So I'd say like the first pitfall is like, think, is this really a technology problem or is it a process problem? Is it a problem that could be solved by training? Is it, you know, I think starting to think about like how and especially how is it that the technology helps solve the problem is the first piece that avoids a lot of pitfalls and a lot of misadventures. Then we have some more concrete ones around privacy. Again, in the U.S., privacy is regulated in a very fragmented way and there's different,

[19:38](https://youtu.be/1MPz913JEZQ?t=1178) it's most by domain, like if you're in Europe and other places, you have like more comprehensive privacy regulatory regimes, but here, so if you have students, you might be subject to FERPA, do you have criminal justice data, you might be subject to CJIS, but generally we think about some principles around privacy. So do people know, like disclosure, do people know that the data is being collected? Do they know how is it that it's being collected? How long are we keeping it? Do we know that it is accurate so that there's no, you know, so that's an important, all important questions. Especially interesting to see if you're leveraging third parties when you're implementing some of the technology, because sometimes the data or parts of the system are not going to be under your control, so other entities might be, you know, like are they sharing the data and some of these pieces with third parties, so you might not want the, might erode trust. Security, again, when we introduce new technology to an environment that changes the landscape of how we manage, I'll tell you one of the things that we, the government, I tend to think closely about is we, if we're introducing new technology, we have to think about it as an asset, the same way that we think about it as any asset. So we need to be able to account for it, we need to know who's responsible for it, we should probably be thinking about like who's responsible for updating, like if there's security issues, like who's going to be handling those. Making sure that we have security the same way that we would think about any system, what are the security controls, what are the passwords to access the system, who sets the passwords, like are there separation of duties, so that we know that the person that might be able to do something nefarious with the system, or that is different from the person that

[21:41](https://youtu.be/1MPz913JEZQ?t=1301) be able, the separation around like how you describe security rules. And lastly, that we talked a little bit about around bias, so does the system behave fairly to everyone? Can you make sure that the outcomes are, that the impacts are equally distributed? I would also say, and I'll, we'll talk about it briefly, some of these new technologies are actually really good at exposing also bias and equity gaps. So in government, human beings and our processes have biases in themselves as well. So sometimes having a different system actually helps you compare and contrast like the biases that are exposed, some of the biases that existed. Next, we'll talk a little bit about how to buy this stuff. So I usually tend to think again, especially because we're talking about emerging technology, we're making the assumption that some of these are fairly new tools, or the application in a commercial setting is like still fairly new.

[22:56](https://youtu.be/1MPz913JEZQ?t=1376) So we'll start first with this, like the technology is still in development, let's assume they're still at the pilot stage. So at this stage, you'll typically find technology in three places, either university research labs, or you might have a startup, or you might have a more mature company that is like playing with a new, you know, there's a new technology that they've developed. So my advice is, if the technology seems to be at this stage, partner with researchers and others to try to validate and make sure that the technology is working in a way that is consistent. So again, we know based on what even OpenAI and some of the companies and Google and others are sharing about their large language models, they're still not very mature. So because they're not very mature, you probably don't want to commit to spend a lot of money or like put a lot of funding and a lot of infrastructure behind something that's still experimental. So work and testing out to see if there's a good use case, make sure that the information that you're getting is accurate and consistent enough and kind of validate the value. At this stage, probably the funding shouldn't even be coming from you as an operator. You could probably find a grant like the National Science Foundation has been working on that through the civic program and through some other programs and getting exposure to researchers to be able to deploy pilots. If it's a startup, they might have VC funding to be able to venture capital funding to be able to send up the pilot.

[24:37](https://youtu.be/1MPz913JEZQ?t=1477) So let's go to number two. So let's say that you have a technology that generally that we believe works. Let's say that we can detect pedestrians very accurately. The next step is, well, we are not entirely sure how we would pay for knowing the pedestrians are paid, what department would pay for it? What is the business model? How would we use it in our decision making in the city? Are we going to use it for planning? Are we going to use it to change the regulations around the curbs? Maybe we would have more cars parked on the road if we thought that it would slow down traffic or something. I don't know. So at this stage, I think that this is where you can buy in a smaller quantity. I think like usually try, this is where I leverage the fact that you can buy below the procurement threshold. So be able to buy a little bit and then build a test case. Hopefully the closer that you are to reality of how you're thinking that it will operate, the better. So I'll share a little bit when we share about the use cases, how some of these, some examples of this. I think that the main goal of this stage is really trying to understand is this technology something that is going to enable your operations beyond the current state? Is this something that will, and what does success look like? So what would, if the technology, you know, like we talked a little bit about the usefulness, how would the technology being there really solve a problem? So like making sure that you have clarity around that outcome and being able to start measuring against that outcome is important.

[26:24](https://youtu.be/1MPz913JEZQ?t=1584) So let's just put it back to the contest example. Are you actually reducing the number of re-offenses on when you said bail? That'd be like a good outcome to measure. So like, if we're going to be piloting that, that's kind of what we would want to have in mind. And then the final stage is like, we do a full procurement. Again, in some cases you might, in some states, some places there's like single source availability, but if the, if a technology is mature enough that let's say that there's a couple of competitors that can offer it, you could do off an RFP or you could again, leverage some of this whole source procurement. But I think that what is helpful is making sure that now that you understand those metrics, being able to incorporate some of those metrics in the procurement and being able to continue to evaluate how is it that the technology is really providing value to you and to the problems that you're trying to solve and be able to adapt as the circumstances change. Before we go there, maybe we stop and do some questions or maybe do the, let's do the next poll around procurement. Yeah, let me launch it. I mean, we're curious to see what kind of experience people have had around procuring technology. So just launching a simple question to see if also this resonates with you, but have you been involved in procuring technology for government? And I see some answers coming in. I'll give it 20 seconds.

[28:01](https://youtu.be/1MPz913JEZQ?t=1681) We're getting 75% participated. Okay, let me close it in a couple of seconds. Okay, I'm going to end poll and share the results here. Can you see that? Yeah, amazing. All right, I'll read that. So about 13% of you said yes, you often buy technology in government, about 23% said sometimes, and then 64% said no. So it's actually very satisfying to see the results come in real time. And I'd say, maybe we'll keep going with the presentation. And then if people have questions throughout, they can pop it in the chat. And I can either feed them to Santiago right now, or I'll hold them to the end, mostly just because we take off the recording for the Q&A so people can feel free to ask questions candidly. But yeah, feel free to post them in the chat and I'll start making note of them. So I'll share a couple of examples where I've had different levels of involvement. There's, again, many, this kind of stuff is happening every day in government, but all these examples that are familiar. So start with the smart sewers project that this work that started in Southland. And the context here is when the federal government, when Congress passed the Clean Water Act, it mandated operators in the municipalities across the US to make sure that our rivers, streams, lakes, and all these things are clean.

[30:00](https://youtu.be/1MPz913JEZQ?t=1800) In the past, the outflow of both the sewage and the industrial output used to go just to these bodies of water. And with the Clean Water Act, there is a regulation that mandated that we start avoiding that and we started building water treatment plants and we started in a lot of older cities, there's a combined system so the sewage and the stormwater traveled both in the same kind of pipes, in the same pipes. And the challenge is that when it rains too much, when there's so much precipitation, the mix of stormwater and sewage would either go in people's basements or it goes into the river or the ocean. And obviously this is not nice, it's not good, there's E. coli, it's dangerous and it's very harmful to the environment. But we'll talk a little bit about the pricing.

[31:01](https://youtu.be/1MPz913JEZQ?t=1861) So the federal government, EPA worked with the Department of Justice and had a number of these communities sign these consent decrees, which were legally mandated agreements that made cities invest in certain infrastructure to try to bring down the amount of overflow and it created these very expensive fines if there's violations around the overflow. So I'll just give you some, in South Bend, the original consent decree was estimated to cost a billion dollars, it's the biggest capital expenditure in the city's history and most of this money would go either into massive underground tunnels or some massive storage tanks, which has very little value to the residents besides the fact that it prevents sewage going into the river once or twice a year. It's not very visible, it doesn't add much value to the residents. So around, I believe now like 20 years ago, the previous director of public works in the city of South Bend figured there's got to be a better way, like we're about to spend a billion dollars, there's got to be a better way to do this.

[32:15](https://youtu.be/1MPz913JEZQ?t=1935) And he went to the University of Notre Dame and he knew some professors in electrical engineering that had a grant from DARPA, from the Defense Research Agency, and they were trying to detect their application using sensors, trying to detect the chemical plumes and cases of war. And he figured, couldn't we use some of the same technology to try to solve these big problems that the city has? And he met some wonderful researchers. The slides I'm going to share are from, I'd asked a while ago, Luis Montestruco, who was the person who founded the company when he was a PhD student and then he went to create this company. And what they figured is they could get really good, they could get much better data around what was actually happening in the sewers using sensors than just like these mathematical models about how the sewers behave.

[33:15](https://youtu.be/1MPz913JEZQ?t=1995) And sewers grow over a year, so it's very hard to, like most cities don't really know how they're fully connected because they're underground and they've been built for a long time. So what you see here on the left side is the location of all of the different places where they put sensors. So they took out the manholes and they put, I'll show you a picture of it later, the sensors that were embedded in the manhole, the manhole became, had an antenna, and then they had sensors around like the speed of the flow, the volume of the flow, and some other information. Later down the road, because they had so much data that they were collecting through these sensors, they were actually able to use machine learning, and in this case these neural networks that we talked about, to start building models about how is it that our sewer system behaves. And the good thing is using the traditional mathematical models for the sewage system, they actually, they're not very accurate, as I mentioned.

[34:24](https://youtu.be/1MPz913JEZQ?t=2064) Using the machine learning models, they were, so the previous models had about a 20% accuracy rate, like there's like off 20% plus or minus, the machine learning models were within 1% of what was predicted, for whenever there's precipitation, it predicted within 1% of what was going to happen in the sewers. And that finally, having all that intelligence enabled them to actually start turning the sewers into almost robots, so they had this automatic gate in the sewer system that were able to dynamically allocate where there was underutilized parts of the sewer system to be able to store water there before it overflowed. And there's some other uses that we're putting, so I'll show you a little bit, again this describes what I was talking about, there's this progression of, in the first stage was just measuring, just giving this ability into something that had been invisible and impossible to measure real time.

[35:25](https://youtu.be/1MPz913JEZQ?t=2125) In the past, like you'd have a crew of people that would be driving around, opening the manholes, sticking a yard or something to measure and they'd be able to tell you how deep the water was, but they would only be able to do one manhole at a time, and it wasn't real time, it was only when these people were working, and all of a sudden you could get every 15 seconds a data point from all of these different manholes. The second stage was building this machine learning model, and then the third stage was again starting to use this system to try to optimize it. This is the picture that I was telling you about, this is the manhole with the radio underneath it. Some interesting, even from a technology standpoint, building a system that is, sewer systems are tough systems for electronics to operate, so they had to build all of this in a way that was very hardened, if you will. The company is Emnet, which was bought by Simon, and again, it goes back to the outcome, so these are year after year, the amount of overflow, the volume of overflow, and this is I think in millions of gallons that would go into the St. Joseph River in South Bend, and you can see here the improvement, just with the monitoring, and one of the side applications that were not conceived at the beginning when I told you this, because all of a sudden they had all these sensor data, in one of the cases they started noticing that one of the sensors, when it was not raining, one of the sensors started to look like there was an increase in water, and they couldn't understand why, so they went and did that inspection, and they realized that there's this massive root system that was clogging the sewers, that

[37:27](https://youtu.be/1MPz913JEZQ?t=2247) if it had rained, it would have probably got clogged, and there would have been a massive overflow event, because it would have caught with other things, but just with the smallest variation, just with the new data, they were able to clear the clog before it had an impact, and with the full automation, you can see how the volume actually of outflow went down, and what you have in the light blue line is actually the volume of rain, so even though there was more precipitation in South Bend than there had been in the past, and climate change is one of the drivers of this, and you could actually see the change in precipitation patterns on the data that the sewers have. You can see that even though there is more rain, there is actually less net water going into the river. The other piece that is exciting, and going back to the benefit realization, using the data that we have from the system, in my second tour of duty in South Bend, the city was able to successfully renegotiate the agreement with EPA, and instead of spending $1 billion, we were only going to spend about, I think it was $400 million, we were saving $400 million, and it is a massive reduction in the capital investment, and actually the new plan focused much more on green infrastructure and other tools that would have a public benefit, and we wouldn't have been able to do it without all the data, because we were so sure about the answers that we were giving EPA, because we had better data than anyone about the quality of our sewers.

[39:13](https://youtu.be/1MPz913JEZQ?t=2353) Happy red line trending down. We will go to the next example, where I mix some Pittsburgh roots into South Bend experience. So, around 2018, when Mayor Peed was Mayor Peed still, we started conducting these municipal services surveys, inspired by work that had been done in Kansas City, and we got the results, and we noticed that most people in northern parts of the country complain about the quality of the roads, because snow and ice and the changes in temperature are tough on roads, but in our case, we noticed that people were disproportionately upset, so we figured there's something here that is not good, so again, only 13% of people were okay with the condition of major street roads, and only 22% were okay with the condition of their neighborhood roads, compared to the national and the regional benchmark in the Great Lakes area.

[40:28](https://youtu.be/1MPz913JEZQ?t=2428) So the challenge that we had was, what roads do we fix, and how do we prioritize this? And traditionally, we would have done the inspections of the roads by hand, have an inspector go around and do a rating based on a scale, and this is very hard to do consistently, it's expensive, it takes time, so either you do it in-house, and then you're taking people who would be doing other jobs to do this, or you contract it out and it takes time and money. So there was a researcher out of Carnegie Mellon who had been using machine learning, computer vision applications, using cell phones as the sensor, and they would be taking pictures with certain intervals and detecting the quality and the issues of the pavement. And so this seemed very interesting, because all of a sudden we'd be able to get this question of what are the conditions of our pavement across the city in a shorter amount of time.

[41:38](https://youtu.be/1MPz913JEZQ?t=2498) So we decided going to what we talked about, procurement. We knew the technology worked, we were still not sure how well it worked, and we were still not sure how our capital and paving program would be impacted. So we did a small procurement, we procured about 10% of our roads, we procured the assessment for 10% of the roads. And then we compared, we actually had the manual assessment on that 10% of the roads, and we compared the two. And the computer system didn't agree 100% with what the people were saying, but we felt that the loss of accuracy, the discrepancy was okay, because the currency, the ability of having more real-time data and the ability of doing it at a broad, at the scale that we needed was beneficial. And we were able to work with the researchers to map their rating system into the rating system that we used, which is an engineering standard, civil engineering standard.

[42:51](https://youtu.be/1MPz913JEZQ?t=2571) So the good thing is, we were able to create this comprehensive examination of where the roads were in poor condition. You can actually see that this is a map that tries to map out the different demographic data with a map. It'll be hard to, but if you kind of go between the two maps, you'll start seeing that a lot of the areas where there are red dots, meaning that there are poor conditions of road, were actually correlated heavily with where our minority residents live, where our African-American residents live, the yellow dots, where Hispanic residents live, the red dots here. So we were able to uncover this structural equity issue. And again, it might have been because we, for a number of reasons, but we just were not taking care of the roads in the lower income minority neighborhoods, as well as that there were in poorer condition.

[43:55](https://youtu.be/1MPz913JEZQ?t=2635) And we built a capital plan investing, oh, yeah, I have it here. So we invested $25 million, this is the largest investment in city streets, so it's very close to the investment that we had done around smart streets that then became kind of had national attention when it was more about complete streets that he did when he was mayor. And we were able to, over three years, the goal was to bring all of the streets that were in the poor condition back to being good. And we know that this conditions deteriorates, so pavement deteriorates, it's something that we need to be managing. As it deteriorates and some roads become poor, we can then push them into future use. But the thought was that $25 million were going to help us get down from the backlog that we had. And the nice thing is they continued to do the survey well past that.

[45:01](https://youtu.be/1MPz913JEZQ?t=2701) So we're able to see that some of these things, it's not perfect, but it's a significant improvement in the satisfaction of residents around the conditions of streets. Hopefully because of the work that we're able to do. And again, the new technology helped us understand a problem at a scale and with a level of currency that would have been. We would have continued to have conversations around, well, where are the streets? What condition are they in? And continue to have this tautology if it hadn't been for the technology that gave us the telemetry of work. This is the last part before we jump into questions, but I know that there's a lot of attention and a lot of questions probably around Chad GPT and generative AI. So as I mentioned before, generative AI is this new set of applications where we're leveraging some of these same models that allowed us to do detection to be able to actually generate content to match a prompt or to match some text.

[46:11](https://youtu.be/1MPz913JEZQ?t=2771) So what I'm sharing here on my screen is the, this is the first draft of some guidelines that we've been thinking about launching and the mindset that we have is one of responsible experimentation. It is still a little bit too early to fully understand the maturity of the technology to understand how it will be useful to the city. But we know that people are using it. I used it and I think that a number of other people have been playing around with it. So the idea is to give a framework to let people know what they should do. So some of the things that we've talked about is, again, some values around what are the kinds of things that we think that people should be prioritizing when they encounter Chad GPT. So privacy, security, risk management, public purpose. So like being able to articulate and tie back. What is the benefit for a resident if we were to choose to do this, to use the technology? What is the benefit to the residents?

[47:13](https://youtu.be/1MPz913JEZQ?t=2833) And then our main guidelines are don't, everything that is generated by generative AI needs to be reviewed by a person and the person who created it is responsible for it. It's like the same as a calculator. It doesn't matter who, what you type, it's not the calculator's fault. If it gave you a bad response, it's the fault of the person who submitted the prompt. Two, making sure that no private or sensitive information is being shared in the prompt. Because again, there's, especially with some of the free versions of what we use for accessing BARD or Chad GPT. It might be shared with others unknowingly. And then again, being aware of bias and inaccuracy. So just because it's giving you an answer doesn't mean that it is the right answer. But it is pretty cool. And in an organization, like in government, we spend a lot of time generating information, writing memos, also consuming information.

[48:25](https://youtu.be/1MPz913JEZQ?t=2905) So we want to, wouldn't it be nice to summarize a long complex document and have it be simplified and a little bit shorter? Chad GPT and BARD are able to do that fairly well, synthesizing and summarizing. I've tried it with my own writing. That's where I felt comfortable first. And I think that if I was going to make a big important decision, I would definitely make sure that I'm reading the entire document that I'm basing my decision on. But it is just such a good write-around for me. English is not my native language. I sometimes realize that my construction, I think it's grammatically correct, or my English teachers from high school would be very upset. But the style, I think it's able to help me build things with a very natural English voice. And I think it's a really neat tool. So let's wrap it up before we have a discussion. So again, some takeaways.

[49:28](https://youtu.be/1MPz913JEZQ?t=2968) First of all, technology is neither good nor bad. And it is also not new. We've been dealing with new advances in technology for some while. It can be a little bit scary and it can be very exciting, but we should have neither the fear nor the excitement take over. Technology can serve a public purpose. We can use it in government to drive better outcomes for our residents. We can quantify the value of the technology. So there are tools and frameworks that allow us to quantify even financially what the value is and how much we should be spending on this. We need to be aware of the pitfalls around the privacy, security, bias, and ultimately, as I mentioned, does it do what we think that it does? Does it do it accurately? And does it actually help us get to the outcome? If it's not, it can be an experiment, but you know the good thing about experiments is that you don't have to be committed to do them forever.

[50:32](https://youtu.be/1MPz913JEZQ?t=3032) It is okay to experiment responsibly. I think experimentation is part of what helps us understand and get awareness. It helps us understand the limits of things, but we have to do it responsibly. We have to know what is the potential harm and risk associated with the experiments. And it's not something that can be an afterthought. We have to be thoughtful about that upfront, document it, and disclose it as well. That was the other thing that we said with Chad's deputy. Make sure that you're disclosing any time that you're using it. Especially in government, we want to build trust, so sharing how we do things builds trust. And we want to manage and share the risk actively. We don't want to be the victims of the technology. We want to be active participants in the thought, even if you're procuring, even if you're using it, if you're not buying technology but you're in a government organization, you care about this.

[51:34](https://youtu.be/1MPz913JEZQ?t=3094) I think that trying to involve yourself and have your voice be heard in the governance process, both how it gets procured, how it gets operated, I think that is really important. And as residents and citizens, I think that's why I think the experimentation is important. It's part of what helps us get a sense of how does it actually work, so that we're not relying just on what a vendor or what someone else is saying. That we have firsthand knowledge of knowing what it is and what it isn't. So, all right, I think that concludes my remarks and we can hopefully have a very fun conversation about this.

* * *

Original page: https://innovate-us.org/making-the-most-of-new-technologies-while-avoiding-the-risks-and-pitfalls · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org