---
id: "workshop:public-engagement-and-institutional-listening-through-generative-ai"
type: "InnovateUS workshop"
title: "Public Engagement and Institutional Listening Through Generative AI"
url: "https://innovate-us.org/public-engagement-and-institutional-listening-through-generative-ai"
date: "2024-09-26"
duration: "90 minutes"
language: "English"
instructors: ["Eric Gordon","Nigel Jacob"]
recording: "https://www.youtube.com/watch?v=tqdR2E_tml8"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Public Engagement and Institutional Listening Through Generative AI

InnovateUS workshop · 26 September 2024 · 90 minutes

- Learn about the importance of rethinking public engagement strategies in the context of trust deficits within public-serving institutions
- Understand how the availability of generative AI tools creates new opportunities for “generative listening” practices in which public sector workers can make sense of large data sets, provide feedback in multiple languages, and for government agencies to process input with transparency and accountability
- Explore real-world use cases to envision the potential applications of AI-enabled public engagement practices while also addressing and mitigating potential harms associated with these technologies
- Register for the workshop [here](https://airtable.com/appdAyP4l2W9gxQdc/pagx9sXF2F53dI7kn/form)

## Learning goals

- Learn about the importance of rethinking public engagement strategies in the context of trust deficits within public-serving institutions
- Understand how the availability of generative AI tools creates new opportunities for “generative listening” practices in which public sector workers can make sense of large data sets, provide feedback in multiple languages, and for government agencies to process input with transparency and accountability
- Explore real-world use cases to envision the potential applications of AI-enabled public engagement practices while also addressing and mitigating potential harms associated with these technologies
- **Register for the workshop [here](https://airtable.com/appdAyP4l2W9gxQdc/pagx9sXF2F53dI7kn/form)**

## Speakers

- **Eric Gordon**, Professor of Civic Media at Emerson College and director of the Engagement Lab
- **Nigel Jacob**, Emeritus Co-Founder, Boston Mayor's Office of New Urban Mechanics

Recording: https://www.youtube.com/watch?v=tqdR2E_tml8

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/public-engagement-and-institutional-listening-through-generative-ai.txt (plain text) and https://innovate-us.org/public-engagement-and-institutional-listening-through-generative-ai.json (every segment with its start and end second).

**Eric Gordon**

[0:00](https://youtu.be/tqdR2E_tml8?t=0) a little bit about me. As Nigel said, I'm coming to this from the perspective of a university, or from higher ed, and started this lab, which was called the Engagement Game Lab, when I met Nigel. And I remember when I met Nigel, I was going to City Hall to... I was fighting a parking ticket, I believe, that's why I was going to City Hall, and we had planned to meet right after that. And I remember being so upset because, and this was in 2008 or something like that, and someone took out an old cassette recorder and placed it on a desk. And the technology just shot me back in time decades. And I met Nigel just after that, and I thought, Nigel, what is going on in City Hall? What is up with innovation here? And that was the start of what I think has been a beautiful relationship. So yeah, so my work has been in this space of civic media and civic tech, and specifically in public engagement. And I've asked these questions about the role that technology and media can play in the way that people interface with institutions.

[1:17](https://youtu.be/tqdR2E_tml8?t=77) And so that's how I've come to this, and I've been really curious to explore new modalities. And as we've mentioned, at first, I did a lot of exploration around games and play specifically, as perhaps a method by which people can engage more readily within institutional context. And I won't belabor that point, but I have certainly bracketed that work, and understanding that there's real value in creating engaging and even fun experiences for people. But it's also quite important that there's reciprocity on the institutional side, so you can create all sorts of fun experiences, but you might just be civic-washing if there's no response from the institution. And that's what I felt a lot of the work that I was doing was kind of leading down a dark path. And so some of the work that we've been developing over the years have very seriously tried to address that. So anyway, looking forward to the conversation, and I'll hand it back over

**Nigel Jacob**

[2:21](https://youtu.be/tqdR2E_tml8?t=141) to Nigel. Cool. Thanks, Eric. I actually think that when Eric reached out, he was hoping that I could give him a get-out-of-jail-free card for getting any more tickets, because I had one, and not really. But I think that's what he was interested in. So let's dive in here. So I think we're going to start out, again, you will be responsive. If you have thoughts or questions, we will totally be responsive to that. But right now, as of now, the idea is that we're going to try to explore four themes in the work. We're going to talk about public engagement. We're going to talk about civic tech and what all that was. We're going to talk a little bit about the application of AI and generative AI to public engagement. And then lastly, we're going to talk about thoughts on where this work is going and how we imagine moving forward in this work. So that's sort of the plan.

[3:28](https://youtu.be/tqdR2E_tml8?t=208) Okay. So, Eric, the work that we did has had many names. It's been called Civic Innovation. But I think, essentially, we were interested in exploring this notion of public engagement and exploring how government can more effectively engage with the public and the reverse as well. In your upcoming book, I believe that you're trying to say that we need to move on to a new paradigm, onto this idea of generative listening. We're moving from public engagement to generative listening, which is a fascinating idea. Could you say a little bit about that? Can you talk about your thinking in this space and why do you think we need to move on?

**Eric Gordon**

[4:22](https://youtu.be/tqdR2E_tml8?t=262) Yeah, of course. So, just to start, I gave a talk to this group several months ago. So, maybe some of you were able to attend that. And in that talk, I kind of laid out some of the ideas around generative listening. So, apologies if this is redundant. But I guess where I'm coming down on this now is that if public engagement as a modality is really a way that institutions can open up to perspectives outside of them, right? So, it's about creating an interface between the publics that institutions serve and the institution. And that's what we do when we do public engagement. So, we're reaching out, we're trying to understand what constituents think and feel, and then we're using that theoretically to inform decisions, right? So, on a very basic level, that's what we're doing. What has occurred to me, and I kind of hinted at this before, what has occurred to me is that so much of the work that we do has been about soliciting voice, meaning that we reach out to communities, we want people to speak. And often, we are recognized for the amount of people that speak, right? So, the more people that we get to show up to things, the better that looks. And we tend to sort of focus in on just the solicitation of speech, because the institutional incentives are aligned towards that. The problem, though, is that once we get people to speak, it's really, really hard to make sense of what it is that they say, right? We know how to listen interpersonally, most of us. And, you know, and often what that means is a sort of an acknowledgement of what it is that the person is saying, and then some feedback, which is the demonstration that

[6:24](https://youtu.be/tqdR2E_tml8?t=384) you've, in fact, been able to process what it is that they've said, and that you are at least willing to alter your own perspective based on what it is that's being said. So now, take that out on an institutional scale, and we're institutions who are attempting to listen, saying, okay, what what is it that you think? It's incredibly hard mechanically to figure out how do we ask that question authentically, and then take in some sort of data, and then actually be open to what it is that we hear, right? So what we do in order to kind of stage that relationship is that we create these mechanisms of feedback and the solicitation of voice. So we hold meetings, we have online surveys, we have all sorts of tools that we do, we also have interpersonal interactions, of course, through kind of public engagement professionals. So we do all of this stuff to sort of bring in some input. So the question that I'm asking now is then what do we do? How do we make sense of all of that data that comes in in different forms? Some of it is coming in in survey form, some of it is qualitative form, some of it is observational, right? And now we're in a perspective where individuals within large institutions are having to make judgments about what it is that we're hearing, essentially, and then have the capacity to change what it is that we do from the institutional perspective based on what it is that we hear. And we know that we're limited in what we can actually change based on that input, right? But the question, the thing that concerns me is that how can we be transparent about what are the possibilities of change in the sort of interaction that we create through public engagement? So when I say public engagement has kind of run its course, or we need space for a new paradigm, what I'm really saying is that there's a kind of

[8:27](https://youtu.be/tqdR2E_tml8?t=507) centralization that exists within public engagement. There's an institutional perspective that dominates within public engagement. And what I'm suggesting is that a paradigm of listening actually opens up some doors for us to be cognizant of the systemic barriers that are stopping us from paying attention, and to actually reimagine what the institutional levers that we have might be. So let me just give you a few examples, and then we can move on. One of the here's an example of a challenge, right? So Community X in City Y gets over and over again, various agencies from the city go and want to consult with this particular community. And with the best of intentions often, you know, we're going to go and we want to hear from this community. And so the Parks Department is going to do that. Streets is going to reach out to the community health and maybe education, all reaching out to that community to understand what it is that they think. One of the challenges that I'm sure that many of you can relate to is that all of those agencies have a really hard time sharing data. So each agency individually reaches out to that community and does their thing. Well, what's happening right now is that you're actually squandering trust that might have existed or never have existed within that community because the institution completely lacks memory and is taking that community for granted essentially by continually reaching out. So that lack of memory is causing significant challenges in the way that we solicit voice because we're not doing the work of listening, which always requires memory.

[10:15](https://youtu.be/tqdR2E_tml8?t=615) It also suggests a lack of accountability in that we can take in information, but then we're not necessarily held accountable to the decisions that we made. We are sitting on mountains of data and we are sitting on mountains of data and communities know that institutions are sitting on mountains of data and we can't figure out how to make sense of that data. And I'm speaking in generalities, so please don't take it personally when I say things like that. I'm saying generally we can't figure that out. And so I'm sure many of you are doing good work and I want to hear, let's please surface this because this is what this is about. And so I guess the question is, how do we solve that problem? How do we solve the problem where we keep reaching out to the same community over and over again, we lack memory, we lack the ability to make sense of things and we lack the ability to provide appropriate feedback. And that's where I think if we start thinking about this in terms of listening as opposed to public engagement, we might actually be able to imagine different structures for us to do that kind of work. And we can get more into the weeds in that as we move on in this discussion. Does that make sense, Nigel? I mean,

**Nigel Jacob**

[11:29](https://youtu.be/tqdR2E_tml8?t=689) with that long answer to your question. That was a good answer. I expect long answers from you, professor. Out of curiosity, I think I have a sense of it, but and this might be a dumb question, but does the word generative, that obviously it hints at generative AI. Is it suggestive of something? Are we saying, what is the connection there between using generative listening and maybe connecting it

**Eric Gordon**

[12:03](https://youtu.be/tqdR2E_tml8?t=723) back to generative AI? Yeah. I mean, putting generative in the title, I think just sells books. So that's really why it's there. But no, I mean, it is one of the, it's operating on different levels. Yes, it is certainly a kind of nod to generative AI, but more importantly, it's a nod to listening as being an active process and not a passive one. So listening is, in fact, when we are actively listening interpersonally now, when we're actively listening, we're paying very, very close attention and we're doing the work of processing what it is that we're hearing. That's active listening. And then ideally, that active listening actually is generative. What it does is that it not only do we take it in, but it generates new ideas. And so that's the byproduct of active listening. So there's something very intentional about putting generativity at the front of that, because I want to make sure that we don't think about listening as passive or even aggressive. There's other forms of listening like surveillance that I think we need to differentiate from.

[13:18](https://youtu.be/tqdR2E_tml8?t=798) And in my book, I make this distinction between surveillance and generative listening in the following way. I talk about surveillance as being listening without values or rather with unarticulated values. And so when we generatively listen, part of what that means is that we are establishing a listening relationship with clarity about where our shared values are. So think about, again, back to the sort of analogy of interpersonal listening, you know, when we talk, when we speak to someone, it feels a lot safer when we're speaking to that person if we understand why it is they're listening to us, right? So our friends and family, our loved ones, right? We have a shared sense of value, an understanding that there's care and concern when people are listening to us. But even when we're talking to other people, if we have a clear sense of what our shared value is, then the speaker is more likely to speak, you know, more free to speak because they understand how their words will be received.

[14:25](https://youtu.be/tqdR2E_tml8?t=865) But if there's no clarity of values, then people don't understand how their words will be received and they will hold back. They will restrain themselves, right? Because they don't know how their words are going to be used, how their data is going to be used. And so when we look at an institutional level, it's imperative for the institution to actually be able to articulate why it is that it's listening prior to soliciting the speech that it is theoretically listening to.

**Nigel Jacob**

[14:54](https://youtu.be/tqdR2E_tml8?t=894) That's interesting. So one of the, as a A former public sector employee, I mean, the work of local government does not usually, we don't usually talk about values inside City Hall. I think we might talk about strategies or goals or intent, but I don't know that we articulate our work in terms of values. And so, doesn't that already make the conversation difficult? If we don't, if the community is framing their set of values that are important to them, and we really don't have, I mean, we can quote the values of democracy, I suppose, but in terms of having a more significant conversation, how do we do that? You know, what does it mean for government to have values or local government to have values, even in principle?

**Eric Gordon**

[15:55](https://youtu.be/tqdR2E_tml8?t=955) Yeah, I mean, I guess I am seeing that more and more. I am seeing local governments try to articulate a sense of values. And sometimes the way that that happens is not by declaring, this is what we stand for. But sometimes the way that that happens is showing up at community events, as opposed to only planning their own events, right? Like things like that, where there's the sort of demonstration of listening is also a demonstration of values and a kind of openness to what it is that's being shared. So, you know, I can point to several examples within local government that I've seen recently, where there's something explicit about values. It's not just, you know, not just government hiding behind procedure, but actually, you know, sort of putting up front values as a way of explaining procedure, and not just assuming that people are going to talk to you just because you're from the government, you know, like, I'm from the government, I'm here to help doesn't work, right?

[17:00](https://youtu.be/tqdR2E_tml8?t=1020) And so, you know, so there's that need to form relationships in an explicit way, is something that I think is really important. So I hear what you're saying that often government organizations are not in the business of declaring values, because it's also it's hard for a government organization that represents everyone to declare values that may not be received as universal, right? So it's challenging, for sure. But on, you know, the process of local engagement, and the people who do that work, you know, the human beings that are doing that work, it's so deeply relational, and there are values that get communicated within that relational work. And so what I'm trying to get at is like, how do we take that very relational human to human work that's happening in most local government contexts, to an extent, how do we take that and actually create systems that support it, right? So it's not just something that happens over here and over there with this one person in a sea of, you know, in an organization with 40,000 employees, but instead, you know, we can actually create systems that that encourage and incentivize that kind of that kind of work.

[18:32](https://youtu.be/tqdR2E_tml8?t=1112) Okay. Do you have a question for you, Nigel? Sure.

**Nigel Jacob**

[18:37](https://youtu.be/tqdR2E_tml8?t=1117) Yeah, so, you know, we have been, we have been working together for a long time. And one of the first projects that we did together was in, I believe it was 2008. We did a project called Hub 2 that was, that was using something called Second Life. And for I guess for anybody under 40, you probably don't know what that is. But Second Life was a was a multi-user virtual environment, you know, kind of like a, you know, sort of a big online game. And people would create spaces within Second Life and cities were creating spaces within Second Life. And Boston was one of those cities. And one of the first projects we did together was we created Boston Island. And we decided that we wanted to create a, an experience for people to facilitate an interaction with spatial planning.

[19:38](https://youtu.be/tqdR2E_tml8?t=1178) And by creating an environment that people that people kind of shared. And so my question for you is like that was really challenging and interesting work, especially 15 years ago. And I'm wondering as you think back on, on that work, and we were trying to crack the nut of how do we engage the, how do we engage publics differently? How do we do it more equitably? How can we use technology to create a kind of interface? As you look back on that work 15 years ago, what did we learn then? And, and what do you think we, what did we learn then that's still relevant today, I guess, is my question for you.

**Eric Gordon**

[20:18](https://youtu.be/tqdR2E_tml8?t=1218) Yeah, that, that is one of my, my favorite projects. And in a lot of ways, the, it formed a pattern for, for the Mayor's Office of New York Mechanics, especially the way that we were working together. It created a lot of like cultural infrastructure to build on, but it was a great project. What did we learn from that time? I think, I think there were a series of assumptions that people had when in, in, in thinking about that project or engaging in that project that I think we, we, we demonstrated were incorrect these assumptions. So this was a time when, you know, the death of the city was something that people spoke about, because why would you come to a city when you could just be online at home somewhere, right? And so people thought, I remember having conversations with people that, who assumed that the reason that we were using Second Life was so that people could, could be remote.

[21:25](https://youtu.be/tqdR2E_tml8?t=1285) But through the doing of the work we actually had the opposite take that weirdly, I mean we we set up this, it was like a, like a computer lab, it was at the edge of Harvard, and it was a computer lab kind of setup, and we, we, we required the participants to be in the same room. And so the idea of like logging in from home, we realized that it was more powerful to have people in the same room. And so, although it was, it seemed to, you know, I think even when, even nowadays when people think about the utility of these kinds of systems that is, oh, anyone can log in from anywhere. But I think, I think we were right then to believe that that's actually a problem, right? Because it creates distance and psychological distance, it's hard to build empathy when your only interaction is like this. And so it was, so I thought, I found that was fascinating that we were, we're able to sort of demonstrate that assumption about how people use digital technology, we can do better. You know, it's about asking harder questions. And I loved that, again, you know, if we, if we look at, if we look at it with the critical perspective that we have today, we would say that there's all sorts of digital equity issues with a project like that.

[22:47](https://youtu.be/tqdR2E_tml8?t=1367) But even at the time, 15 years ago, we were already thinking about it in terms of like, we knew that a lot of people would be critical of having to touch computers and would need support. And so we employed young people to actually be the eyes and ears and the hands of people interacting. If you didn't want to use, if you didn't want to engage with a computer directly, you could ask, you know, a middle schooler to come over that we had trained and they were standing by to come and be like your digital Sherpa and how you engage with it. Like that was, it was fascinating to me. And so, you know, like I found my sense was that we got like as much in terms of rich dialogue from the, like, think about like we had, we had the, the dialogue happening in the platform, on platform, like the chats that people were having.

[23:49](https://youtu.be/tqdR2E_tml8?t=1429) Also we people having were yelling at each other across the room or nudging each other, you know, next door. So we had like multiple levels of interaction. So it was actually richer than a digital or a in-person meeting where you only had a subset of those things. So that was fascinating. So I think we learned a ton. I think we've actually forgotten some of those lessons, I think, and how we've done digital engagement over the past few years. I think we've, because we haven't had the ability to sort of be in space, in the same physical spaces during the pandemic. We had to do everything over, essentially over Zoom. And so, you know, I think like projects like Hub 2 demonstrated the, you know, if you augment, that's what we, it was called, that's what we called it, that's what you called it, augmented civic deliberation that by overlaying space, the space, the physical space with the digital space, we actually were able to engender all kinds of new behaviors. So that was, that was fascinating to me.

**Nigel Jacob**

[24:52](https://youtu.be/tqdR2E_tml8?t=1492) One of the things about that project that I've been thinking about a lot is, I don't know if you remember, but we had some pushback from, from some of the architects that we were working with. Because with the, we were using the word, we were using the word designs, like that people would create these things in Second Life and we call them designs. And the architect said, no, no, no, we can't call them designs. We can't call them designs. They have to be called virtual sketches. And the, and the concern was that, that the, by asking people to use this new tool to create a vision of something that they wanted in the future, the institution in this case, the architects and the planners, the institution in this case was, was afraid of the accountability that they would have if we called it design. And so, and, and there was, and there was no way for them to take it in and be accountable to it. Right. So, so we had to sort of change the name to virtual sketches. So I've been thinking about that a lot, you know, as, as, as we think about like, how do we create reasonable accountability structures for our institutions?

[25:58](https://youtu.be/tqdR2E_tml8?t=1558) Because I feel like during Hub 2 that 15 years ago, like we were sort of pushed into, you know, keeping it at arm's length. Right. Because, because of the, just so many unknowns about, well, what does it mean if, if somebody creates a design and then submits it, then will they expect for it to be built? You know, like, is that what's going to happen? Right. And so we know that that's not what would have happened, but I, but it was, it was interesting that there was so much fear around that. And, you know, there still remains some, you know, it takes a different shape, but there still remains some concern and some fear around when you ask for input and you get it.

**Eric Gordon**

[26:38](https://youtu.be/tqdR2E_tml8?t=1598) You know, what is the accountability? How can the institution protect itself while also being accountable? Yeah, that is, that is fascinating. Like that whole time period of, we'll call it, so it was often called civic tech or sometimes civic innovation. Now we might call it civic design sometimes. There was a lot of active work being done. You know, it was like 10, 15 years, you know, sometimes it was app based or sometimes it was like platforms that we were exploring the Hub 2 and so on, but there was a lot of that happening. And we were actively engaged in it. You know, I think like both the engagement lab and the mayor's office were trying to take a forward leaning approach and thinking about how we, like, what do we do with this, with these new tools? When you, when you think back at that time of that time period, what, what would you say? What did we learn? What did we, what did we get wrong? You know, what did, what did we, what did we miss entirely? I'm just thinking about your reminiscences and thinking about like the richness of what came out of that time.

[27:49](https://youtu.be/tqdR2E_tml8?t=1669) Yeah. Yeah. I think, you know, as we, as we look back on, on civic tech and I'm sure many, many people here, you know, kind of affiliate or associate with, with that, that movement or that trend as we did and do, you know, I feel like one of the things that happened that, that we did a lot of is we built a lot of apps and, you know, it was, you know, it was sort of the solution for everything. Within, within the government space is that we'll build an app for that. And, you know, organizations like Code for America did some extraordinary work in sort of changing some of the cultures within government to be able to create things. But a lot of what came from that culture shift was the, was the creation of an app or apps. And, and, you know, and the, and the, the, the sad and the funny part about all of that is that like most of those apps didn't persist, right? We like, we created these apps and now we have a massive app graveyard and, you know, and we're, we're, we're sitting on all these things because we didn't think about the infrastructure that was going to be necessary to sustain the apps.

[28:59](https://youtu.be/tqdR2E_tml8?t=1739) Right. So I feel like when I look back on civic tech and not that it's over, but like when I look back on that era of civic tech, I mean, I really think about like we were so quick to, you know, to solve a single problem with a single app. And we, we weren't ready or equipped to address some of the kind of infrastructural or systemic, systemic problems. And so, you know, I think that's, and so I also think about, you know, in a kind of similar vein, the open data movement, which was, you know, you know, very, very powerful and mobilize a lot of people to make data accessible to people. And I think what's, what's interesting about open data is that we did a lot of great work collectively in, in creating data standards and figuring out pathways to, you know, to clean up data and to make data accessible in the form of, or at least available.

[30:00](https://youtu.be/tqdR2E_tml8?t=1800) In the form of, you know, in spreadsheets. But the challenge was, is that, you know, you had to kind of be a data scientist to, you know, to use the data. So we made a lot of data available, but we didn't do the work of making it accessible. And, you know, we didn't really have the tools to make it accessible. But so I feel like, you know, whether we were building apps, or whether we were making data available, we were we were kind of thinking about, we were thinking in the form of transactions, like we were we were getting things done, we were solving the problem that was immediately in front of us. And that was totally appropriate for the time. And now, as we look back on it, we can say, wow, we build all those apps, how do we how do we build apps that are more sustainable? And how do we create? How do we make data available to people in a way that is viable? Right? Like, how do we, how do we actually take that data and make it useful for people? And and I think that's really where where I mean, I know, we're sort of jumping ahead to like, what we're excited about. But like, I think, for me, this is this is where I think we need to go, you know, we need to go to like, really making making things accessible and usable. And I think we have, you know, at this moment of some new gen, gen AI tools, like we have some opportunity there to make some to make this data accessible and usable in a way that it hasn't been in the past. I don't know, what do you what do you think?

**Nigel Jacob**

[31:30](https://youtu.be/tqdR2E_tml8?t=1890) I'm gonna, I'm gonna throw that question back at you, actually. That's, that's fair. I think, so we learned a lot, I think. Because of the, there's a couple of issues. So I think we learned a lot, do we, I don't know, we remember a lot of what we've learned, because of the problem of memory. And so we learned a lot. I don't, but I don't think we remember a lot, because we're doing a lot of this thing, same things over. But I think we learned, we learned how, well, I think one of them was was institutional, I think that, that government can can create cool things, I think, you know, it's sort of a simple minded idea. But like, at that point, there wasn't a really good model for certainly let local government to build high quality, let's call them digital experiences, we'll say, you know, we hadn't done that before. You know, when we were building our first app, app, our two in one app, it was it was called citizens connector. Originally, we got a lot of pushback from everybody from from the public from but most of all from the private sector. Because they said, Why are you building this, we will build it for you and sell it to you. And then we saw it, we asked them, let's see what you would build. And they would show us and it was like, we can do better. These are ugly. You know, it was like Oracle sending us an app that they would build it was it was it was nasty. And so we knew that we could do better by just pushing the envelope in terms of what we're asking the app to do in sort of like the quality and thinking critically about the digital experience. And so I think we learned good lessons about that. I think we learned we learned also that it's not easy to build those things. And and we also learned the problem of sustainability. You know, it's after you've done that once or twice, how do you make sure that it's fresh? You know, and so that brings in all sorts of other

[33:34](https://youtu.be/tqdR2E_tml8?t=2014) ideas. You know, I remember in Boston, we the instinct was, you know, given that we we have so many startups in the Boston area, that the idea was that we will create a product manager and attach the product product manager to the to citizens connect. And the challenge was, and this is this is sort of a I don't know if it's controversial, but it's my thought, you know, I think when I looked at how we we worked with citizens connect, I don't really think it was a product. It's not a product in the usual way that we think about a product in the private sector. It's a service. But it's like, we're not, you know, we're not by thinking of it as a product, I think we're trying to put into a certain category. But training it like, like thinking about it, like, like thinking about the usability in terms of product management is on that made sense. But I think that, you know, the, the, the, the, the logical leap might be to think that, oh, you know, government makes products, too. I don't think it does. No, I think we deliver services, but to go the extra mile to say that they are products. I don't know if that's true. You know, they, they, they can be product like, but I think, um, but, you know, like, we don't have the option to turn these things off once they're out there.

[34:53](https://youtu.be/tqdR2E_tml8?t=2093) Like our responsibility is to deliver service to the public. And so we have less flexibility in how we think of these services. So, I mean, but, you know, these, the questions about, is it a product? Is it not? I mean, these are useful questions. And I think that we did never answer them or, or come to any kind of answer. And so we just sort of kick the can along. And then, like, in the case of, of Boston's Citizens Connect, it sort of, it started great, I thought, and then it was going really good. And, you know, people were using it, and then it started to suffer a little bit, you know, there were a little, and then, so it got, it got less sexy over time. And, and, and, you know, at some point we should have probably just turned the lights off on that. But that's, again, something that government doesn't easily do. You know, how do you, like, when is enough enough? You know, so, and those are, those are questions that we need to ask, but we, we don't know how to answer those kinds of questions. So lots of- So that's interesting. So you're, you're, you're saying it's not just that we don't invest in infrastructure, which we know that infrastructure is hard to make sexy, but, so we don't invest in infrastructure, but you're also saying almost the opposite of that, which is, which is that we also don't know when to quit and that will sustain things without, you know, without evaluating them. And we'll just, we, we, we need procedures to understand when things ought to be sunsetted.

[36:19](https://youtu.be/tqdR2E_tml8?t=2179) And we, and we don't have that. So either, either we build apps and they die immediately, or we build apps and then they just become zombies and they just like live forever. Right. That's so like figuring out ways of, of, of assessing value with these tools that we're building, I think is super important. Yeah. And that's certainly true of government services in general. I think, you know, we, in New York mechanics, we were, we worked across, you know, a million different departments and agencies, homelessness and economic development and so on. And often the criteria for assessing whether or not something should live on, it was really, like, can we say that we have a program that, that supports a particular, you know, demography, right. Is, is, are we delivering services to homeless veterans, you know, and if we do good check, there was never a question of, is it a good service? Do the, in this case, the homeless vets like the service that we didn't ever, we would, we would often use, you know, vanity metrics to assess whether or not things should continue but never qualitative metrics about do people like it? Is it, you know, would, are they, would they recommend it all things being equal and so on? So I think those are, you know, I don't think local government is sort of rigged to be able to, to engage in that kind of dialogue, at least, at least my local government. And so that was an issue.

[37:52](https://youtu.be/tqdR2E_tml8?t=2272) I mean, I think that's, that's something that, you know, when I look at the civic tech and specifically organizations like, like Code for America, one of the things that they did was they sort of brought in that ethos of, of, you know, of sort of, you know, usability, right into government services. So I think what was interesting about that is that like, yes, tech was a product, but tech was also a way of thinking that, you know, did do some interesting things, you know, and was able to sort of change the way that, that, that people within government solve problems when they were able to think more like product developers. Yeah. Yeah. Yeah.

**Eric Gordon**

[38:31](https://youtu.be/tqdR2E_tml8?t=2311) Um, should we talk a little bit, we just had a paper published. Should we, should we talk a little bit about that? Um, so yeah, let's do that. Yeah. So we, and some friends and colleagues of ours, uh, our paper that we were working on through the spring just went live. It's called, um, people powered generative AI. And it was part of an ongoing effort. You know, it was, it was sort of a, like a cast of, it felt like hundreds, uh, of, of the people working on this paper, but we had, you know, a huge range of people. We had legal scholars and we had folks studying the history of, uh, technology and planners and so on. And, and, and, and us looking at, um, like what, how do we use like generative AI in that context of public engagement? Like what, what can we, what can we say? What, you know, what questions can we ask? You know, what, um, is it useful even in general? You know, it was, it was, it was the range of all these things that we cover less in territory. And we talked about planning and we talked about trust and institutions. Um, what did you, what did you kind of draw from, from, from that work in writing the paper? Yeah. I mean, I think that, I mean, it all kind of blurs together in my mind and I'll talk about the paper and some of the insights in the paper, but then I think it'll also be useful to, to kind of move into some of the work we're also doing together around our Rethink AI, um, uh, uh, program. So, but let me just talk about this. I mean, I think what's, what's interesting is that, you know, we were asking the question about, about how, how generative AI, um, can impact, um, civic engagement in the ways that we've been, we've been talking about. And we, we define that very broadly, like more broadly than what we're, we're defining right here. Um, and we looked at the way in which, um, LLMs are being,

[40:37](https://youtu.be/tqdR2E_tml8?t=2437) mostly LLMs are being incorporated into, into government agencies. The majority of the use cases that we were able to, to find are, are internal to government. So we see that a lot of governments are using LLMs now, um, for, uh, for aggregation, for synopsis, um, so for insight generation of, of documents that, that, that exists and also for some, um, some aids in mental writing and, and, uh, and things like that. So that, that there's almost widespread use of, of, of, um, LLMs in, in that context, that doesn't get at the interface problem that we talked about earlier, right. It doesn't get at like, um, this question of, of, you know, it's, it's great that, that within institutions there can be better communication or easier use or, or, um, kind of optimized efficiency. Um, but how should we be thinking about the, these tools as sort of breaking that, that porous barrier between the, you know, between the institution and the publics that they serve. And here's where it got interesting, right. Because, because what's, what's happening on the kind of public facing side of things is mostly not a lot. And, you know, what we have are some, some pretty small scale, um, experiments that are, that are taking place.

[41:55](https://youtu.be/tqdR2E_tml8?t=2515) And for a lot of reasons, right. We, we see that, um, you know, there's all, there's tons of questions, uh, that still exist, both about the viability of the technology itself, about the safety of the technology, obviously privacy concerns, copyright concerns. I mean, they're like just on and on and on. Like there are concerns that are sort of policy and regulatory in nature. Um, and so, so we don't see, we're not seeing a ton, but we're seeing some. And, and I think what's interesting is that, and what we do in this, in this paper is highlight some of those things that are, that are emerging now. So we can kind of get a glimpse of, of what these, of what these practices might be. Um, and, and I think that, that, you know, the, one of the big takeaways is that there is this balance between policy and regulation and safety and then experimentation and exploration, right. And so how do we do that right. Um, and, um, and how can we create public facing things that don't put people at risk? And there are some bad examples. I mean, there are good examples of bad behavior, right? Like there's, there's, you know, there's examples, there's examples of the, the use of, of, of LLMs that, um, you know, that have, that have gotten cities and, and people into trouble, you know, like, like there's sort of missteps in New York and, and, and other places where LLMs have just spat out, um, you know, inaccurate hallucinations and, uh, and, you know, and, and who's liable for that. So, so I think that the, so the paper, I think is a, is a good primer, uh, for, uh, for just sort of looking at that, looking at that landscape. And I guess the, you know, the question that, that I'm really motivated to ask, and I know you too, you are too, Nigel is, is, um, you know, how can we create the conditions to safely experiment, um, or to explore, um, and, uh, what are the things that need to be in place so that we can actually identify, um, public value and not just public sector value in these tools, right?

[43:57](https://youtu.be/tqdR2E_tml8?t=2637) So, um, the, the public sector value I think is, is being relatively well rehearsed in the form, in the forms that I've just described, um, you know, in, in synopsis of, of complex documents and in memo writing and things like that, and efficiency, but the public value is different. And, and what are the ways that we can actually source what that is? Like, we don't know how valuable this will be for, um, people who live in cities, right? Like we don't, we don't know. Um, we don't know if, um, and I should say that not only do we not know what the value is going to be, but, uh, but there's, there's still lots of fear and unknown around the technology so that in the rollout or the implementation of these tools is going to require, um, a lot of kind of, of, of, of explanation and a lot of safeguarding, um, you know, in order to even explore. So, so this is, this is where I think is fascinating about.

[45:00](https://youtu.be/tqdR2E_tml8?t=2700) And I'm going to jump ahead and talk about one of the things that has come out of this is that we started this collective that we call Rethink AI. And this is a collective of people across the country, and it's based now in the New America Foundation. But what we're doing with Rethink AI is that we're actually trying to figure out ways of doing these safe explorations that we're creating. We want to create sandboxes within cities, and we want to work both with institutions and communities so that we can figure out how public value can actually get sourced and identified. And so it's inspired a lot of us to figure out how to create use cases, because we're in a moment with this technology where the use cases are small and somewhat boring, and and it doesn't have to be there. The technology is far enough along for us to actually identify what it can be good for so that it's not completely driven.

[46:05](https://youtu.be/tqdR2E_tml8?t=2765) I already mentioned sort of public sector value, but of course, there's also private sector market value, right? That they're like all of these forces are being sort of pushed on what sort of public value might be. And we have to take this opportunity right now to be able to stop for a moment, create a space for exploration so that we can sort of get some signal from the noise, right? And so that's where we are with Rethink AI. And that's, to me, is what is an exciting sort of next step from this publication that we just produced.

**Nigel Jacob**

[46:42](https://youtu.be/tqdR2E_tml8?t=2802) That's cool. Yeah, I mean, I'm excited the Rethink work. In so many ways, it's sort of the evolution or the next step of the work that we've done together. But I think of the field more generally is looking at, I think, I will say, I used to be a computer scientist. I'm excited about general AI. I'm also skeptical. There's such hype. The hype is, I've never heard, this is like, what is a more extreme version of hype? Super hype. I don't know. But I don't know how we'll ever be able to deliver against that. But I do think we have to try because a lot has been said about how it will fix climate change and all these things and how it will do this and that. And that all sounds like BS to me, but I think we should try to do that. And so I think both in Rethink, I think we're, you know, I'm excited to sort of dig in and try to kind of pull out some really meaty, difficult challenges that we're not even sure are possible, but also to hopefully encourage and support and inspire other people to do similar things wherever they are.

[48:02](https://youtu.be/tqdR2E_tml8?t=2882) And I think in a way, you know, thinking about what did we learn about from the civic tech period or that period of civic tech? I think we have to be casting a broader net. So it can't be government as the only supporter of this work. I think there's, they can be part of that, but I think it needs to be more of a network. And so I think, you know, finding ways for community organizations to work, to be working in collaboration with government and with CDCs and local philanthropy to help pay for it. I think I'm excited for that work to happen and to sort of look in a more kind of a sophisticated way about like, what kind of questions can you ask? And if something works, like how does it live on? I think that was a question as we've sort of already mentioned about, you know, that came up for civic tech is, you know, how do you, what do you do after these things don't work?

[49:07](https://youtu.be/tqdR2E_tml8?t=2947) Or what do you do if they do work? You know, thinking about those things.

**Eric Gordon**

[49:12](https://youtu.be/tqdR2E_tml8?t=2952) Yeah. And I also want to call back to something that we talked about. Just one thing, five minutes. I mean, you remember, you asked me to remind you. No, this is great. I mean, we'll shut up in a second, but I want to just want to, maybe this is a nice kind of cap and then we can open it up. But I want to kind of loop back to, you know, to something that we talked about earlier and something we learned with Hub 2 15 years ago. And Nigel, you talked about it, which is that we created this digital experience, but we did it in physical space. And the thing about AI right now is that we don't have to think about it as replacing human interaction, right? We should, we should always, in fact, we get into trouble when we think about AI as a decision-making tool. Humans have to be the decision-making tool, right? Not tool, but humans have to be the ones making decisions. And so we have to actually design for where the humans exist within the loop, right?

[50:13](https://youtu.be/tqdR2E_tml8?t=3013) So where are, so we should not be thinking about AI right now as a way of just doing the work for us or in the public engagement context of just assembling opinions and then spitting out decisions. Like that's terrifying. So what we can do is we can create, we can create a context for, you know, for these tools to provide humans the insight that we need in order to make better decisions. And we need to design, as we're designing these tools, we need to design where humans exist within the loop at all times, right? Because that's going to be our ethical check. That's going to be the way that we make sure that we're accountable, not only to each other, but also to the, to the outputs of the machine. You know, all of that has to be sort of factored in to the, to the way that we imagine moving forward. And again, these are, these are lessons that come up over and over and over again.

[51:15](https://youtu.be/tqdR2E_tml8?t=3075) You know, that no matter how long we do this, every time we try to use technology to replace humans, we get into trouble and we sort of lead down the path towards dystopia and we don't need to do that. You know, technology can actually humanize us and our institutions. It doesn't have to further dehumanize them.

**Nigel Jacob**

[51:37](https://youtu.be/tqdR2E_tml8?t=3097) Yeah. And so with that, why don't we, why don't we open it up to, I know there's, I see activity in the chat, but I haven't had the attention span to look at it. So if you don't mind, I'm going to start with the previous comments and then going on. Some of them are really long, so it's better if the person can ask the question. So I'm just reading, Daniel said, civic washing, civic washing, wow. Natalie, never heard of this term before, but they explain a lot. Civic washing is perfect. Then I came here because of that talk, the previous talk. So here's the link on the chat to the previous talk for all those who want to access that. Michael, link to a book. This is for Eric. So then we can post that. Thank you.

**Eric Gordon**

[52:22](https://youtu.be/tqdR2E_tml8?t=3142) Oh, Michael, I got to finish it first, but as soon as I'm done with it, I'll link to it.

**Nigel Jacob**

[52:26](https://youtu.be/tqdR2E_tml8?t=3146) Julia says, StoryCorps is an example of listening with a shared sense of values around sharing stories about the human experience. It's a Library of Congress project. I'm not going to comment on that. Daniel says, same with Deb Roy's real talk on democracy next. Okay. And then there's a link to StoryCorps. A question for Eric from Irvin. I don't know, Irvin, if you want to make the question directly, if you are here. If not, I can read it for you. Feel free to unmute.

**Audience**

[53:06](https://youtu.be/tqdR2E_tml8?t=3186) Hi, Eric. So my puzzlement here is, is this listening, generative listening, conceptually throwing out a large fishing net to capture as much comment as you can about things? And, you know, if you've attended any kind of government meeting and there's public input, all sorts of people have all sorts of issues. And those people are reflecting things that may not be representative of any other group, other than a little group of people talking. So how do you, how do you connect that with the information that might be of use to the people in the agency that's looking for information?

**Eric Gordon**

[53:55](https://youtu.be/tqdR2E_tml8?t=3235) Yeah, I mean, it's a great question. And I think the reality is, is that those, often those people who are the loudest in the room are the ones who get the most attention right now, right? Even without the technology mediator. And, you know, I've been in many, many government meetings, you know, sort of planning, planning meetings where we're trying to figure out how to, how to not pay as much attention to the loudest people in the room. Right. And so, so we have that problem already. We have that problem already of just figuring out what, how to weight different kinds of different kinds of inputs. So yeah, that doesn't get rid of that problem entirely. But what it potentially does is that it does, as you say, kind of, it casts a wider net. It opens up, it opens up sort of opportunities to collect more unstructured data. And then it allows us, hopefully, to be able to make sense of that data in a way that is, that, that can better represent sort of majoritarian positions or other positions that are, that exist outside of, you know, that are, that are outside of the values of, of, of what is being said.

[55:08](https://youtu.be/tqdR2E_tml8?t=3308) So we have this, we have an opportunity through the tech to, to be able to make better sense of what it is that we're, that we're taking in. And let me give you one more example here is in the, in the listening literature, and yes, there is a listening literature. There is this distinction between closed system and open system listening. And closed system listening is this, is this idea of like, you know, surveys are closed system listening. So, you know, in a survey, we ask a question, we get an answer to that question. And often we're, we're, we're quite capable of responding to the answer to that question because we've created very narrow parameters for input. Same, same is true with a lot of public meetings, right? We have narrow parameters for input. We're able to take that in and make some sense of, sense of it within that, those narrow parameters. Open system listening is a little bit different. So open system listening actually is a kind, a kind of listening that doesn't actually articulate these narrow parameters or even ask, lead with questions, but instead has an opportunity to take in information that is more unstructured, take in both commentary and activities in, in different forms, and then attempt to make sense of it.

[56:22](https://youtu.be/tqdR2E_tml8?t=3382) With, you know, with LLMs, we are able to make some sense of unstructured and, you know, diverse data sets. And so here's where I think there's opportunity is that we can actually do that kind of open system listening. And then we're able to do some analysis that can create insights of that unstructured data in a way that was not possible before. So it doesn't totally solve the problem that you're pointing out, but I guess my, my, my first reaction is that problem already exists. And, and actually I do think that this addresses that problem. It allows us to be a little bit more, a little bit more deliberate and open to different kinds of input. And it doesn't, it doesn't preclude possibilities, you know, through our kind of closed system listening. So this is, this is why I'm optimistic. And I think that listening is a better framework because ultimately public engagement is a form of closed system listening for the most part.

[57:25](https://youtu.be/tqdR2E_tml8?t=3445) And there are opportunities now for us to be thinking more expansively.

* * *

Original page: https://innovate-us.org/public-engagement-and-institutional-listening-through-generative-ai · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org
