---
id: "workshop:equitable-public-engagement-with-technology-february-3"
type: "InnovateUS workshop"
title: "Equitable Public Engagement with Technology"
url: "https://innovate-us.org/equitable-public-engagement-with-technology-february-3"
date: "2022-02-03"
language: "English"
instructors: ["Alexis Bonnell","Beth Simone Noveck"]
recording: "https://www.youtube.com/watch?v=1XtvpCB8YxE"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# Equitable Public Engagement with Technology

InnovateUS workshop · 3 February 2022

Join us to hear from Alexis Bonnell, former head of innovation at USAID, to understand why and when collective intelligence can be most helpful, learn about innovative engagement tools, and understand key considerations for collaborating in effective and equitable ways.

## Learning goals

- Understand why and when collective intelligence can be most helpful
- Learn about innovative engagement tools
- Understand key considerations for collaborating in effective and equitable ways

## Speakers

- **Alexis Bonnell**, Former Head of Innovation, USAID
- **Beth Simone Noveck**, Founder of InnovateUS and Director, Burnes Center for Change and the GovLab

Recording: https://www.youtube.com/watch?v=1XtvpCB8YxE

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/equitable-public-engagement-with-technology-february-3.txt (plain text) and https://innovate-us.org/equitable-public-engagement-with-technology-february-3.json (every segment with its start and end second).

**Beth Simone Noveck**

[0:01](https://youtu.be/1XtvpCB8YxE?t=1) I'm going to go ahead and start on time. Graham or Kate, are there any, any announcements before we go ahead and get started.

**Unknown speaker**

[0:11](https://youtu.be/1XtvpCB8YxE?t=11) I think you're good to go. We had an announcement about innovate us in the chat that I'm going to repost in case people join since I did it but I think you're good to roll Beth.

**Beth Simone Noveck**

[0:20](https://youtu.be/1XtvpCB8YxE?t=20) Wonderful and we're happy to see our colleagues from across state government, and I am, as I said, the chief innovation officer for the state of New Jersey but I see colleagues here from California and Colorado and Maine and other places. We are part of our newly launched innovate us initiative to bring 21st century skills training to public servants around the country, and to connect people around the country to one another to share the tribulations travails and joys of being in state government across the United States so we hope that you can connect with one another, whether on this program or on future ones and we're just delighted to have you here. We have a colleague and Paulino from the Office of Innovation, and he will be in helping me out with this presentation today as we talk about equitable forms of public engagement. So let me start with a quick anecdote from the state of New Jersey, just to illustrate why I think equitable engagement is so important.

[1:24](https://youtu.be/1XtvpCB8YxE?t=84) So from here it won't come as any surprise to you that the coven 19 information hub coven 19.nj.gov which Graham and and and others spend a lot of their time on has been an invaluable resource that serves the people of New Jersey we to the tune of a million people a week. We have only 8.8 or so million people in the state, but the site has served Graham, can you tell me how many up until now.

**Unknown speaker**

[1:53](https://youtu.be/1XtvpCB8YxE?t=113) I think we're at well over 10 million users have visited it's hard to know if there's some duplicate counts in there though.

**Beth Simone Noveck**

[2:00](https://youtu.be/1XtvpCB8YxE?t=120) Oh no no much more Graham that's 160 million over the course of the pandemic I'm not just talking about recently. There are a lot of people who use this site and it is really popular I think because the site was designed with the benefit of engagement. That is to say engagement of a variety of various sources. So on the one hand taking the engagement that we got in the form of calls from people to our coven 19 hotline the 211 phone number that helps to drive the choices of what information gets put front and center on the site so by using the knowledge we're able to reorganize the information in the way that makes most sense and is most useful to people. The site lets you also type in your questions and so that data those analytics that come from people about what people are asking about further lets us customize what we do. But there's more to the engagement than just that kind of passive engagement with the public. We also test all of the things that we do by actively asking and speaking with residents to ensure that the tools that we design are actually intelligible and useful to people. We've engaged with the universities across the state in the past to create an editorial team of students who have volunteered to help out at what I would call translating from government speak into plain English, so that what goes up here is actually easy to read and easy to understand in a format that people can use.

[3:37](https://youtu.be/1XtvpCB8YxE?t=217) And there's so much more to it than that. But essentially it's just one example that I wanted to give of the importance of engaging with people in an effort to deliver more effective services and better policies that are more responsive to people's needs. So what we're going to do here today is to, in a very short hour we're going to do a lightning fast exposition on the value of public engagement, going to then talk about how to design effective engagements, and hopefully do so with an eye towards focusing So how do we engage not only with the people we anyway would speak to, but in particular with vulnerable populations who otherwise don't generally have a voice, and are generally not the people that we reach. So there's a number of examples that relate to new technology and websites and things like that. None of what I'm talking about is limited to technology. So even if your job has nothing to do with websites or technology, everything we're talking about applies to offline as much as online.

[4:44](https://youtu.be/1XtvpCB8YxE?t=284) So with that said, if we have time at the end I'm going to try to introduce a couple of examples of new platforms for engagement that we can take advantage of. And that might help you to do your work. Last set thought here which is that we want to make this a conversation so even though we don't have a lot of time, we very much want to hear from you about your experience engaging with people. So with that said, let me just ask Graham if you would push out a quick question to people, which is that we're interested in hearing from you about the ways in which you're already engaging with people. Graham, do you have access to that poll, if you can push that out. I did a minute ago did someone else log on as innovate us it's telling me I'm logged on another device and. Perfect like I have the power or why but I'm going to push that out so a quick question to you for those who are on the phone and can't see this. We're asking people if you currently use resident engagement in your own work again offline online.

[5:48](https://youtu.be/1XtvpCB8YxE?t=348) And if you said yes, if you are using new technology to facilitate that engagement. Just a quick poll of people here to understand how we're actually what background you have and actually doing engagement. From the looks of the poll, it looks like a lot of people are already doing engagement. I'm going to push out the results in just a moment but I'm getting a lot of yeses. So we definitely want to hear from you as we focus on this question of how we can do a better job not only engaging people, but engaging people in a more equitable and inclusive way. Okay, I'm going to end the poll here for a moment. I think we've got most people participating. So you should be able to see that let me see if you can see those. So you'll see again 68% of you said you're already using engagement 32% said you're not. And then we got a 50% of people saying you're using new technology to engage with people 50% said you're not.

[6:51](https://youtu.be/1XtvpCB8YxE?t=411) And as always, we have a healthy number of people who I hope after this may be able to expand what you're doing, or for those of you who aren't doing engagement might be able to do more than you're doing today so please feel free as we go along to ask any questions that you have. Our goal here is to impart experience and skills and examples from that are going on not only in our own jurisdiction but around the world that we can bring to bear for our own work. Just a quick note on process if you want to interject with a comment or a question, you can just go like this and wave your hands, you can wear a small enough group that I think you can probably unmute, but for those who would like a more disciplined way of doing it there's a button at the bottom of the screen. Let's see I don't see reactions under. Whoops. Trying to see where's the raise hand button.

**Unknown speaker**

[7:54](https://youtu.be/1XtvpCB8YxE?t=474) I can't see myself so I can't tell. So Anna has raised her hand so Anna tell us where's the button for you. It's not on my screen.

**Audience**

[8:03](https://youtu.be/1XtvpCB8YxE?t=483) So it's in reactions, and then it has the raise hand and then I also has the remove raise hand.

**Beth Simone Noveck**

[8:10](https://youtu.be/1XtvpCB8YxE?t=490) Great. I think since I'm the presenter I don't have a reaction button anymore I do seem to have some nifty new buttons like annotate, so I can doodle. before. So I'm excited about that but I will erase that. So that's a very helpful so again reactions and raise hand or type in the chat or wildly gesture, and we'll go from there. So, right from the get go, let me say that citizen engagement crowdsourcing open innovation, public engagement. Any of these terms are all fine. I also like the term collective intelligence, or the wisdom of communities. We're not going to spend a lot of time today on nomenclature or on the differences. The nuance differences between these different terms. I'm going to use them pretty interchangeably and generally just refer to public engagement. So let me state that right off the bat.

[9:15](https://youtu.be/1XtvpCB8YxE?t=555) So I might say public engagement, I might say collective intelligence. In every case I'm just talking about how do we engage and talk to people understood very broadly. To help us get more informed about understanding a problem or developing a solution to a problem. The second thing I want to just get right out of the way at the outset is to say that engagement is not always the right thing to do. Whereas philosophically morally participatory democracy and engaging with communities is the right thing in principle. In practice, it is not always the right thing to do. That is to say in this chart which I know is hard to read so we're going to send this to you. If you are not in a position where you can actually use what people tell you, then you should not waste their time of asking them. It will only create more frustration more distrust with government and more challenges to not be prepared to ask people to use what people tell you, whoops. That is now I have managed to, can you still see the slides or have they gone away for you.

[10:33](https://youtu.be/1XtvpCB8YxE?t=633) You're back on my slides page. Let me do that one more time again. But I'll keep going here and slideshow. Here we go. Sorry about that. And one more time. Okay. I'll just go forward again. Let me give you a quick example of what I mean. So, there is a citizen engagement website, put out by the city of Madrid that's existed since about 2015 had a lot of publicity when it was launched it was an early kind of progenitor of citizen engagement especially online engagement done by government. And when they first started this site called the CD Madrid Madrid decides they got about 400,000 people signing up in the beginning. Long story short, it does a lot of things the site. It has places where you can have discussions and it has places where government puts up consultations and people can respond and comment.

[11:36](https://youtu.be/1XtvpCB8YxE?t=696) It also has a part of the site called proposals or proposed us. And on that part of the site. Citizens can propose a policy or a piece of legislation that they want the city council to consider. Since 2015. How many citizen engagement proposals made by citizens would you imagine have become law in Madrid. I see you looking at me, can you how many things you can tell by my question how many, how many citizen proposals would you imagine have become law. Oh, you're on mute. I should turn my camera off so I don't get called on now I'm just teasing. I might say, maybe zero I don't know, you have, you are correct. This was an I wouldn't call I believe me I'm a softball a softy when it comes to question asking. It's a disaster this part of the site because it doesn't work.

[12:38](https://youtu.be/1XtvpCB8YxE?t=758) Well, I won't go into the details but basically in order to get a proposal considered you have to get 1% of the population of the city signing on to your proposal. The people who then have to register in person to have their address validated before they can support your proposal. It's so complicated that there is no point in doing this, and I think it just is incredibly frustrating to people and in fact what we've seen is since 2015 although a lot of people signed up. Nobody's using this part of the site anymore because it just doesn't work. So all that to say, not worth doing it if you can't make use of what people are telling you if it's too complicated. If you can't and here's where I come to a list that if you're on the phone maybe we'll try to read some of these as we go along but if you can't answer these nine questions for yourself. And we want to go over these questions and see if we can provide a little guidance about how to do things like articulating the goal of the project.

[13:43](https://youtu.be/1XtvpCB8YxE?t=823) So I'm going to try to dive in and breeze through these. I'm going to talk about each of them, both from the perspective, generally, of doing engagement. And then from the perspective of equity and being more inclusive and I really want to get your input on what you're doing or what you would advise us based on your own experience. So question one that you have to be able to answer is can you clearly articulate to people what it is that you're actually trying to accomplish. Can you define a problem that you're hoping that people can help you to solve. If you can't do that clearly for people, then there's really no point in asking. So a quick example here. Last year during COVID Department of Education in New Jersey with our help, did an online consultation and asked parents and students and teachers. What are the greatest challenges that you are facing with regard to your child's education during COVID.

[14:44](https://youtu.be/1XtvpCB8YxE?t=884) And because there was a clearly articulated question and there was a clear statement that we're going to revise our policies, based on what you're telling us. I think that made a real difference to why we were able to hear from 17,000 people in a two week period. is because it was a clear statement of why we were asking and why it mattered. So let me ask you guys your advice when it comes to this question again of articulating the project's goals. What do you think is important in terms of doing this in a way that will also reach vulnerable communities? And Ed, this is where I'm going to turn to you to share some of your thoughts and then we'll see if anybody else has anything they want to chime in in the chat or out loud.

**Unknown speaker**

[15:26](https://youtu.be/1XtvpCB8YxE?t=926) Sure, thanks Beth. So one overarching theme you're going to see throughout the presentation is this idea of equity being a lens through which you see all the different component parts of what you're doing. So it can't be an afterthought and that's why when it comes to project goals that's something that you want to do you know at the outset. It can't be something that you're like oh shoot I forgot you know I have to do this or other thing. It has to be something you kind of bake in from that project planning and project goal setting process. I'll pause there Beth. I don't know if we want to go to the next slide with the group's equity insights. Right so we're going to before we go on to the next question let's see if there's anything else that the

**Beth Simone Noveck**

[16:13](https://youtu.be/1XtvpCB8YxE?t=973) folks want to add or anything coming up in the chat otherwise I'm going to keep on going. Okay we'd love to again hear from you about your own experiences as we go along with regard to what you're learning but we don't have a monopoly here on good ideas. So second question I think we need to get answers to is the question of whether you can identify who should participate. So we talk about public engagement the tendency is to assume that we need the public to participate. But it might we might have a need for a more specific audience than that. Let me say a word about that and then come back to this concept of selection and self-selection in a moment that's on the slide. There's a wonderful project out of the UK called Good Sam. We have a related project in the United States called PulsePoint which maybe Graham or Kate could put in the chat because it's an awesome project. And what PulsePoint and Good Sam do is assist the public in identifying who should participate. And what Good Sam do is essentially allow people who have first responder expertise. Basically you've been trained in first aid or CPR by the Red Cross. And what it does is it sends you a message if somebody near you is having a heart attack or who's suffered an accident the data from 911 calls to emergency to geolocate the person having the accident with the person nearby who might have a cell phone. And basically sends a message to you and says can you rush the aid of that person and help to perform CPR. Data shows that when every minute sooner that a person suffering cardiac arrest gets bystander CPR whether it's CPR or use of a defibrillator it increases their chances of living by 50%. So all that to say if I'm doing a project like Good Sam or PulsePoint I'm not trying to engage with everybody. I'm not looking

[18:18](https://youtu.be/1XtvpCB8YxE?t=1098) for the public. I'm looking for people specifically who have CPR training and the ability to use a defibrillator. So I have a specific audience that I'm trying to get to. So we want to think about ways in which we can target whom we're trying to speak to for our engagement. It is the case that we may not want to talk even to members of the public at all. We may actually want to talk to other colleagues across our own agency, colleagues across different agencies, colleagues like we have here on the call across different states, maybe it's people in universities, maybe it's people in industry. We can have different audiences. It doesn't have to be the public just generically and we should be able to answer for ourselves what is the public we're trying to reach. So Ed let me pass it to you to say a little bit more about the idea of combining selection and self-selection here in and also put it to everyone else to see if anyone else has

**Unknown speaker**

[19:22](https://youtu.be/1XtvpCB8YxE?t=1162) advice about how to identify participants. Great. Yeah so that point on whether to use selection or self-selection is so important. So the self-selection if you're doing something where you have you kind of recruit an audience that's super enthusiastic right they raise their hands up they're interested in engaging whether it's because of their life experience or some kind of intrinsic passion that they have. There are a lot of projects where that's a huge resource to tap into and it serves a project's goal as well. Similarly you also want to look at selection. So there might be people who are not part of the conversation whose voice is not being represented and so this question of kind of periodically asking you whose voice is in the room is really important and whether that's through barriers, whether that's through you know other issues of they're not being able to participate. Being able to ask that question can be very illuminating and will inform what kinds of things you can do to make sure you're a little bit more representative.

[20:35](https://youtu.be/1XtvpCB8YxE?t=1235) You're on mute Beth.

**Beth Simone Noveck**

[20:36](https://youtu.be/1XtvpCB8YxE?t=1236) There we go. Sorry I was having trouble getting the controls to come up. There we go. Let me try to go back a second here. I'm sorry about the slides. I'm just trying to activate the controls and instead I'm advancing the slides here. So quick example of what Ed is talking about and we'd love to again hear what you're working on in the region of Brussels, Belgium. So they have a local government and a regional government and then a national government in the region. They have started as part of their parliament to create what they call a citizens assembly. That is to say they have a representative sample of citizens who serve on every parliamentary committee. So to achieve greater equity, they need to have a representative sample of the population there. Of course, what representation means in Belgium looks a little different than it does here. But they use representative participation, rather than just a representation of the population. So they have a representative sample of the population.

[21:43](https://youtu.be/1XtvpCB8YxE?t=1303) So to achieve greater equity, they need to have a representative sample of the population. Of course, what representation means in Belgium looks a little different than it does here. But they use representative participation, rather than just what we often see, which is self-selection where people just volunteer to tell us what they think. Anybody else here have good examples of or experience with either representative or self-selected participation? Ben, I think you had your hand up previously.

**Audience**

[22:17](https://youtu.be/1XtvpCB8YxE?t=1337) Yeah. Beth and Graham, I was going to share the experience that here in the DEP for the state park service. Last year we had engaged the public, and it was kind of precipitated out of COVID, engaged the public on a Liberty State Park restoration plan. Where because of COVID, we couldn't have a typical public meeting. It wasn't feasible to do that. So we were leveraging to go to meeting to allow, and I was part of the team running the meeting, like hundreds of people participating. Where probably even if we had a public meeting, we wouldn't even have that ability. And we had reached out at Liberty State Park to try to get feedback from a lot of the local people and organizations there to have that voice in what was going on in terms of the restoration of the wetlands and impacted areas of that park.

**Beth Simone Noveck**

[23:27](https://youtu.be/1XtvpCB8YxE?t=1407) Ben, did you select who participated or did people self-select and volunteer to participate?

**Audience**

[23:36](https://youtu.be/1XtvpCB8YxE?t=1416) I don't know. I guess I got to think about that. I know we use our website and social media to get the message out that this public hearing was going to be out there and public meeting was going to be out there. Was there some targeted selection? Management might have tried to get at certain groups to try to get them to participate. But yeah, so probably a combination of both, to be honest.

**Beth Simone Noveck**

[24:11](https://youtu.be/1XtvpCB8YxE?t=1451) Very interesting. Let me move us forward as a way to advance this part of the conversation and would love to bring in other folks to hear about your experience here. Even when we can identify who should participate, it doesn't always mean we can reach them. So this is building on Ben's point of just you have to get things up on the website, you have to get things out in newsletters, you have to get them out on flyers. I was talking the other day to folks from a foundation called the David Prize. I don't know if any of you have heard of it because it's a very local prize. And I asked them, how are you so successful at getting so many people to apply for this prize? And what they said was they're a foundation, they have lots and lots of money. Their most successful way of reaching people is by putting up flyers in the laundromat. So they're really keen on reaching people who are not necessarily going to be reading government websites or reading foundation websites.

[25:13](https://youtu.be/1XtvpCB8YxE?t=1513) So it's not always a technological thing, but there needs to be a process for actually reaching people. One quick thought here, and then I'm going to pull in more people here, is to create what you might call a media kit or what you might just call some content to make it easier for other people to amplify your own, your engagement. So when we did an engagement in the state of New Jersey to ask workers about the impact of technology on the future of work for them. Those policymakers were thinking about lifelong learning, we're thinking about how to fund lifelong learning, what kinds of policies we need to put in place to protect people. We wanted to know what were the issues that really mattered to people in the labor space. And so we reached out to about 40 different, I should say, not we, Ed here did all the heavy lifting of reaching out to about 40 different interest groups and created for them a media kit.

[26:15](https://youtu.be/1XtvpCB8YxE?t=1575) Here's a sample newsletter post, here's a sample tweet, here's a sample of a paragraph or a page. And we did it in English and we did it in Spanish to make it easy for people to do the hard work of reaching out for us. So let me sort of see who else has some good experience here. Ed, I'll start with you and then we'll see who else we can bring in, in terms of reaching out to people.

**Unknown speaker**

[26:40](https://youtu.be/1XtvpCB8YxE?t=1600) Sure. One thing that always comes to mind is this idea there on that first bullet of meeting people where they are. And I know that's a concept a lot of us may be familiar with. I like to think about it two ways. One, figuratively, where you're inhabiting, you're being empathetic with your potential audience. You're thinking about how to frame things in a way that's relevant to them. You can also think about that literally, like the way that Beth mentioned with flyers in the laundromat, we've seen the efficacy of getting mobile testing sites and vaccination sites, making them mobile, getting them out into the community. This idea of going to people and in doing so, bridging that gap that might otherwise be there. And so the media kit is a huge way to kind of help your stakeholders and be amplifiers. They're probably very busy with providing a lot of important services to the community. And so any step you can take to make participating and engaging in your program will go a long way.

**Beth Simone Noveck**

[27:40](https://youtu.be/1XtvpCB8YxE?t=1660) Anything from friends in Maine or California that you want to share? Try to bring in some more states here. John, you put something, John from Maine, you put something in the chat. Can I embarrass you and call on you?

**Audience**

[27:55](https://youtu.be/1XtvpCB8YxE?t=1675) Yes, sure. We have an ad hoc construction board formed by law and it has labor management union and non-union reps. And basically it's to help us figure out the technical issues with respect to construction wages, setting construction wages for state projects.

**Beth Simone Noveck**

[28:15](https://youtu.be/1XtvpCB8YxE?t=1695) So you have a kind of statutory framework that requires you to engage stakeholders in the process. And that can be very helpful for ensuring that diverse voices are brought to bear in a process, which is useful. The challenges with a lot of these things, just the participation often tends to be either we don't even measure who participates. Or when we do, we learn that it tends to be middle-aged male participants, usually of a certain income level. And so anything that we can do to try to diversify that participation is crucial. Here in Rio del Sul, which is the picture you're looking at here, a state in Brazil, to foster a public engagement, they actually drove around with a van with computers to help people log on to the citizen engagement project that they had in order to help people to participate more easily. Okay, I'm going to keep us moving forward here, and then maybe we can also bring in some more voices here.

[29:21](https://youtu.be/1XtvpCB8YxE?t=1761) You have to determine for an engagement, who is the right owner of the engagement. Now, it can be the case that it's very important for government to be the owner. As was the case, here is an example, a picture that we're showing from a project in Mexico called the City Challenge Project, which was done in the city of San Pedro in Mexico, a community of about 150,000 outside of Monterrey, Mexico. And it was really important for the city to be the owner, because the city was able to say, not only are we going to listen to you, but we're going to implement what you tell us. But it's not always the case because people often don't trust government or don't have a good relationship with government for the government to actually be the owner. So maybe, Ed, I can go to you first, and then others might have experiences of where you've collaborated with someone outside of government to run the engagement for you.

**Unknown speaker**

[30:22](https://youtu.be/1XtvpCB8YxE?t=1822) Right. To your point, Beth, there's thinking about who that right owner is, right? Who is the right messenger or the right program manager for the particular process is really important. Sometimes having a kind of in-between person or some more of a neutral organization could lend itself to soliciting more candid responses and candid engagement from the audience. But sometimes you do kind of want that legitimacy that either a government partner might lend, or someone that the public has a lot of confidence that's actually going to end up running with what the public is recommending.

**Beth Simone Noveck**

[31:05](https://youtu.be/1XtvpCB8YxE?t=1865) Great. Let's see if we have any additional experiences that people can share in the chat. And I'm going to keep moving forward with, oops, sorry, another slide here. I forgot. But I think you've made all these good points. And I'm going to go forward to our next kind of question, five out of nine on our list. Next question, I think it's important to be able to answer in order to do an effective engagement, again, whether it's online or offline, whether it's the public or whether it is with university professors or businesses or other public servants, is the ability to address the incentives for participation. Now, in the academic literature, academics tend to talk about the incentives as being either extrinsic or intrinsic. Extrinsic are those incentives that are what we would simply think of as financial incentives. So we can pay people to participate. But in fact, most of the research shows that intrinsic incentives, that is to say, things like status rewards, the idea of community building or being part of something, there are lots of things we can do that are non-financial that are often the better incentive for people to participate. And in fact, the most valuable incentive for people to participate is the feeling that their time is being well spent. So if people feel their engagement will be worthwhile, if they feel their engagement is relevant, they tend to participate.

[32:40](https://youtu.be/1XtvpCB8YxE?t=1960) And in fact, that's exactly what we've seen in Madrid, to go back to the earlier example, the part of the site that I mentioned where people are making proposals for legislation, participation has dramatically dropped off because it's not relevant. Yet, the part of the site where government is putting up a policy and then soliciting comments on a policy and explaining how they're going to use people's feedback, that part of the site is actually seeing more participation is growing over time. Another quick example, back to the United States, do we have anybody from Colorado here? Kate, we got any Colorado folks on the line?

**Unknown speaker**

[33:21](https://youtu.be/1XtvpCB8YxE?t=2001) I don't know off the top of my head. I know we, I'm not sure. I know we have California, Maine, New Jersey, Colorado. Put it in the chat. We'd love, if you know where Lakewood, Colorado is, we'd love to hear from you. If you're here from Lakewood, Colorado, we'd love to know. Sorry?

**Audience**

[33:42](https://youtu.be/1XtvpCB8YxE?t=2022) Anna's on the line. I'm not from Lakewood, Colorado though. I'm from Northwest Colorado in Steamboat Springs. Anna, do you know anything about the Sustainable Neighborhoods Program in Lakewood, Colorado? I do not, but I can imagine several communities, especially in the front range, have similar programs. I know in our community, we've got a Sustainability Council. That's all public. It's whoever wants to join and certain key players, but nothing

**Beth Simone Noveck**

[34:19](https://youtu.be/1XtvpCB8YxE?t=2059) formal from the city. So in, it started in Lakewood and it's spread now to at least five, four or five cities in Colorado to the best of my knowledge, but maybe in a different part of the state. They are creating the incentive for residents to get involved in sustainability efforts by giving people what they call credits, points basically. It's sort of like frequent flyer points. So it's a non-monetary reward, but it's essentially like a status, like earning points in a game that creates an incentive above and beyond just, you know, fighting climate change and doing good works in your own community. They have 20,000 people participating in 500 different citizen-led sustainability projects. So all that to say you can create incentives for people that go beyond the monetary. I see there are two hands up and I can't see people. So I'm going to

**Unknown speaker**

[35:15](https://youtu.be/1XtvpCB8YxE?t=2115) ask Graham or Kate to help moderate here and tell me who's raised their hand. Yeah. I think Anna had one up and put it down, but Ben, it looks like you have something you wanted to share.

**Audience**

[35:29](https://youtu.be/1XtvpCB8YxE?t=2129) Ben, if you're sharing, we can't hear you. You're on mute. I got to do that on mute thing. I always get caught by that. No, I wanted to share, you know, because they had come to mind, especially to talk about incentives, excuse me, was that the State Park Service, again, ended up doing as part of the COVID issues was to have an incentive of getting vaccinated, VaxPass. And that was a effort where during, it was like the end of May to the beginning of July, if you had to show proof of getting vaccinated, you got a pass, a free pass to the state parks for the rest of the year. So we, you know, actually how that incentivized people to, it was kind of open to people who had already gotten vaccinated, but the hope was that that was going to incentivize people to get vaccinated so that they can then get free access to the parks. And it was pretty successful. There was about 137,000 showings of people saying, Hey, I got my vaccination to the visiting the state parks. A lot of it was like Island Beach State Park, which is a very popular park in New Jersey, right at the beach, of course. So yeah, that was an example of an incentive that, you know, was kind of used as part of that participation.

**Beth Simone Noveck**

[36:58](https://youtu.be/1XtvpCB8YxE?t=2218) Thank you so much. Great example, Ben. And actually, I think around the same time, the New Jersey also ran a beer and a shot incentive. And I know that got some...

**Audience**

[37:09](https://youtu.be/1XtvpCB8YxE?t=2229) And meet the governor as well, right?

**Beth Simone Noveck**

[37:12](https://youtu.be/1XtvpCB8YxE?t=2232) And meet the governor. A lot of efforts to get people vaccinated, which is...

**Unknown speaker**

[37:16](https://youtu.be/1XtvpCB8YxE?t=2236) One of the takeaways there is to not be afraid to get a little creative, right? And to think about your intended audience, right? Think about what are some things that are going to get them excited and involved with kind of what you're doing. And again, it's kind of one of those things that it's useful to think about on the front end, right? So not just like, Oh, at the end, we'll, you know, give someone a gift card or something like that. But really think about how the process itself can be a really fulfilling experience for the participants.

**Beth Simone Noveck**

[37:46](https://youtu.be/1XtvpCB8YxE?t=2266) Okay. I'm going to keep us going in the interest of time. But again, please, we'll share all the notes with you after the fact. It may seem pretty obvious, but it's really important to be able to clearly define for people exactly what you need and want them to do. And hopefully tell them how long it's going to take for them to participate. You also want them to understand how what you're asking them to do connects back to what the goal is of the project. So quick example here of a project that I'm signed up for, and some of you may be too. There is a nationwide program to get all of us, as the program is called, signed up to contribute public health data back to a national research project. And what I like about All of Us is it has really clear instructions about how you can participate. And it uses icons, simple text, very clear language, and exact time estimates. Every time it tells me to do something like answer a health question or tell them my blood type, it tells me exactly how long it'll take me. Now, this survey it mentions on the slide is a very long survey to do the enrollment and the consent forms. It's very time consuming. But the subsequent ways of participating are actually very, very brief. But the clear delineation of tasks, I think, is incredibly important. It can often be very vague for people what it is you're asking them to do, and more importantly, why you're asking them to do it. So Ed, from an equity perspective, what would you say? How can we think about this concept of right sizing the commitment?

**Unknown speaker**

[39:25](https://youtu.be/1XtvpCB8YxE?t=2365) Sure. Building on those points on clarity and being concise, there's a lot of value where, to the extent possible, thinking about different ways to capture a lot of that data, different channels, whether that's through in-person engagement, whether that's thinking about leveraging SMS messaging, online engagement, email, phone, all these different ways that you can potentially reach out to people and have them engage with you. And there might be certain types of activities that take a little bit more time. It's something that takes a little more commitment. And there might be other things you can do where it's more of a one-off, getting together with someone, trying to capture that snapshot and making sure that's something that's not an intensive time commitment. And again, always thinking about accessibility and what

**Beth Simone Noveck**

[40:15](https://youtu.be/1XtvpCB8YxE?t=2415) are things that can be done by non-English speakers. Thank you, Ed. And let's see. I'm going to keep us moving forward and then we'll see if at the tail end we can get some other examples from people. So related to this tasks thing, I think is also being able to have a clear process or what tech people would call the workflow. Tech people or business consultants would call the workflow. So a clear definition, working backwards from the end to the beginning of how the engagement connects to the outcomes of the project. And so you want to be able to have a clear steps that are not just the consultation or the engagement, but what's the timeline for feedback? How will that connect then to what you're actually asking people to do? And I think it's a really useful to people to see the whole process to understand how their feedback will be relevant. Another quick example here that I love, this is a project developed actually initially out of MIT called Risk Map in English. It was first pioneered in Indonesia under the name Petabanchana, but it has since found its way to Florida and back into the United States. It is a project designed to use new technology to get residents engaged in finding, measuring, and pointing out instances of what we might call fair weather flooding. So basically, streets that are waterlogged, which in many places is not just an inconvenience, it is dangerous to the point of deadly for people when flash floods flood a neighborhood or flood a street and create really dangerous conditions. What I love about this is that the tools are really, really easy to use. It sends you a text message and allows you essentially to point out on your phone how high the water is. It doesn't ask you to say the water is six inches or eight inches, since most people have no idea how big six inches is. What it asks you is it

[42:21](https://youtu.be/1XtvpCB8YxE?t=2541) shows you a picture of a person and it says, tell me how high it is. Is it knee high? Is it waist high? Is it ankle high? And it shows you the entire process then about how your data is going to be used, how the report will be entered, exactly what the tasks are, and it essentially lays out the process for using that data. So mainly just wanted to give you that example, but to say that having a clear sense of the process from end to beginning and beginning to end is incredibly useful for ensuring that people feel a sense of trust in the process. What can we do to improve equity? Ed, let me

**Unknown speaker**

[43:01](https://youtu.be/1XtvpCB8YxE?t=2581) go to you first. Yeah, I think you captured a lot of it even when you're giving that great example of making something very clear and accessible. I know something that we always rely on with our design standards is thinking about how we can use visuals to communicate an idea, right? Something that's accessible to everyone, and also thinking about ways to make sure you manage expectations, right? So kind of let people know what they're doing, why they're doing it, and what we're going to do

**Beth Simone Noveck**

[43:26](https://youtu.be/1XtvpCB8YxE?t=2606) with that information. Okay, last two points here. Number eight, being sure you have a way to judge and evaluate what people tell you. So not every public engagement is going to be a competition. So sometimes we do things that are what we might call crowdsourcing, where we ask people to give us answers to something, and we're picking one winner in a process. But if you go back to Ben's example where you had a public meeting, you know, it's not a competition. You want to hear everybody's input. But even where it's not a competition and you don't have to pick one winner, you still have to have some way to tell people what kind of contributions you're looking for, which sort of things you'll consider, what you, what the essentially the judging criteria are. And the better that you can articulate those criteria, the more useful is the input that you're going to get from people. So I'm going to, I've forgotten the Syria tracker example, so Ed, I'm going to go over to you to talk about the equity insight here. And I'll just wrap up on this point by saying that where you do need some concept of formal judging, you can actually have people from the community participate in that evaluation process. So right now I have colleagues who are working on a community engagement in Oakland, California, and they are asking residents to come up with ideas for how to solve.

[45:00](https://youtu.be/1XtvpCB8YxE?t=2700) Problems of Urban Blight in Oakland. And they are asking not just government leaders, but also community leaders, civil society leaders, to be part of evaluating the proposals that citizens make to ensure that those proposals are actually useful and can be implemented at the back end.

**Unknown speaker**

[45:19](https://youtu.be/1XtvpCB8YxE?t=2719) Ed, over to you. Great. Yeah, I think you touched on that second point there of the making the evaluation process, whether that's a committee or some kind of a panel representative, is always a strength and a testament to the process. On the other side of that, on gathering inputs, you know, we talked about making it simple. Sometimes you can't boil something down to, you know, just icons or iconography or something super simple. But there's an opportunity there to work with the community and actually kind of build in a capacity building process. So having folks in the community link up with folks who are maybe kind of subject matter experts, right, and work with them to build their capacity to think about these concepts in ways that make sense to them. And now after going through that process, they're going to be experts too, right, after kind of going through that. So that's always another

**Beth Simone Noveck**

[46:11](https://youtu.be/1XtvpCB8YxE?t=2771) way to handle the kind of the inputs of the evaluation process. Great. Okay, last point here. Can you actually use what people tell you? And again, to reiterate what I said at the beginning, if you can't answer this question, you probably shouldn't start. It is very, very common to engage in some kind of public consultation, to have a town meeting, to have a public hearing, because it's required of you to do so. But there isn't really a commitment to listen to what people actually have to tell you. The reason something like PulsePoint or GoodSAM are so effective as engagements is because there's immediate gratification and awareness of how your engagement is useful. You go and give CPR, use a defibrillator and save somebody's life, you are very aware of how your participation is useful. And in fact, they share back to participants on the network. I get it because I'm signed up, I get a text message telling me, the system was activated and somebody helped somebody. I get that notification no matter where it happens in the country, and I love it. And then on the website, you can go and see how many lives were saved and how many people were helped. And just incredibly, incredibly gratifying.

[47:35](https://youtu.be/1XtvpCB8YxE?t=2855) It is very rare that we actually do that in government. So last thought here on equity, Ed, beyond the fact that just telling people how we're going to use their input, anything more we

**Unknown speaker**

[47:50](https://youtu.be/1XtvpCB8YxE?t=2870) can do here to really promote equity? There's one question that's always helpful when thinking about actually using the input. And that's, will someone who participated in this process want to do it again? Are they going to be a repeat customer? And not that you necessarily have to repeat every engagement that you do, but you want the user experience, the resident experience after that process to be a positive one, right? And because this is more about developing a relationship, right? About building trust. That's not something that happens on a one-off. It's something that has happened over a series of engagements and a series of different conversations

**Beth Simone Noveck**

[48:30](https://youtu.be/1XtvpCB8YxE?t=2910) and that you have with the community. So I'll leave it there. Thank you. So I'm going to wrap up here and just see if there are any, with one last point that I alluded to before, and I won't belabor right now, but just to say again, there are different kinds of engagement. Obviously, some things do look like a competition, like what we would think of as crowdsourcing, like asking people to guess the number of jelly beans in a jar. And there is a right answer. More often than not, though, we're looking for lots of good ideas from people. If you've been to the federalchallenge.gov site, it's been around for the last decade, where a thousand federal agencies, more than a thousand federally, I'm sorry, several hundred agencies have put up more than a thousand requests of the public to solve problems. Sometimes those are about getting a single answer. Sometimes they're about coming up with a couple of good answers, but more often than not, you're going to see a lot of things that look much more like what I would call co-creation or co-design. You're getting people to come up with, and I won't go into the details here, multiple good solutions and to work on them together, as is the case again in Lakewood, Colorado. They don't want one sustainability project. They want 500 sustainability projects.

[49:54](https://youtu.be/1XtvpCB8YxE?t=2994) So there's more to be said at another time about how to think about designing those different types the point is that there are different for all of these types, whether it's a competition or it's a collaboration or a co-design, you want to be able to answer these nine questions. So we have about 10 minutes left, and we can use those 10 minutes for, let me end the slide sharing here for a moment, we can use those for looking at some platforms, or we can use them for questions, reactions, reflections, or other comments for people. So my preference is to hear from you. So let's see if there's any comments or questions, suggestions, reflections first.

**Audience**

[50:39](https://youtu.be/1XtvpCB8YxE?t=3039) I will ask something. Hi, my name is Matt. I'm from the AG's office. So I haven't been here that long, but I've been here long enough to sort of watch a rollout for a basically like taking the public's pulse on use of automated facial recognition by police departments. So we have a poll, if you go to NGOAD.gov right now, it's like the top headline on it. And I, you know, we're frankly not doing almost any of the things that you have suggested we do. I mean, it's basically just a survey, we threw it out there. I don't think there was any, you know, targeted effort to do outreach. I think a lot of the questions that we're asking are a little bit on the abstruse side and possibly not very accessible. I don't think like we released in Spanish, all that stuff. I will, so maybe we are like way behind everybody else in government. I'm going to make the assumption that we're maybe somewhere in the middle. So as a, you know, an agency that's like really, you know, quite behind on this stuff, what would you recommend as a sort

**Beth Simone Noveck**

[51:49](https://youtu.be/1XtvpCB8YxE?t=3109) of incremental step to do better? So it's a great question. I don't think you're behind. First of all, you are probably typical. You're probably even ahead to some extent in the fact that you're asking in the first place. We do most things in government without asking anybody, or at least we used to until recently, I think until recently, we thought it was okay. And especially if you were a lawyer, and I say that as somebody who's trained as a lawyer, before you get offended, I'm making fun of myself here, is we were educated with the belief that we should do things behind closed doors. It's what in government we call deliberative work product, right? We don't share it until it's ready for sharing. So the notion, you're not a lawyer. Oh, well, I, okay, good for you, that I'm sorry, you're the you're the non-lawyer among lawyers, which was making very popular in the AG's office then. I think the things that we've mentioned here, and we're happy to both share the deck and also take this offline are all things that are concrete enough that there are very, you know, they're very easy things to do. So to the extent to which you don't have to now launch a new platform, or suddenly start a citizen assembly the way that they've done in Belgium or in lots of other places, there are things that can be done, for example, just in terms of reaching out to interest groups to then get those interest groups to help amplify the message to other communities who might not otherwise participate, number one. Number two, to get people who are not lawyers to review the site and make sure it is in plain English and not in legalese or government speak, that maybe it is simplified and gives people easy ways to participate. We are happy, the tool, for example, that we use to hear from 17,000 workers in two weeks, we used a free open source tool that costs no money to stand up. It was developed

[53:53](https://youtu.be/1XtvpCB8YxE?t=3233) by a professor at Princeton University. So Home Corn Advantage, but it is a free and open source tool. And there are others like it where, you know, it makes it easy to put up something online and, you know, get people's feedback in 30 seconds. In other words, a lot of the way we typically do consultation in government, if you think about something like the federal governments regulations.gov site, where you as a member of the public have had the right since 1946 to comment on federal regulations, not online, but for 20 years, you've had the ability to do it online and before that on paper. That's very daunting, because not only do you have to be able to make sense of a really long regulation, but you then have to type some long comment. Whereas there are other ways of engaging with people where you could do a quick pulse check or sort of which of these things is more urgent and important for you, where you're just picking, you know, this one or this one, or you're responding with a text message. So there's some pretty simple things to be done. And we will point you maybe Matt or Graham or Kate could put a link to the guide from the collective intelligence site that this is based on, which provides just, you know, a summary of some of these things in like two pages, that can provide some very concrete and simple advice. And we'll also provide our emails in the chat. So sorry, that was a very long answer, but to a very good question.

[55:18](https://youtu.be/1XtvpCB8YxE?t=3318) And we're really here to help people apply these things in practice. So don't hesitate to ask us for more help offline. Let me see if there are any other comments in the chat or questions or interjections. If not, what I'm going to do is I'm going to show you actually, since my next slide is actually showing that free site that I was just mentioning, so maybe it would be helpful to do that. The important thing to recognize when it comes to using tools is there is no such thing as a one size fits all. It's the case with technology. It's also the case with process. So when a friend of mine uses the expression that I like different horses for different courses that you might have heard. If I'm trying to find out from people what they think is the greatest problem with facial recognition, it's a different process than if I'm asking them to solve the problem of facial recognition. And I'm going to want a different tool for the process. So this tool, for example, is the free tool I was mentioning. You ask people a question like, what is your greatest concern about facial recognition technology? You give them, you essentially load the system with 50 answer choices, and then it presents people simply with two options. And then they can vote between these two options as many times or as few times as they want. It means the participation takes 30 seconds. You can keep going and spend more time, but there's a way to get involved that, you know, you can be done in just 30 seconds. And we're happy to give you access to these tools.

[56:56](https://youtu.be/1XtvpCB8YxE?t=3416) Again, this is really good for defining the problem with people. Here's the example of the thing we, the application of the same tool that we used in education. If what you want to do, though, is get solutions from people. Here's another example of another free tool. This one developed out of Iceland, but hosted in the United States and has passed all the security clearances from DHS and blah, blah, blah, Homeland Security. We've been through all these hoops. Nonprofit, nonpartisan, free and open source tool. It's been used in at least 20,000 different resident engagements. Right now they're using it in Vienna to get kids participating in policy there. They're using it in Scotland to get help with their policymaking there. It's, we've used it in New Jersey with public servants to get people to supply good ideas for solving hard problems. And last example here with one minute to go is another free and open source tool that was developed by MIT. Here's an example from use in Brazil and it's designed something different. It's designed to help people provide feedback on what's going on on the ground. Like how is a policy working in practice? So happy to provide links to this offline. The point of this was just to say that there are free tools that are available to make it easy for you guys to apply these things in practice. If what you're looking for is to do online engagement, online has its advantages in that as we heard before, it can sometimes make it easier for people to participate who would never show up for a public meeting in a church basement under normal conditions. So let me thank you for your participation. We will send links and slides, et cetera, and the recording to this.

[58:46](https://youtu.be/1XtvpCB8YxE?t=3526) And you can, as well as our contact info, so that if we can provide any follow-up help, we are happy to do that, including to help you or your agency go deeper on these things. We're happy to do training that takes more than an hour. We're happy to spend two hours with you and your colleagues to figure out how to apply these things in practice. And as Kate is putting in the chat, we have more of these one hour mini classes coming up. So we welcome you to sign up. Thank you, everybody.

**Audience**

[59:18](https://youtu.be/1XtvpCB8YxE?t=3558) Thank you, guys. Thank you. Thanks, all.

**Beth Simone Noveck**

[59:22](https://youtu.be/1XtvpCB8YxE?t=3562) Great to have you. Thank you very much. Take care. Bye. And if you would remember to end the recording.

* * *

Original page: https://innovate-us.org/equitable-public-engagement-with-technology-february-3 · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org
