---
id: "workshop:an-ethics-ecosystem-for-ai-and-big-data-why-what-how"
type: "InnovateUS workshop"
title: "An Ethics Ecosystem for AI and Big Data: Why? What? How?"
url: "https://innovate-us.org/an-ethics-ecosystem-for-ai-and-big-data-why-what-how"
date: "2023-10-26"
duration: "90 minutes"
language: "English"
instructors: ["John Basl"]
recording: "https://www.youtube.com/watch?v=VvuIN_65vrk"
transcript: "automatic, from the recording"
source: "InnovateUS (innovate-us.org)"
licence: "CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org"
---

# An Ethics Ecosystem for AI and Big Data: Why? What? How?

InnovateUS workshop · 26 October 2023 · 90 minutes

Join us for this 90-minute virtual training with John Basl, Associate Director of the Northeastern University Ethics Institute, where he leads AI and Data Ethics Initiatives. In this workshop, Basl will discuss the challenges in developing ethical artificial intelligence (AI) based systems in the current absence of a robust ethics ecosystem that helps distribute ethical reasoning, decision-making, and responsibility. Basl will highlight some of the key questions surrounding responsible use of AI, including fairness and transparency, and will discuss what we can do individually and collectively to navigate an increasingly challenging ethical landscape.

## Learning goals

- Learn about the challenges in developing ethical AI based systems in the absence of a robust ethical ecosystem 
- Understand some key questions surrounding the  responsible use of AI
- Learn what we can do individually and collectively to navigate an increasingly challenging ethical landscape

## Speakers

- **John Basl**, Associate Director, Northeastern University Ethics Institute

Recording: https://www.youtube.com/watch?v=VvuIN_65vrk

## Transcript

Automatic transcript of the recording: names and terms can be misheard, and the times can be a few seconds off. Each time links to that moment of the video. The transcript alone: https://innovate-us.org/an-ethics-ecosystem-for-ai-and-big-data-why-what-how.txt (plain text) and https://innovate-us.org/an-ethics-ecosystem-for-ai-and-big-data-why-what-how.json (every segment with its start and end second).

**John Basl**

[0:00](https://youtu.be/VvuIN_65vrk?t=0) Here, I look forward, look forward to talking, to hearing your questions and providing feedback. A few things to get out of the way. I'm a moral philosopher, I work so far from the public sector, I give this talk to lots of audiences, never to philosophers. So I'm going to be talking about this from a totally different area, but hopefully the lessons and the things I'm going to say, touch on and are applicable to you. And I'm happy to work with you and talk with you about how that might be so. So I'd like to start by, what I want to do today is motivate the need for what I'm going to call an ethics ecosystem for AI and big data and talk about what an ethics ecosystem partly consists in. And then I'm going to talk through an example of how to try to contribute to building it or per modeling such an ecosystem. I'm going to do this by talking across a bunch of different examples, none of them are particularly public sector examples, one of them very closely touches on stuff at the public at the intersection of the public sector. But really, it's to motivate why we keep making mistakes in this area, and what a real solution is going to have to look like and what it means for individual practitioners that are interacting with AI, that they have to be on the lookout for as they try to navigate the ethical challenges raised by these technologies, until we get to something like an ecosystem. I'm going to warn you're going to do something that makes me as a philosopher very uncomfortable.

[1:26](https://youtu.be/VvuIN_65vrk?t=86) And that's that I'm going to say a bunch of things that I will not defend and excruciating painful detail, it goes against my very nature. But so I'm going to give some arguments, but not make them in much detail. So if you want to follow up anything, please, the most comfortable philosopher is when someone's disagreeing with them in the q&a. Okay, so let me start with this talking about what it is that we want. AI holds incredible promise. And I think people genuinely want to realize that promise ethically. So what we want, I think, is to realize the promise of AI, while mitigating against ethical pitfalls and wrongdoing. But what we have is not that we have an unstructured mess of concepts. If you work in the space at all, and try to like do some research on you'll find things like responsible AI, trustworthy AI, fair AI, you'll see many commitments to broad principles like fairness, transparency, privacy, but few concrete ways to realize those in any part of practice. And then of course, we have lots of examples of unethicality, I could have easily have replaced this slide instead of this like teetering mess of concepts represented by the image representing that I could have easily have put from the past six weeks, just articles where companies using AI have have sort of making a moral made a moral misstep. Now, why is that? Why is it that despite the fact that we most actors, I think are good and genuinely want to realize the promise? Why do we keep tripping over ourselves?

[2:53](https://youtu.be/VvuIN_65vrk?t=173) Why do we keep making these mistakes? And I think the answer to that, as I'm going to try to motivate is that we lack what I'm going to call an ethics ecosystem. And we need to aim to cultivate that rather than trying to address issues piecemeal on a case by case basis or with a single set of principles that sort of return the cranks on and get the right answers. Okay. So what I'm going to do is talk through a variety of challenges that we face. And it's not just companies or for profit companies that keep making mistakes, or that's not that they don't care about ethics, I think the problems are systematic, and they're not associated with bad actors. And so to convince us, I want to talk about three kinds of recurring mistakes that I see in the space, or issues that plague the research to application through practitioners pipeline for AI applications. So the first is at various levels of decision making, from individuals and organizations to leadership to governance spaces, we lack concrete operationalized guidance or evaluation tools. And because of that, we have tended to grab on to existing tools from other contexts and misapply them. And the example I'm going to give to showcase this is privacy and informed consent, we'll get into details in a bit. Second, where there has been genuine movement to develop actionable guidance or context sensitive tools and approaches for AI specifically, the recommendations are often very poor. And this is for a couple reasons. One, many of those with the expertise in ethics, policymaking, things like that people from my discipline, they fail to appreciate how ethical frameworks and tools interact with the technologies that they hope to provide guidance for. So they don't know enough about how AI works to realize how they need to adapt their toolkits to solve these problems. And I'm actually going to give two examples in this space, one is going to be autonomous vehicles, just because it's until recently, it was the most widely known example of AI ethics.

[4:45](https://youtu.be/VvuIN_65vrk?t=285) Now, the other example I use is large language models. And we'll talk about that too. And then the third kind of issue or it's related to the second one is that there are many people interested in solving these problems. And they have lots of technical proficiency, they understand the technical details of these systems, but they don't understand ethics or policymaking or regulation or related disciplines. And so they don't really know how to move from value commitments to technological solutions. And so as it happens, you know, they give they have their technical expertise, they turn out technical solutions, but they miss the ethical mark. And the example I'm going to give it that is something you may be familiar with, which is issues of algorithmic fairness and a little bit opacity. So I'll talk briefly about some examples of these to do the work of motivating what I'm claiming is the need for an ethics ecosystem. Okay, so let's start with the the rush to find some off the shelf solutions. So the idea is that we have these problems, but they're not totally new problems, we know that AI raises challenges of privacy and fairness and all these things. But we know that other other contexts have these kinds of problems. So what we do is we rush over and grab a tool that was built for those contexts, and we apply it and it turns out not to be very good.

[5:55](https://youtu.be/VvuIN_65vrk?t=355) This happens because the tools are built for specific contexts to realize foundational values. So in another area, like say, bioethics, we didn't just start by saying, hey, let's use informed consent. And let's anonymize personalized data. What we did is, we got a bunch of people together, and they settled on what are the core foundational values that we should care about? So for example, respect for persons, and then they asked, Well, what does it mean in the context of clinical care, or in the context of human subjects research, or in the context, this is for human subjects, where we have a similar thing for animal subject research? What does it mean to show respect for persons? Well, we settled on the idea that we try to promote and protect their autonomy. And then we asked, Okay, what's a concrete tool we can use to make sure we're doing that. And we came up with a couple informed consent and anonymization. So informed consent, anonymization are not foundational values, they're just tools. But in the context of clinical care and human subjects research, they do a pretty good job of realizing those foundational values via those intermediary norms. Now, if we look at how these tools have been deployed and adapted for big data analytics and other contexts and divorce them from the systems in which they're embedded, in that where they do help realize values, we can start to see a disconnect. So in the context of big data analytics, we see an adaptation of informed consent to what we call notice and consent. So informed consent, if you're not familiar with this, when you want to propose a piece of clinical research or human subjects research, you write out a protocol for what you're going to do how you're going to interact with your subjects, you then you tell the IRB, an oversight board that's actually going to look at your consent procedures, exactly what you're going to tell your participants.

[7:33](https://youtu.be/VvuIN_65vrk?t=453) And then you have duties to ensure that you're informing your participants of what's going to happen to them to disclose certain information to make sure you're avoiding coercion. And so not only are you designing it to make sure you get informed consent, but there's an oversight board that's making sure you're doing a good job of that. If we transition to notice and consent, and what I mean by that is think about anytime you go to a website and you're confronted with a chance to say agree or disagree, or you download a piece of software and a screen of text pops up. And at the bottom, you click I agree, most of you don't read those. In fact, I think there's some interesting research that shows if you try to read all the license agreements, or the notice and consent agreements for all the software on your computer, you would do nothing but read and user license agreements. But if you look at the language, if you actually do take some time to read those, you'll see that they're not oriented towards making sure you're well informed about what the software is doing. Instead, it's a block of legalese, it's really oriented towards covering the company's ass not towards disclosing information, it is disclosed, it's disclosing lots of things, but not in a way that helps you understand. And so there's a certain sense when we took this tool from bioethics, and we applied it in this new context in the hope of showing respect for persons, but it fails because we've totally transformed the tool. Now, even if you fixed notice and consent, even if you tried really hard, which would be hard given the scale at which you're using the software, to like actually provide informedness, it's not clear that the tool would be any, any, any better. And that's for this reason. In the context of AI and big data analytics, my getting consent from some individuals allows me to make inferences about other individuals that have not consented. And so it's unlike clinical care. I mean, this happens sometimes in clinical research when we do genomics research, and it requires special consideration. But for the most part, when you consent to be a part of my research, protocol, you're not making it possible for me to make inferences invasive inferences about other people that haven't consented. But big data analytics is all about getting information from some parties to make him to make inferences about others. We can tell the same kind of story about

[9:36](https://youtu.be/VvuIN_65vrk?t=576) anonymization. In the case of bioethics, not giving access to every not giving access to people's social security number, birthdate, blah, blah, blah, can protect them from invasive treatment, it can make sure that they're not fine, we don't know who they are. And so we can't do certain things to them. But anonymization of those characteristics in the context of big data analytics is basically useless. Facebook meta, these companies do not need to know your birthdate or social security number to learn a lot about you. The data that seems like it's sort of not pertinent in clinical context becomes extremely relevant in the context of big data analytics. So these practical tools outside of their existing ecosystem or existing frameworks actually don't serve the values that we think they do in those other ecosystems. So that's one kind of problem that we that I see come up a lot, we grab whatever tools are there, because rightfully, we're worried about protecting certain values. But because those tools were not crafted to preserve those values in a particular context, we just go about mucking things up. Sometimes it looks like ethics washing, like we seem like we have a thing where we're protecting research participants or protecting individuals, but we're in fact not. Okay. So deploying ethics tools built for other purposes is just one issue that plagues our goal of realizing the promise of AI or the potential of AI. But the attempt to craft new tools for the context have not all have also not been so great. So this is the part of the talk where I crap on philosophers like it's partly our fault. I really genuinely believe philosophers if they if they, if they could overcome certain problems could really contribute a lot to the debates about how to craft good, useful regulation, governance and tools for managing the ethical issues associated with AI. But we keep messing up because philosophers, especially but other people as well, don't really understand these technologies. So we keep coming up and saying, like, here's what you should do. But we don't see how the technologies change what we should do. So I'm going to give two examples of a sort of deep misunderstanding of the technology that results in misguided device. The first one is the case of trolley case and autonomous vehicles. I'm going to avoid a lot of the technical

[11:38](https://youtu.be/VvuIN_65vrk?t=698) details. I've written a lot about this, but I'll just give you sort of a mini version of this. And the mini version is this. I don't know, maybe you've seen popular media articles, or maybe you've seen some of the sort of coverage of this idea of autonomous vehicle ethics. This is a slide, a screenshot of a website called moral machine, moral machine.org, which is a tool that came out of MIT, when people were debating what autonomous vehicles should do when they're in a crash scenario. So a crash scenario is when an autonomous vehicle is going to crash, it's going to have it's going to harm someone, how should we think about how to divide up those harms? So moral machine released this project, half of the moral machine team, I'm not numerically, but some subset of them moral machine team, just released this as a tool to figure out what people's general opinions were about how to solve these issues. They didn't think you should actually use it to program autonomous vehicles. But some subset of the team really thought they should use this data, which has millions and millions of user inputs to figure out how we should program cars to behave in accident scenarios. There was lots of popular media coverage of this. And when people ask me about AI ethics, until large language models blew up, this is what they asked me about, this is what they knew about AI ethics. But it's there's a sort of deep Mr. understanding of the technology that comes with thinking that we're trying, we should be thinking about programming cars for how they should respond in these edge case scenarios. And it's not because we shouldn't think about edge cases, it's important to think about edge cases. The problem is that the way you build these cars is using AI.

[13:08](https://youtu.be/VvuIN_65vrk?t=788) And a promissory note, that means that how you train it to act in any given scenario, like everyday driving scenarios is not unrelated from how you program it to act in these scenarios. So with AI, we don't just say like, hey, when you're in a crash, do X, Y, or Z. But when you're in a normal stop sign, do A, B and C. Instead, because the way these things are trained, training it to be really good at stopping at stop signs might make it worse at doing things in accident scenarios. So these things are not to be treated independently. And the standard way that philosophers and most of the public thinks about programming is that we control individual scenarios. But once you get into the realm of machine learning, that's not what's going on. And once you understand that you have to offer very different advice for how you should program these cars, you have to think much more holistically about how you build the training regimes governing the driving behavior of these vehicles. I'm happy to get into the details of that. It's a fascinating subject for me, I didn't know that it would be for you. So I cut out some slides here to instead talk about this case. Maybe you saw this in the news, but a lawyer went to court with a chat GPT generated brief. And the brief was very well written, in some sense, it looked like a document of legalese, it was turned over to the judge, but it had what are typically now known as hallucinations. A hallucination is what a large language model does is it spits out text, the standard for which is, will it read like decent human text to a human being not is are the things I'm saying true. And so some of the case law cited in the brief was totally hallucinated, it was made up that those citations just didn't exist. Of course, the lawyer didn't know this, because he didn't understand exactly how chat GPT generated the text, he assumed, not unreasonably, they're like, when you do a Google search, you're going to get things that have in some sense been vetted, like top Google search results for specific citations are typically But the judge then like looked at it and want to look at the case law, could not find the relevant case law. Turns out, then they asked the lawyer, the lawyer said, Oh, I don't know, I got this from Chad GPT, and then got fined for it.

[15:12](https://youtu.be/VvuIN_65vrk?t=912) This is a case where our failure to understand how the technologies work just results in us using it in poor ways. But you see the same thing in people's recommendations or worries about the ethical issues associated with things like large language models. I've seen philosophers worried that these things are slowly going to take over our governance or someone's going to use it to hack the Pentagon, and it just betrays a deep misunderstanding of how these technologies work. And so when they're making recommendations about how to manage these technologies, they're completely missing the mark. Now let me talk about the other side of the coin. Just as that there are ethicists and policymakers and people who understand the normative side of things, but they're insufficiently attentive to the technologies. There are lots and lots of technologists and practitioners that are insufficiently attentive to the foundations of ethics and don't understand how ethics and policymaking work. So let me work through a case in fairness and AI, but this lesson also applies just as easily to transparency and privacy and I'm happy to talk through those if people have questions about them.

[16:12](https://youtu.be/VvuIN_65vrk?t=972) So one of the challenges we face in leveraging algorithms and decision making is that they inherit the biases of their training data. And indeed, we see many examples of this. Perhaps the most common example, though I think things are more nuanced than the public discussion has suggested, is COMPAS. COMPAS is an algorithm that was used to predict recidivism. So it was used to assign risk scores to people who were being evaluated for pretrial detention. But it was designed to develop risk scores and the risk score was supposed to correlate to the chance of re-offense of a crime. It was then used to determine whether people should be held for pretrial detention or not. Now the problem is, it will not surprise you to find out, that COMPAS mistakenly labeled black defendants as having a higher risk of re-offense than otherwise what seemed like similar cases for white defendants. Another example where we have a fairness problem is Amazon spent a ton of money, well a ton by my standards, probably nothing by Amazon standards, to develop an algorithmic hiring tool.

[17:16](https://youtu.be/VvuIN_65vrk?t=1036) But they had to shelve it once it was discovered that it was discriminatory against women. And we were actually kind of lucky the way we discovered that it was discriminatory against women because we found that it was noticing things like going to an all-women's college or playing on a women's field hockey team was counting against candidates. Because if it was picking out more subtle things that weren't so obvious, this might have gone into use. So the problem, these are cases of algorithms that are unfair. The problem of algorithmic fairness is especially worrisome, perhaps because A, people read the outcomes of these systems as objective and often defer to them. So a thing says an individual is high risk and they just say, okay, they must be high risk. And B, it can be very hard to tell that they're being unfair because they can rely on proxies for race and gender that aren't always recognized as such. So in the Amazon case, we had very obvious proxies going to a women's college. But the proxies can be less obvious. I mean, they can be semi-obvious, like using zip code, which will correspond with race and other characteristics.

[18:21](https://youtu.be/VvuIN_65vrk?t=1101) But they can be much more subtle than that too. And so they can be very hard to detect. So they're, on the one hand, read as objective. On the other hand, invisibly relying on proxies for obviously discriminatory characteristics. So one response to the problem of algorithmic fairness has been developed what are called fairness metrics. These are tools we can use to, they claim, ensure a model meets some metric of fairness or operationalization of fairness. So for example, we can say what we want a model to do is have the same error rates for independent of race. So we want to make sure that the false positive rate for falsely accusing a white individual of being high risk for reoffense is exactly the same for a black defendant. And so then we can use these metrics to evaluate the algorithm. We get the same kind of thing in the context of transparency. We try to develop tools for what it means to say that an algorithm is transparent. I'm going to set this example aside, but happy to come back to it if people want. So why is this solution inadequate?

[19:22](https://youtu.be/VvuIN_65vrk?t=1162) Why is it bad to focus on things like fairness metrics? So I don't think there's something intrinsically bad with developing a kind of fairness metric or a tool for increasing the interpretability of an opaque model. I don't want to say that those are problematic for their own sake, or they're just always wrong. But these approaches to addressing algorithmic fairness and opacity are often extremely myopic or completely free-floating from the things we already know. Completely free-floating from the things we actually care about from the ethical point of view. They're disconnected from a careful moral evaluation of what we actually owe to people we're making decisions about. There's a tendency, especially in computer science, to see problems of fairness or opacity as technical problems with technical solutions. And for computer scientists, the place in the pathway from data collection to application is where you build and validate the model. So it's unsurprising that there's been a drive to develop these kinds of solutions. Tech companies have been yelled at for being unfair. Tech companies have computer scientists. Computer scientists are going to see where they can intervene, and that's mostly going to be in model building and validation.

[20:24](https://youtu.be/VvuIN_65vrk?t=1224) The other place you see it is in trying to come up with clean data, like make sure the data collection practices are good. But the relevant point is that these aren't purely technical systems. They're socio-technical systems. And the way in which these technologies interact with values is complex. So first, once we take a socio-technical perspective, we should just stop caring about whether models are fair. We care about the fairness of decisions, and we should make sure our model meets some metric. It doesn't matter if our model meets some metric of fairness if the decisions it rolls in are no more fair. So for example, we can imagine that if judges are going to misapply the data that comes out of the compass thing no matter what, if they're going to be biased regardless of what the risk scores are, then what do we care if the thing meets some metric? That just gives us the illusion of fairness when it's not there. Secondly, whether a decision is fair or unfair in the first place is extremely context-sensitive. And it's sensitive to not just the context, but the type of fairness we care about.

[21:27](https://youtu.be/VvuIN_65vrk?t=1287) So let me talk about that in the case of compass. So let's say we've settled on some fairness metric. We've made sure that our model is fair. We make sure the model is fair according to whatever metric anybody prefers. Think about how things change if we think about two different uses of compass. There's the original compass use. It's not actually fair to call it the original use because the company didn't say it should be used for this. But what it was actually used for was to determine whether to hold an individual defendant on pretrial detention before trial. That's what it's actually used for. But imagine instead we use it to assess the risk of defendants, and then we use that as a resource allocation tool. So we say, hey, this class of defendants or this defendant is high risk of reoffense. We need to dedicate more resources to keeping them from reoffending, whether that's educational resources, social services, things like that, that lets them go back into their community and be a functioning adult human and minimizes the chance of reoffense. Those are two radical different uses of the same model. Now, here's the fairness thing to point out. You might think it's unfair to Black defendants if they're at a higher risk of if they're labeled as high risk in the first use case.

[22:33](https://youtu.be/VvuIN_65vrk?t=1353) But you might actually think that disproportionately labeling Black defendants as high risk when it's going to mean more resource allocation to Black communities is exactly what you want. The reason the data is biased is that Black people are over-policed. And so we want to do things that dedicate more resources to Black communities. And so having the bad data, the biased data in there actually lets us make more accurate decisions about where our resource allocation should be to meet the demands of restitutive justice. We owe restitutive justice to certain communities for the fact that we've generated this bias in the data. And that would be lost if we tried to achieve the fairness metric. And so the kind of questioning we're answering and the kind of fairness we care about is going to determine what our fair outcome is and what we should care about. And so just focusing at the point of the model is not going to do the trick. Okay, so that's just a quick and dirty overview of some of the mistakes that I just see over and over again in this space.

[23:33](https://youtu.be/VvuIN_65vrk?t=1413) And then the question is, okay, what should we learn from those mistakes? Now, here are some lessons you might draw from these case studies or examples. One, it's important to understand the technologies we're reasoning about. Technical solutions can't be what I call normatively free-floating. We have to actually think about the ethics and not just build a technical solution because we can. We should keep context in mind. We should think about socio-technical systems instead of simply technical systems. Those are okay lessons to draw. They're all, I think, true. I endorse them, but they're not the lessons I want you to draw. It's unreasonable to expect every individual that interacts with these problems to manage the full suite of ethical issues and nuance. Not every humanities faculty can go get a degree in computer science. Not every computer scientist can learn all of ethics. Not every public policy worker or policy worker or person working at the public sector can learn everything they need to do to advocate for the right kinds of values to solve their specific problem in their specific space. Instead, I think the lesson is that we need what I'm going to call an ethics ecosystem that shapes work in the field in a way that achieves the promise of AI while avoiding the pitfalls.

[24:37](https://youtu.be/VvuIN_65vrk?t=1477) So, what is an ethics ecosystem? It is a coordinated system of components that distribute the workload of managing ethical issues in a domain or a context. These ecosystems or those parts include ethics infrastructure. That's guidelines and training for practitioners to help them to identify and resolve some ethical issues. It gives them practices and procedures. It gives them checklists. It gives them mechanisms and pathways for escalating issues. Ethics infrastructure also includes oversight and auditing, both human and AI-based auditing. Another component of the ecosystem is outreach, education, and enculturation. We need training in school to equip people in the domain with an understanding of the ethics infrastructure and how to navigate it. We need enculturation to get people in domain to see the world through the lens of the ethics ecosystem. And I'll give example of this in a minute. We need outreach to others outside the domain so they can more effectively engage with technologies in the domain. Like the public needs, we need outreach to the public so they know enough about these technologies to be good advocates for themselves.

[25:40](https://youtu.be/VvuIN_65vrk?t=1540) We need what I'm going to call translational research programs. There are just foundational ethical issues that need to be resolved, and that's true for policy issues too. And we need to do research in that space. We don't, there's no off-the-shelf solution. We need to come up with new solutions and then translate the core research out to the ethics ecosystem. To actually put it into practice. We need interdisciplinary practices. It needs to be the default that there's interdisciplinary engagement and broad stakeholder engagement between computer scientists, practitioners, the public, ethicists, social scientists to think about these problems to avoid all the blind spots that we have. We need informed policy and regulations. We need politicians and regulators that understand the detail of the technology and can craft useful regulation that actually provides guidance and put some teeth behind it. And I think at the core of an ethics ecosystem, what other ethics ecosystems really have that facilitates all this is a sort of common language and a set of interdisciplinary practitioners or transdisciplinary practitioners.

[26:43](https://youtu.be/VvuIN_65vrk?t=1603) In other areas, when we talk about ethical concerns, we have sort of standard pieces of language that we use, like informed consent, which captures something very specific. And we can use that to talk across different areas. And everyone sort of knows they're talking about the same thing. Instead of every meeting I go to when someone talks to me about fair AI, we have to spend an hour or two just making sure we mean the same thing by what fairness is. So when we think about where we are with respect to AI, I think it's very instructive to think about how it compares to biomedical ethics. And that biomedical ethics itself has three robust ethics ecosystems, one for clinical care, like patients with patients in hospitals, one for human subject research, where we're doing research, which has different ethical demands, and then animal subject research. Each of these is crushing it compared to AI ethics. Biomedical ethics has longstanding guidelines for practitioners. Doctors and medical researchers receive relatively unified training and rules to follow. They are entrenched in the practice of medicine, those rules.

[27:43](https://youtu.be/VvuIN_65vrk?t=1663) There's concrete oversight infrastructure, like before I mentioned the IRB. That varies, of course, country to country, but there's some form of oversight for both human and animal research. And they work to sort of not only reinforce the norms, but ensure compliance with the norms, help satisfy certain guidelines, review research before it starts, for example. There are accountability frameworks. Those IRBs and oversight committees have real power to monitor and intervene. They have tools for auditing compliance with norms. They have regular reviews to ensure, for example, that the ethics infrastructure is living up to its responsibilities. Bioethics also has a robust research infrastructure. There are conferences to share work and the latest research. There are robust collections of case studies, training materials. Because of all that, participants and biomedical researchers in biomedical ethics speak a common language. They can easily communicate concerns and justifications to one another. Finally, there's a group of people that are trained intermediaries between ethics and medicine and health research. Those are called bioethicists. Bioethicists are not philosophical ethicists like me.

[28:45](https://youtu.be/VvuIN_65vrk?t=1725) They aren't doctors. They aren't lawyers. I mean, maybe they're one of those. Maybe that's where they started. But it's its own specialty, focused on drawing on and deploying the resources of the ecosystem to help solve ethical challenges in that space. If we contrast that with AI ethics, we don't have specific guidelines for practitioners. That's why this seminar exists, I assume. There's no concrete robust oversight infrastructure. There's no well-developed accountability frameworks. There are growing research programs and information exchange, but there are still deep disconnects between key participants in the work. AI ethics specialists and industry partners are not in very close contact with one another. Only the most, I was going to say the wealthiest companies, but it's really the companies that are under most PR scrutiny have dedicated AI ethics teams. Technical practitioners and ethical experts do not speak a common language. Like I said, when I talk about fairness with my colleagues, especially in computer science, there's a lot of calibration that has to happen between us.

[29:47](https://youtu.be/VvuIN_65vrk?t=1787) There's no equivalent to the biomedical ethicist, at least not as a profession. There are some individuals that serve that sort of role. I have a lot of colleagues that we hire in to try to do that work, but not in a way that provides a nearly enough capacity to handle all the work that needs to be done. Now, I don't want to overstate things. The systemic capacity in medical ethics and research ethics isn't perfect, and I'm sure there's an ethics ecosystem that is part of your profession in some way. I mean, I'm sure it's not perfect and you also have complaints. But I have two things to say. So in the not-so-distant past, there were just significant failures to live up to the standard of ethics in medicine and medical research, so it's not like it's been perfect forever. And there are many ways that I think bioethics can improve. But I do want to say that the best criticisms of those ecosystems come from the ecosystems. People that grew up in them, understand them, are trained to evaluate them, provide good reasons to move them forward. So we have lots of criticisms of the current lack of ethics ecosystem, but don't have the sort of infrastructure to help move us further along.

[30:48](https://youtu.be/VvuIN_65vrk?t=1848) I want to close with, just checking my time, looks like we're doing okay. I want to close by going through an example where I've done some work to try to build an ethics ecosystem or contribute to an ethics ecosystem for a very, like, let's call it a mini-ecosystem because it's a very, very small problem in the space, a relatively small problem in the space. So I'm part of a project called the National Internet Observatory. What the National Internet Observatory is, is an infrastructure project where what we're trying to do is the following. Currently, if you want to research how platforms influence internet users' behavior or how internet users engage with the internet, you have to rely on data from the platform directly. You have to either create a scraper or an API. Maybe you've seen some of this in the news about companies shutting down their API. Researchers are freaking out about this, for example, because Twitter was one of the ways we tracked information, misinformation, was by using the API for Twitter, which let us track what people were tweeting. And that access became very expensive. So researchers no longer have access to that. Furthermore, the data is often structured in databases that are organized towards helping companies make profit, not do academic research on how misinformation flows. What Neo is, is a project that tries to collect data from users, clean that data, and then give researchers, academic researchers, access to that data so they can more easily see how internet platforms interact with users, how information flows and things like that.

[32:15](https://youtu.be/VvuIN_65vrk?t=1935) So we created a set of tools. People consent to our tools. They install them. It lets us track a lot of what they're doing on the web and create an anonymized database. Now, I'm the co-lead of the ethics component of Neo. And we were tasked with building up an ethics toolkit that we could manage the issues that are not covered by traditional IRB oversight. So an IRB is going to oversee any time we try to pull in participants to give us their data. But once we collect the data, any research that people from the outside come do on that data does not count as human subject research and is not subject to regulation and oversight. But we think that can have broad social effects. So we've been working to build a system that helps us manage protecting a bunch of values for our core stakeholders, for a lot of stakeholders. So here's just a map of sort of the timeline of the project and when stakeholders become relevant. So at the beginning, there's the people we want to recruit, and then us, the research team. And then eventually, we're going to recruit some direct data participants, the people who are going to give us their data. They're going to take surveys, they're going to use our mobile app, they're going to use our browser.

[33:17](https://youtu.be/VvuIN_65vrk?t=1997) But there are also indirect data subjects, people who might use the computer of someone that's agreed to come into our study. And they're contributing data maybe unknowingly. Once we've collected all the data and we're going to start making it available to researchers, we get new stakeholders. There's the researchers themselves that want to come use our data, but they're going to be able to make inferences about people who are not in our study. They're going to use our study data to make inferences more broadly. And those can have broad social impacts both for those inference subjects, but also for citizens, policymakers, who knows what the research is going to generate in terms of like, what kind of policy recommendations come out, it could have broad social impacts on citizens, depending on what research get published. It also impacts the research communities of Facebook's decides, for example, that they don't like that Neo exists, I don't I'm supposed to call it Facebook anymore. If meta decides that they don't like that Neo exists, or we're causing trouble for them, they might make it harder for researchers like us and other researchers to access the data. So we have to think about a broad range of stakeholders and how to manage the ethical issues that arise in this context. So we sort of just have gone through the process that IRBs did, we started with what are our core ends. And we settled on a small set, we want to minimize privacy risks, we want to enable scientifically and socially responsible use of this data, we want to enable and ensure compliance, we want to make sure that when our researchers come and they're saying they're doing something, and we and we approve it, we say, yes, you can do that, that they're actually doing it, that they're living up to the norms that we set, which are, which are more than just the norms set by their research institution, we want to enable transparency, and we want to protect and promote participant autonomy.

[34:48](https://youtu.be/VvuIN_65vrk?t=2088) Now, once we had that we sort of started like, okay, what's the sole suite of tools that we need to develop? And how do we get to that suite of tools to achieve those ends. And we have a sort of tripartite approach. One of the core elements of our approach is a value integrated design or embedded ethics approach. Our whole team meets regularly, like the ethicists are in the room when the programmers are deciding what they're going to program things to do. And we're asking them to translate it into language we can understand. And they come to our ethics meetings, I feel so bad for them. And they listen to us talk about ethical issues. And then they ask us how they how do we translate what we're saying into something they can actually put in the code. We try as much as possible to draw on existing analogs, models and disciplinary insights, there is really good research out there on ethics of AI and data ethics that hasn't yet been incorporated in ethics ecosystem, but there is stuff to draw on. There's also examples outside of data ethics where you can get useful insights as long as you're careful to craft them to your own ecosystem. I'll talk about one example of that. And we also have a really broad ethics advisory board, because we know things are likely to fall through the cracks in our team. So just quickly, ethics advisory board that consists of bioethicists, data ethicists, media studies, people, lawyers, one philosopher, plus me, covering all different areas, in the hopes that we just recognize these are interdisciplinary problems. And we need we can't just have bioethicists in the room or data ethics in the room, we need people from every discipline in order to hear things that we might not otherwise catch through our own disciplinary ears. We also there's lots of just like I mentioned, there's lots of analogs, models, insights. One thing we've really drawn on is, interestingly, in the in this bioethics space, genomics research raises a lot of the same kinds of problems as big data analytics research, because knowing someone's genome lets you infer things about people that are related to them, even though they didn't consent. And so there are there's this big project called all of us, which is about collecting the genome of many people in the US, so you can facilitate precision meta medicine research. And they've had to try to work

[36:50](https://youtu.be/VvuIN_65vrk?t=2210) through these things from the ground up to because they recognize that standard protocols are not protective enough. So we've been trying to draw on those two. I'm gonna have to talk about any of those details. But we use all this to develop a toolkit, there's a publication, you can find it on it's open source on nature.com. Where we sort of said, like, hey, here's our core ends. This is only there's actually two pages of this diagram that lists out here all the individual tools that you need to try to achieve these core ends across we have what we call the Swiss cheese model, which is we don't think there's any one intervention that's going to solve a problem, things are always going to fall through the cracks, and just hope there's another layer below, or you try to make it so there's another layer below that catches that problem. That's all just to say, I don't expect all of you to build such an ecosystem. But there's this, there's one way to think about the problem is like, give me this set of tools or the set of guidelines that helped me solve this problem. And the other is thinking ecosystemically about how do you build the whole system of components necessary to actually think about these problems? How do we enable people at the front line to know when to call for help and who to call for help and what a solution looks like? So I'll stop there and open up for questions. Thank you.

**Unknown speaker**

[37:59](https://youtu.be/VvuIN_65vrk?t=2279) Thank you so much, john. Let me just stop the recording as

* * *

Original page: https://innovate-us.org/an-ethics-ecosystem-for-ai-and-big-data-why-what-how · Licence: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/), as stated on innovate-us.org