Responsible AI for Public Sector Legal Professionals
Designing AI Policy, Projects, and Societal Impact - Resources
Find all the resources mentioned in the online course by selecting a module from the dropdown menu below. Have questions? Contact us
Ready for a deeper dive? Take a course to
Module 2: Overcoming risks: Data governance
Discusses best practices for data segregation and the use of “walled gardens.” Discusses the Criminal Justice Information Security (CJIS) standard and how to ensure compliance. Distinguishes between tasks that require paid AI tools versus those that can be handled by free tools and how to evaluate AI vendors.
Learn more
-
InnovateUS Workshop: An Ethics Ecosystem for AI and Big Data: Why? What? How? By John Basl
-
Workshop: ACM FAccT Conference Tutorial 2: Using the NIST AI Risk Management Framework (note: audio issue during section covering Govern function)
-
Interactive explainers of core AI ethics concepts
-
Fairness - Measuring Fairness (pair.withgoogle.com)
-
Hidden Bias - Hidden Bias (pair.withgoogle.com)
-
Diversity - Measuring Diversity (pair.withgoogle.com)
-
AI Ethics toolkits with government focus
-
Government specific AI project team toolkits https://ethicstoolkit.ai
-
Governance maturity assessment toolkits
-
Workshop: Managing AI Risks: Reflections on NIST AI Risk Management Framework and Beyond - Jeanna Matthews
-
Article: Evolving AI Risk Management: A Maturity Model based on the NIST AI Risk Management Framework
-
State Data maturity framework - Beeck Center - https://bit.ly/beeck-data
-
Governance committee formation toolkit -Ethics Institute at Northeastern University - https://bit.ly/ethic-committee
-
Organizations
-
National Institute of Standards and Technology (NIST) - https://www.nist.gov/about-nist
Citations
-
NIST AI RMF https://bit.ly/nist-ai-rmf
-
NIST AI RMF Playbook https://bit.ly/ai-rmf-playbook
-
NIST GenAI Profile https://bit.ly/nist-genai
-
Techbetter - Evaluating AIGovernance Insights from Public Disclosures - https://bit.ly/techbetter-study
-
About NIST - https://www.nist.gov/about-nist
-
Reuters - Insight - Amazon scraps secret AI recruiting tool that showed bias against women - https://reut.rs/4aKpnNu
-
NPR - An eating disorders chatbot offered dieting advice, raising fears about AI in health - https://n.pr/4bJcMvo
-
NYTimes - Microsoft Created a Twitter Bot to Learn From Users. It Quickly Became a Racist Jerk - https://nyti.ms/3yBABXl
-
Techcrunch - Blackbox welfare fraud detection system breaches human rights, Dutch court rules - https://tcrn.ch/3KmZGbn
-
Guardian - Royal Free breached UK data law in 1.6m patient deal with Google's DeepMind - https://bit.ly/guardian-deepmind
-
Science - Dissecting racial bias in an algorithm used to manage the health of populations - https://bit.ly/science-health-algo
-
Nextgov - NIST delivers draft AI guidance, generative AI pilot program - https://bit.ly/nextgov-nist-genai
-
Nextgov - NIST’s new AI safety institute to focus on synthetic content, international outreach - https://bit.ly/nextgov-aisi
Module 3: Designing your own projects: From concept to creation
Guides learners through the process of identifying problems within their own work that AI tools can help to solve. Explains how to take these solutions from idea to implementation, covering the process of developing and implementing AI-enabled tools. Profiles several existing AI tools built by public sector legal offices.
Learn more
-
Problem Definition
-
InnovateUS Course: Innovation Skills Accelerator - Module 2 Introduction to Problem Definition
-
InnovateUS Course: Human Centered Design - Module 3 Define
-
Surfacing and selecting projects - Common problem type or ‘typology’
-
Blog series from DataSF on how they run their DataScienceSF program (Part 1, Part 2, Part 3, Part 4) [Archive]
-
Blog by Nolaytics on their problem typology (blog) [Archive]
- InnovateUS Workshop: Delivering Data Scient Projects in Government by Joy Bonaguro (video link)
-
Surfacing and selecting projects - Bottom up experimentation and crowd sourcing
-
InnovateUS Course: Innovation Skills Accelerator - Module 6 Collective Intelligence: co-creation, collaboration, and crowdsourcing
-
Surfacing and selecting projects - Fast field scanning
-
InnovateUS Course: What Works: Fast Field Scanning
-
Lists of government use cases
-
HHS AI use cases [Archive]
-
DOL AI use cases [Archive]
-
Commerce AI use cases [Archive]
-
Pilots
-
InnovateUS Workshop: Effective Measurement and Pilots by Carin Clary & Kate Lawyer
Module 4: Societal challenges
Explores the ethical implications of employing GenAI at a societal level and dissects potential challenges like copyright infringement, deepfakes, astroturfing, issues related to private platforms, and the challenge of developing effective and proactive AI regulation. Additionally, this module delves into ongoing efforts aimed at mitigating these risks.
Learn more
-
Data quality characteristics
-
Blog: Data camp data quality dimension cheat sheet w/ examples
-
Blog: IBM data quality explainer
-
Toolkit: Data culture maturity assessment guide by Beeck center
-
Guidebook: DataSF Guides: How to Ensure Quality Data (gitbook.io)
-
InnovateUS Workshop - Empowering Data Quality: A Metrics-Based Approach for Agency Excellence
Citations
-
Detroit Free Press - Data glitch was apparent factor in false fraud charges against jobless claimants https://bit.ly/mi-ui-data
-
DM-BOK - https://bit.ly/dmbok-site
-
DCAM - https://bit.ly/dcam-site
-
DataSF - https://bit.ly/datasf-dq
Module 5: Writing your own AI governance and use policy
Guides legal professionals on how to develop and implement an AI acceptable use policy for their office in line with best practices for data governance when using AI tools.
Learn more
-
InnovateUS workshop: Bringing AI and Tech Talent into government by of Tech Talent Project
-
Article: Bias and retraining interactive explainer- Datasets Have Worldviews (pair.withgoogle.com) [Archive]
-
Guidebook: GSA’s AI Guide for Government
-
Chapter 2: How to structure an organization to embrace AI https://bit.ly/gsa-ai-org
-
Chapter 4: Developing the AI workforce bit.ly/gsa-workforce
-
Blog: See this blog from Kim Hicks at DataSF to get a window of the culture differences and motivation of private sector data scientist coming to public sector
-
Organizations
-
Tech Talent Project - Tech Talent Project
-
US Digital Response - Talent Division
Citations
-
GSA AI Guide for Government - https://bit.ly/gsa-ai-org
-
Meritalk - OPM Reclassifies Data Scientist Criteria for Agency Hiring - https://bit.ly/opm-reclass