AI for Impact 
How Not to Buy Stupid AI: A Practical Guide to Evaluating AI Products and Tools
How Not to Buy Stupid AI: A Practical Guide to Evaluating AI Products and Tools
Agencies face increasing pressure to “use AI,” but not every automation or predictive tool is appropriate, and not every product delivers what it promises. Building on lessons learned from how researchers evaluate the effectiveness and fairness of AI tools, this session offers practical strategies for government. Participants will discuss how to interrogate vendor claims, what to demand of those selling us AI, work with partners in academia and industry to measure outcomes, and avoid paying for systems that don’t perform in practice or, worse, cause more harm than good. We discuss the limitations of AI auditing and practical strategies for getting smarter about what we buy.
-
Use evidence-based methods for evaluating vendor claims and proposals.
-
Identify red flags such as black-box models and unverifiable performance metrics.
-
Come away with a “buyer’s guide” grounded in scientific rigor
Regulating Algorithms: What Governments Around the World Are Doing—and What Public Servants Should know