USDR Showcases AI Prototypes for Government: Project AIDA and GALA

On August 21, InnovateUS hosted a workshop with members of U.S. Digital Response (USDR). The workshop was led by Krista Canellakis, Marcie Chin, Neil Shah, and Google.org Fellow Polina Dudnik, who discussed two recent prototyping projects leveraging generative AI (GenAI) to assist government agencies.

USDR is a nonpartisan non-profit organization that works alongside governments of all levels to ensure they have the capacity to meet the public's needs. Powered by 9,500 volunteers, USDR has built relationships with 400 government agencies across the country completing more than 500 projects. The group builds prototypes for common government use cases such as permitting, unemployment insurance, language access, and data analysis.

“[USDR is] product and tech neutral,” Canellakis said. “So we really listen to our partners to understand their tech environment and build what will work best for them.”

Project AIDA (AI Draft Assistant for Benefits Review)

The first project shared was Project AIDA. For one government partner, less than 50 staff handle tens of thousands of unemployment benefits appeals annually. Given the case volume and manual processes, timeliness, and quality standards can be challenging. USDR held a design sprint, focused on generative AI opportunities, to help increase response times, consistency, and quality of case handling.

USDR hypothesized that AI could help automate the process of decision letter drafting to reduce processing time and case backlog. In lieu of heavily modifying existing workflows or processes the partner and USDR created a standalone tool that lists applicable laws and drafts a letter based on the examinerʼs decision, the facts, and applicable laws.

Dudnik explained that building a unique tool is not always necessary for these kinds of projects, but for this particular use case, it was the most effective way to address the problem.

“Often the case is that if you have a large user base and you want a standard use case and kind of control the experience, more likely than not it is advisable not to give them the LLM chatbot directly, but to wrap it in something personal,” she said.

It was also important to note that the LLM did not make the decision, and required human input to function. 

The pilot results for the project showed about 88% accuracy and produced positive user feedback, scoring an average score of 4 out of 5 across the three measured criteria: ease of use, helpfulness, and accuracy.

Project GALA (Generative AI for Language Access)

The second project, Project GALA, was a collaboration with the state of New Jersey to improve language access for unemployment insurance (UI).

Workers with limited English proficiency struggle to access UI benefits and are at higher risk of being targeted for UI fraud. Governments often the lack capacity to provide meaningful translation services. To address this issue, USDR partnered with the state to launch their first ever Spanish language UI application form, developing a framework for Spanish language access using generative AI. By fine-tuning a large language model (LLM) with training data grounded in user research and UI policy expert review, the team was able to use the model to produce translations close to human expert quality. 

NJ’s human-centered revamp of its UI application form has reduced the average application time from 3+ hours to 28 minutes and reduced the number of claims requiring manual review by 14%. With the release of the form in Spanish, the state is reporting parity in form completion time between English and Spanish-speaking claimants. 

Chin highlighted the project's potential for broader impact.

“The commercialization of GenAI technology really gave us the opportunity to scale all of that human-centered work that was put into developing New Jersey’s new UI intake form,” Chin said. “What we’re proposing here is that all of this work is still necessary, but other states and benefit programs don’t have to keep reinventing that wheel.” 

To maximize project value, it is essential to keep a human in the loop. This meant having translations reviewed for accuracy and readability.

USDR is continuing this work with the New Jersey Department of Labor to expand their unemployment glossary, so that the catalog will be available in plain language in both English and Spanish. 

The workshop then went into breakout groups for participants to dive deeper into each project. At the end of their presentation, USDR offered their support to other government agencies interested in exploring the potential of GenAI responsibly.

“Generative AI presents an opportunity to benefit the public by scaling staff in government. Our approach focuses on aiding humans in complex decision analysis, repetitive tasks, and synthesizing information–please reach out to see how we might help,” Shah said. 

To learn more watch the workshop recording and reach out to USDR here.