State and local governments share their journey to build AI sandboxes

As artificial intelligence (AI) reshapes how we work, government agencies face a unique challenge: How can they utilize AI responsibly? In a recent InnovateUS workshop, leaders from the State of New Jersey and the City of Boston shared their experiences developing AI sandboxes to enable responsible use of generative AI across their government workforces. 

The workshop was led by experts across the industry, including Santiago Garces, Dave Cole, Naman Agrawal, Amani Farooque, and Ruthie Nachmany.

New Jersey's comprehensive approach to AI Integration

New Jersey launched their AI Assistant sandbox in July 2024 alongside a comprehensive responsible AI training program. The platform, developed by the New Jersey Office of Innovation in partnership with the New Jersey Office of Information Technology, provides state employees with a secure environment to safely and responsibly experiment with generative AI in their government work. The State also recently announced the release of its AI Task Force report.

"Our journey toward establishing an AI sandbox started over a year ago with the Governor's executive order," Cole, the Chief Information Officer of New Jersey said. "The focus was clear: use AI to improve service delivery and enhance our state's economy."

The AI Assistant is a secure platform where state employees can experiment with AI in their daily work and try to make government services and programs even more efficient and responsive. Farooque, Director of Product at the New Jersey Office of Innovation, emphasized the office’s goals using the sandbox. 

“We wanted to provide not just a safe space for employees to apply their AI training, but also to lower the barriers to AI adoption and offer a secure alternative to free AI platforms," she said.

Real-World Applications

State employees are putting the AI Assistant to work in creative and practical ways:

• Document analysis: Workers are using the tool to digest lengthy PDFs and regulatory documents, making it easier to extract key information and respond to resident needs more quickly

• Public feedback processing: The system helps analyze resident feedback at scale, enabling an improved understanding of community needs and concerns that can be addressed in future program updates

• Communication enhancement: State employees are using the tool to draft emails and letters for both internal and external communication, streamlining communication processes while maintaining professional standards

Within just five months of launch, over 10,000 state employees (16% of the workforce) were actively using the platform, with a remarkable 79% positive feedback rating. The system has maintained a 97% success rate in handling prompts, all while staying well under budget.

Boston's innovation through partnership

Boston is taking a different, but equally innovative approach. Given the limitation of commercially available AI tools, the plans to leverage the sandbox built in partnership by  the Commonwealth of Massachusetts and Northeastern University's Burnes Center for Social Change and its AI for Impact program to develop a solution tailored to government needs.

"Existing access to generative AI tools is primarily through freely available models or individual subscriptions, resulting in limited accessibility and security concerns. We needed a solution that would give us greater control over user data and session records while managing costs effectively" said Garces, the Chief Information Officer for the City of Boston and the Technical Lead for AI for Impact at the Burnes Center.

Garces highlighted the City’s key motivations for their sandbox strategy, including: 

• Cost efficiency compared to per-user commercial licenses

• Enhanced security and privacy through controlled cloud infrastructure

• Ability to tailor features specifically for government use cases

• Platform for measuring outcomes and ROI of AI tools

Common themes and lessons learned

Both these initiatives offer valuable insights and guiding principles for other government bodies considering AI adoption.

1. Manage costs: Both jurisdictions emphasized the need to balance broad access with fiscal constraints, finding sandboxes more economical than individual commercial licenses

2. Prioritize security: Maintaining control over data and ensuring compliance with government security standards is essential

3. Present learning opportunities: Use platforms as tools for workforce education and responsible AI adoption

4. Plan a measured rollout: Both organizations are taking thoughtful approaches to deployment moving forward, with a strong emphasis on training and gathering user feedback

These initiatives represent more than just technological advancement; they're about making government work better for everyone. By providing secure, responsible ways to use AI, these programs are helping public servants work more efficiently while maintaining the high standards of security and accountability that government work demands.

The success of these programs suggests we're entering a new era of government innovation - one where AI is not just a buzzword, but a practical tool for improving public service delivery.

As more government organizations look to embrace AI technology, these early examples provide valuable blueprints for responsible and effective implementation. The emphasis on open-source solutions and shared learning across jurisdictions suggests a collaborative path forward for government AI adoption.