NSW Government

An AI assistant to improve the complaints experience

In partnership with NSW Government

Yesterday

We partnered with a NSW Government agency responsible for ensuring that the government’s decisions are lawful, reasonable, and fair.

The size, complexity, and unfamiliar structures of government make it hard for people to lodge a complaint with clarity and confidence.

The objective was to develop a custom-built assistant that leveraged Artificial Intelligence (AI) to simplify access to government services and complex information. The challenge was to integrate the intuitive, natural language interface offered by LLMs, while ensuring the responses and guidance were accurate – without hallucinations.

The solution aimed to enhance the citizen experience and improve internal operational efficiency by reducing manual workloads.

Today

There was a level of uncertainty that we could create a product that was accurate enough to be helpful. A risk of using LLMs is that they are adept at confidently delivering believable but false information.

Our project employed an innovation methodology, involving stepped hypotheses, experimentation, testing, and refinement.

We apply a Human-Centred Artificial Intelligence (HCAI) approach, which places human needs, values, and well-being at the heart of how AI products and services are imagined, designed, engineered, and operated.

As a first step, we established an Ethical AI Framework to guide our product. The framework included clear guidance around user privacy, data management, and AI boundaries.

We chose a Retrieval-Augmented Generation (RAG) architecture, which enables us to combine the conversational power of a large language model (LLM) with the ability to retrieve specific, relevant, and accurate information.

This approach enabled us to develop an AI assistant that not only provides information but also builds context for user intent and needs through natural conversation.

This product was developed as a full-stack solution, focusing on robust, scalable, and secure technologies.

Key components of the system:

User Interface (UI): Our chat interface uses React, ensuring a responsive and accessible user experience across all devices. The application offers a seamless conversational flow, mimicking natural dialogue to facilitate user navigation of their queries.
AI Core: The natural language understanding and generation capabilities are powered by AWS Bedrock, leveraging the Claude 3.5 Sonnet model. The foundation model was chosen for its advanced reasoning and ethical alignment, crucial for handling sensitive government-related inquiries.
Knowledge Retrieval: Responses are grounded in a comprehensive knowledge base, leveraging AWS Knowledge Base and OpenSearch to efficiently retrieve trusted information. This Retrieval-Augmented Generation (RAG) architecture ensures that the AI’s answers are accurate, up-to-date, and directly sourced from approved information.
Observability: The complex AI interactions, including conversation context management and orchestration, are handled by the LangChain framework. For continuous monitoring, evaluation, and debugging of the AI’s performance, we integrated the LangSmith platform, allowing us to fine-tune responses and identify edge cases.
Data Integration: A key component of the solution was the integration of a backend system to manage complaint forms. This alleviated manual tasks for internal teams by creating a direct, seamless connection between the public-facing forms and the agency’s internal case management system.

The magic lies in the seamless integration between content strategy, design, and AI technology. Users can have natural conversations about their needs without navigating complex menus or organisation-first structures.
Jason D'Souza, Technology Director, Today

Tomorrow

Our solution demonstrates that conversational interfaces can adapt to user needs in ways that static websites simply cannot. It’s a paradigm shift that enables us to truly personalise and tailor knowledge to a user’s needs and context.

Less broadcast, more genuine assistance.

The outcomes for the NSW Government are:

Reduced staff time spent answering basic questions
A static knowledge base is now transformed into personalised, contextual guidance.
People face fewer barriers and are armed with better information to form their complaints.