Building an AI-Powered SQL Assistant with Metabase
Built an AI-powered SQL assistant for Metabase that lets anyone query databases using natural language — no SQL expertise needed.
The Inspiration
As a data scientist, I've worked across multiple companies, leveraging databases to extract insights for machine learning models. Over time, I found myself frequently assisting BI and BA teams in solving complex SQL queries, especially when launching new projects or defining new metrics. These teams often struggled with limited knowledge of database structures and SQL syntax, leading me to wonder: Is there a better way to make data accessible to everyone, regardless of their technical background?
Inspired by Pinterest's text-to-SQL approach and Uber's QueryGPT, I decided to build an AI-powered SQL assistant — an accessible tool that enables users to query databases using natural language.
Why Metabase?
To build this tool, I chose Metabase, an open-source data visualization platform I had years of experience working with. Metabase offers several advantages:
By leveraging Metabase, I could focus on designing an AI-driven query assistant without worrying about security, access control, or user interface development.
Implementation Overview
To enable AI-driven SQL generation, I designed a two-step agentic process within Metabase:
QueryDesigner Agent
Processes the user's natural language question, leveraging a complete database schema to determine the necessary tables and query structure.
QueryArchitect Agent
Retrieves detailed table documentation from a vector database, generating a refined SQL query that users can execute within Metabase.
Architecture Breakdown
Vector Database (Pinecone)
Stores indexed metadata and table descriptions for retrieval-augmented generation (RAG).
LLM Service (OpenAI)
Translates natural language queries into SQL, using database metadata as context.
Metabase UI Integration
Seamlessly integrates into Metabase's query builder, allowing users to enter plain-language queries and receive executable SQL statements.
Designing the AI Query Agent Flow
To build an AI agent that delivers meaningful insights from data regardless of a user's SQL expertise or familiarity with database structures, I needed to start with a foundational approach. My goal was to create the simplest possible architecture that could function within Metabase while maintaining efficiency and ease of use.
The Two-Step Process
1. QueryDesigner Agent
When a user submits a question, this agent processes the full database documentation, including schema details and an overview of the tables. Based on this information, it determines the relevant tables, extracts the necessary data, and outlines the SQL query structure.
2. QueryArchitect Agent
Using the information from the QueryDesigner Agent, this second agent retrieves detailed table documentation from the vector database. The vector database contains structured documentation for every table and field, including relationships and user-provided descriptions. With this enriched context, the QueryArchitect Agent generates an optimized SQL query. The final query is then presented in the Metabase interface, allowing users to execute it within their database and select their preferred visualization format.
Setting Up the AI Configuration
Before processing queries, the system requires initial configuration:
1. Database Documentation Storage
- Users select tables to index and provide a high-level database description.
- Metabase's metadata service extracts schema details, including field types and relationships.
2. AI Service Integration
- Admins securely add API keys for Pinecone (vector database) and OpenAI (LLM processing).
- A new AI Configuration section in Metabase's settings allows easy management.
3. Indexing Process
- Once API keys are added, users click a Create Index button.
- The system generates structured documentation, providing a comprehensive database overview.
- This documentation enhances the AI's ability to infer relationships (e.g., linking orders to addresses and cities for location-based queries).
User Experience: Querying with AI
To keep the workflow simple, I integrated the AI assistant directly into Metabase's query builder:
- User inputs a natural language question (e.g., "Show total sales per city").
- QueryDesigner Agent determines required tables and relationships.
- QueryArchitect Agent fetches metadata from the vector database and constructs an SQL query.
- SQL query is displayed in Metabase, allowing users to review, modify, and run it.
- Users select visualizations based on the retrieved data.
This approach minimizes disruptions while making data queries accessible to non-technical users.
Future Enhancements
This is just the first iteration of the AI-powered SQL assistant. Future improvements include:
Leveraging Pre-stored Queries & User Logs
- Identifying commonly used queries as authoritative examples.
- Enhancing AI accuracy by learning from past queries.
Adding Feedback Loops
- Automatically validating generated SQL queries before presenting them to users.
- Iteratively refining queries based on detected errors.
Get Involved
I'm actively looking for companies interested in testing this platform. The project is available on GitHub:
Conclusion
By integrating AI with Metabase, I aim to democratize data access, allowing anyone, regardless of technical expertise, to extract insights effortlessly. This is just the beginning, and I'm excited to refine and expand this tool with real-world feedback. Let's make data truly accessible!
At Churnless AI, we're committed to helping organizations leverage AI to make better use of their data.
If you're interested in implementing a similar solution for your team, contact us to learn more.