PromptGuard AI
PromptGuard AI
Version, Test, and Validate AI Prompts for Reliable Structured Outputs.
Indie hackers and small dev teams frequently build AI-powered features that rely on Large Language Models (LLMs) to generate structured data (e.g., JSON, YAML, XML). The core problem is the inherent unreliability and inconsistency of LLM outputs. A small change in a prompt or an update to the underlying LLM can silently break downstream parsing logic, leading to corrupted data, application errors, and a frustrating debugging loop. Developers waste countless hours manually testing prompts with various inputs, trying to ensure the output adheres to a strict schema. There's no robust system to version prompts alongside their expected output schemas, nor a simple way to automate the validation of LLM outputs against these schemas, quantifying their consistency and quality. This lack of structured testing and versioning leads to brittle AI integrations, slowing down development cycles and eroding confidence in AI-driven features.
PromptGuard AI provides a web-based platform for developers to define, version, test, and validate their AI prompts, specifically focusing on structured data outputs. Users can create 'Prompt Projects,' define an output JSON schema (or paste existing JSON for inference), and then write prompts targeting that schema. The unique selling proposition is its integrated test runner: users add various input texts ('test cases') and PromptGuard AI will execute the prompt against selected LLMs (OpenAI, Anthropic, Gemini), attempt to parse the output against the defined schema, and report validation success/failure. It tracks prompt and schema versions, allowing developers to see how changes impact output quality over time. An API endpoint enables continuous integration, allowing external systems to validate LLM outputs programmatically before consuming them. This ensures robust, consistent AI integrations, saving developers debugging time and increasing feature reliability.
Tech Stack
System Architecture
āāāāāāāāāāāāāāāāāāāā
ā Client ā
ā (Browser/Mobile) ā
āāāāāāāāāāā¬āāāāāāāāā
ā
ā¼
āāāāāāāāāāāāāāāāāāāā āāāāāāāāāāāāāāāāāāāāāāāāāā
ā Next.js App ā ā External LLMs ā
ā (Frontend/API) ā ā (OpenAI, Anthropic, ā
ā ā ā Google Gemini) ā
āāāāāāāāāāā¬āāāāāāāāā āāāāāāāāāāāāāā²āāāāāāāāāāāā
ā (API Requests) ā
ā¼ ā (LLM Calls)
āāāāāāāāāāāāāāāāāāāā ā
ā API Routes āāāāāāāāāāāāāāāāāāāāāāāā
ā (PromptGuard BE) ā
āāāāāāāāāāā¬āāāāāāāāā
ā
ā¼
āāāāāāāāāāāāāāāāāāāā
ā Supabase ā
ā (PostgreSQL DB, ā
ā Auth, Storage) ā
āāāāāāāāāāā¬āāāāāāāāā
ā
ā¼
āāāāāāāāāāāāāāāāāāāā
ā Stripe (Payments)ā
āāāāāāāāāāāāāāāāāāāā
āāāāāāāāāāāāāāāāāāāā
ā Resend (Emails) ā
āāāāāāāāāāāāāāāāāāāāDatabase Schema
CREATE TABLE organizations (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name TEXT NOT NULL,
owner_id UUID REFERENCES auth.users(id) ON DELETE CASCADE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE projects (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
name TEXT NOT NULL,
description TEXT,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE schemas (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
project_id UUID REFERENCES projects(id) ON DELETE CASCADE,
version INT NOT NULL DEFAULT 1,
definition_json JSONB NOT NULL, -- The JSON Schema definition
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
UNIQUE (project_id, version)
);
CREATE INDEX idx_schemas_project_id ON schemas(project_id);
CREATE TABLE prompts (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
project_id UUID REFERENCES projects(id) ON DELETE CASCADE,
schema_id UUID REFERENCES schemas(id) ON DELETE SET NULL, -- Link to a specific schema version
version INT NOT NULL DEFAULT 1,
name TEXT NOT NULL,
content_template TEXT NOT NULL, -- The prompt text with placeholders
model_settings_json JSONB, -- e.g., { "model": "gpt-4o", "temperature": 0.7 }
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
UNIQUE (project_id, name, version)
);
CREATE INDEX idx_prompts_project_id ON prompts(project_id);
CREATE INDEX idx_prompts_schema_id ON prompts(schema_id);
CREATE TABLE test_cases (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
prompt_id UUID REFERENCES prompts(id) ON DELETE CASCADE,
name TEXT NOT NULL,
input_text TEXT NOT NULL, -- The input text to feed the prompt
expected_output_json JSONB, -- Optional: for comparing with parsed LLM output
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_test_cases_prompt_id ON test_cases(prompt_id);
CREATE TYPE validation_status AS ENUM ('success', 'schema_mismatch', 'content_mismatch', 'llm_error', 'timeout');
CREATE TABLE test_runs (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
test_case_id UUID REFERENCES test_cases(id) ON DELETE CASCADE,
prompt_version_id UUID REFERENCES prompts(id) ON DELETE CASCADE, -- Snapshot of prompt used
schema_version_id UUID REFERENCES schemas(id) ON DELETE CASCADE, -- Snapshot of schema used
llm_model_used TEXT NOT NULL,
raw_llm_output TEXT, -- The raw text output from the LLM
parsed_output_json JSONB, -- The parsed JSON from LLM output
validation_status validation_status NOT NULL,
error_details TEXT, -- Details if validation failed or LLM error
duration_ms INT,
cost_usd NUMERIC(10, 6),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_test_runs_test_case_id ON test_runs(test_case_id);
CREATE INDEX idx_test_runs_prompt_version_id ON test_runs(prompt_version_id);
CREATE TABLE api_keys (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
key_hash TEXT NOT NULL UNIQUE, -- Hashed API key
name TEXT NOT NULL,
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
last_used_at TIMESTAMP WITH TIME ZONE
);
CREATE INDEX idx_api_keys_organization_id ON api_keys(organization_id);API Endpoints
/api/auth/loginUser login/api/auth/signupUser signup/api/organizationsList user's organizations/api/organizationsCreate a new organization/api/projectsList projects for the current organization/api/projectsCreate a new project/api/projects/[projectId]Get project details/api/projects/[projectId]/schemasCreate a new schema version for a project/api/projects/[projectId]/schemasList all schema versions for a project/api/schemas/[schemaId]Get a specific schema version/api/projects/[projectId]/promptsCreate a new prompt version for a project/api/projects/[projectId]/promptsList all prompt versions for a project/api/prompts/[promptId]Get a specific prompt version details/api/prompts/[promptId]Update a prompt (creates new version)/api/prompts/[promptId]/test-casesAdd a new test case to a prompt/api/prompts/[promptId]/test-casesList test cases for a prompt/api/prompts/[promptId]/run-testsExecute all test cases for a prompt against selected LLMs/api/prompts/[promptId]/test-runsGet results of previous test runs for a prompt/api/validateExternal API for programmatic prompt validation (requires API key). Takes prompt_id, input_text, and returns validation status and parsed output./api/checkout-sessionCreate Stripe checkout session for subscription/api/webhook/stripeStripe webhook handler for subscription updatesStart Building with AI
Copy this prompt for Cursor, v0, Bolt, or any AI coding assistant
BuilderDaily Team
VerifiedIndie hackers and full-stack engineers creating validated Micro-SaaS blueprints with production-ready tech stacks.
Related Blueprints
More Micro-SaaS ideas you might like to build
MediBrief AI
AI-powered clinical note summarization for healthcare professionals.
AssessGenius AI
Instantly create engaging quizzes from any text using AI.
DevWallet
Automate tracking and optimize your SaaS & cloud spending as a developer.
Gap Alert
Today's gap expires in ~14 hours
Get tomorrow's blueprint delivered to your inbox so you never miss a profitable idea.
(Email delivery launching soon ā sign up to be first!)