PromptGuard AI

AI/MLIntermediateJan 6, 2026

PromptGuard AI

Version, Test, and Validate AI Prompts for Reliable Structured Outputs.

The Problem

Indie hackers and small dev teams frequently build AI-powered features that rely on Large Language Models (LLMs) to generate structured data (e.g., JSON, YAML, XML). The core problem is the inherent unreliability and inconsistency of LLM outputs. A small change in a prompt or an update to the underlying LLM can silently break downstream parsing logic, leading to corrupted data, application errors, and a frustrating debugging loop. Developers waste countless hours manually testing prompts with various inputs, trying to ensure the output adheres to a strict schema. There's no robust system to version prompts alongside their expected output schemas, nor a simple way to automate the validation of LLM outputs against these schemas, quantifying their consistency and quality. This lack of structured testing and versioning leads to brittle AI integrations, slowing down development cycles and eroding confidence in AI-driven features.

The Solution

PromptGuard AI provides a web-based platform for developers to define, version, test, and validate their AI prompts, specifically focusing on structured data outputs. Users can create 'Prompt Projects,' define an output JSON schema (or paste existing JSON for inference), and then write prompts targeting that schema. The unique selling proposition is its integrated test runner: users add various input texts ('test cases') and PromptGuard AI will execute the prompt against selected LLMs (OpenAI, Anthropic, Gemini), attempt to parse the output against the defined schema, and report validation success/failure. It tracks prompt and schema versions, allowing developers to see how changes impact output quality over time. An API endpoint enables continuous integration, allowing external systems to validate LLM outputs programmatically before consuming them. This ensures robust, consistent AI integrations, saving developers debugging time and increasing feature reliability.

Tech Stack

Frontend
Next.js 14ReactTailwind CSSZod (for schema validation in FE)
Backend
Next.js API RoutesTypeScriptZod (for schema validation in BE)json-schema-to-ts (for schema inference)
Database
PostgreSQL (Supabase)
APIs
Stripe (for payments)Resend (for transactional emails)OpenAI APIAnthropic APIGoogle Gemini API

System Architecture

Database Schema

CREATE TABLE organizations (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  name TEXT NOT NULL,
  owner_id UUID REFERENCES auth.users(id) ON DELETE CASCADE,
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE TABLE projects (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
  name TEXT NOT NULL,
  description TEXT,
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE TABLE schemas (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  project_id UUID REFERENCES projects(id) ON DELETE CASCADE,
  version INT NOT NULL DEFAULT 1,
  definition_json JSONB NOT NULL, -- The JSON Schema definition
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
  UNIQUE (project_id, version)
);
CREATE INDEX idx_schemas_project_id ON schemas(project_id);

CREATE TABLE prompts (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  project_id UUID REFERENCES projects(id) ON DELETE CASCADE,
  schema_id UUID REFERENCES schemas(id) ON DELETE SET NULL, -- Link to a specific schema version
  version INT NOT NULL DEFAULT 1,
  name TEXT NOT NULL,
  content_template TEXT NOT NULL, -- The prompt text with placeholders
  model_settings_json JSONB, -- e.g., { "model": "gpt-4o", "temperature": 0.7 }
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
  UNIQUE (project_id, name, version)
);
CREATE INDEX idx_prompts_project_id ON prompts(project_id);
CREATE INDEX idx_prompts_schema_id ON prompts(schema_id);

CREATE TABLE test_cases (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  prompt_id UUID REFERENCES prompts(id) ON DELETE CASCADE,
  name TEXT NOT NULL,
  input_text TEXT NOT NULL, -- The input text to feed the prompt
  expected_output_json JSONB, -- Optional: for comparing with parsed LLM output
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_test_cases_prompt_id ON test_cases(prompt_id);

CREATE TYPE validation_status AS ENUM ('success', 'schema_mismatch', 'content_mismatch', 'llm_error', 'timeout');

CREATE TABLE test_runs (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  test_case_id UUID REFERENCES test_cases(id) ON DELETE CASCADE,
  prompt_version_id UUID REFERENCES prompts(id) ON DELETE CASCADE, -- Snapshot of prompt used
  schema_version_id UUID REFERENCES schemas(id) ON DELETE CASCADE, -- Snapshot of schema used
  llm_model_used TEXT NOT NULL,
  raw_llm_output TEXT, -- The raw text output from the LLM
  parsed_output_json JSONB, -- The parsed JSON from LLM output
  validation_status validation_status NOT NULL,
  error_details TEXT, -- Details if validation failed or LLM error
  duration_ms INT,
  cost_usd NUMERIC(10, 6),
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_test_runs_test_case_id ON test_runs(test_case_id);
CREATE INDEX idx_test_runs_prompt_version_id ON test_runs(prompt_version_id);

CREATE TABLE api_keys (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
  key_hash TEXT NOT NULL UNIQUE, -- Hashed API key
  name TEXT NOT NULL,
  is_active BOOLEAN DEFAULT TRUE,
  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
  last_used_at TIMESTAMP WITH TIME ZONE
);
CREATE INDEX idx_api_keys_organization_id ON api_keys(organization_id);

API Endpoints

POST/api/auth/loginUser login
POST/api/auth/signupUser signup
GET/api/organizationsList user's organizations
POST/api/organizationsCreate a new organization
GET/api/projectsList projects for the current organization
POST/api/projectsCreate a new project
GET/api/projects/[projectId]Get project details
POST/api/projects/[projectId]/schemasCreate a new schema version for a project
GET/api/projects/[projectId]/schemasList all schema versions for a project
GET/api/schemas/[schemaId]Get a specific schema version
POST/api/projects/[projectId]/promptsCreate a new prompt version for a project
GET/api/projects/[projectId]/promptsList all prompt versions for a project
GET/api/prompts/[promptId]Get a specific prompt version details
PUT/api/prompts/[promptId]Update a prompt (creates new version)
POST/api/prompts/[promptId]/test-casesAdd a new test case to a prompt
GET/api/prompts/[promptId]/test-casesList test cases for a prompt
POST/api/prompts/[promptId]/run-testsExecute all test cases for a prompt against selected LLMs
GET/api/prompts/[promptId]/test-runsGet results of previous test runs for a prompt
POST/api/validateExternal API for programmatic prompt validation (requires API key). Takes prompt_id, input_text, and returns validation status and parsed output.
POST/api/checkout-sessionCreate Stripe checkout session for subscription
POST/api/webhook/stripeStripe webhook handler for subscription updates
šŸ¤–

Start Building with AI

Copy this prompt for Cursor, v0, Bolt, or any AI coding assistant

šŸ‘·

...

builders copied today

Found this useful? Share it with your builder friends!

BD

BuilderDaily Team

Verified

Indie hackers and full-stack engineers creating validated Micro-SaaS blueprints with production-ready tech stacks.

AI/ML
Code TestedSchema ValidatedProduction Ready
Coming Soon in Beta

Gap Alert

Today's gap expires in ~14 hours

Get tomorrow's blueprint delivered to your inbox so you never miss a profitable idea.

(Email delivery launching soon — sign up to be first!)

No spam, ever•Unsubscribe anytime