AI Safety Test Kit
Test your AI assistant before each release and add a safety layer that logs, flags and can switch it off.
Your first result: a pass/fail test report for the included example assistant.
$49
One-time · instant download
Coming soon. Leave your email and we'll tell you the day it opens.

What you get
What's inside the kit
A test runner for AI assistants
Run the same test questions before every release and get JSON, Markdown or JUnit reports.
11 ready-made scoring checks
Score answers automatically instead of reading them all by hand.
Checks for prompt-injection and data leaks
Helps catch common attack and leak patterns (it can still miss some).
A redacted audit log
Keeps a record of what your AI assistant did, with secrets masked.
A kill switch
Stop your AI assistant instantly without redeploying.
A before-and-after comparison
Fails your build if something that used to pass starts failing.
How to put it to work
How to get started
Follow these steps in order. After you buy, the same steps appear as a checklist in your kit viewer, with links to the exact guide for each step.
- 1
Run the built-in checks
5 minRun npm test, then npm run eval to see a report on the 12 example cases.
- 2
Write your own test suite
20–30 minCopy the example JSON suite and add your assistant's real questions and what should happen.
- 3
Connect your real assistant
15–30 minPut your assistant behind the safety wrapper so risky tool calls wait for a person to approve them.
- 4
Save a baseline
5 minRecord today's results, review and commit them. From then on, the test run fails if a passing case breaks.
- 5
Know your kill switch, then add CI
10 minCreate the kill-switch file (or set the env var) to stop the assistant. Then copy the read-only GitHub workflow.
Is it right for me?
Who it's for
A great fit if…
- Developers and agencies delivering AI assistants to clients
- Node.js projects (your assistant connected through a small JavaScript file)
Probably not for you if…
- You need a security audit or compliance certification
Questions
Frequently asked questions
Will it stop every attack?
No. The safety checks are pattern-based and can miss paraphrased, non-English or encoded attacks. They help catch common patterns.
Does it call OpenAI or Anthropic?
Not by default. Everything runs offline with a mock model. Live adapters are optional, off by default and untested live.
Do I need to install packages?
No. It has no npm dependencies; Node.js 20+ is enough.
Good to know before you buy
- The guardrails are pattern-based and can miss attacks (paraphrased, non-English or encoded injection, jailbreaks and leaks). They help catch common patterns; they don't fully detect or prevent anything.
- Provided as is, without warranty; liability capped at the price you paid.
- Not a security audit, penetration test or compliance certification.
- No promise of any outcome, time savings or income.
Technical details (for the technically curious)
Technical words on this page, explained
- AI agent:
- An AI assistant that can take steps for you (like running a command or calling a tool), not just chat.
- LLM / AI model:
- The AI "brain" behind tools like ChatGPT or Claude. You usually rent it from a provider with an account key.
- Evals:
- Repeatable tests for an AI assistant: the same questions every time, so you can see if it got better or worse.
- Guardrails:
- Safety checks around an AI assistant that flag or block risky requests and answers.
- CLI / terminal:
- Typing short commands into a text window on your computer instead of clicking buttons.
- JSON / Markdown:
- Simple text file formats. Markdown is readable notes; JSON is structured data for programs.
- CI / GitHub workflow:
- Automatic checks that run every time code changes, so problems are caught early.
Formerly / technical name: Agent Eval & Guardrails Harness
Full description
A small, dependency-free Node toolkit to help you test, guard and audit LLM agents: eval runner and CLI, regression gate, redacted audit log, kill switch and release checklist. Runs fully offline by default. Not a security audit or compliance certification.
Full parts list
- Eval runner + CLI (JSON / Markdown / JUnit reports)
- 11 scorers
- Pattern-based injection/leak checks (can miss attacks)
- Guarded agent wrapper
- Redacted JSONL audit log
- Kill switch
- Regression comparison
- 12 offline example cases
- SHA-pinned CI workflow, release checklist
Requirements
- Node.js 20+
- Your agent wrapped in a small JS adapter
Tested on
- Node 20.19.2 / npm 11.21.0
- Linux 6.12.94+
Tested offline with a mock model and recorded fixtures. OpenAI/Anthropic adapters untested live; CI not run on GitHub. (2026-10-28)
What's in the download (42 files)
- 📁 .github/ (1 file)
- 📁 bin/ (1 file)
- 📁 docs/ (2 files)
- 📁 evals/ (5 files)
- 📁 reports/ (4 files)
- 📁 src/ (15 files)
- 📁 tests/ (5 files)
- 📄 .gitignore
- 📄 .gitleaks.toml
- 📄 CHANGELOG.md
- 📄 LICENSE
- 📄 package.json
- 📄 README.md
- 📄 RELEASE_CHECKLIST.md
- 📄 REVIEW_STATUS.md
- 📄 SETUP.md
Version and changes
v0.1.0 · tested 2026-10-28
- 0.1.0 · 2026-10-28: Initial release: eval runner, scorers, guardrails, wrapper, logger, kill switch, reports, adapters. Compliance review: license, disclaimers, live-adapter gate.