A small AI lab in public. Notes, tools, and experiments.

A tiny harness makes prompt work honest. Fix data and settings, change one thing at a time, and record errors. Download the JSON template and use the script to get a quick pass/fail snapshot.

How to use

  1. Download the JSON template.
  2. Edit cases and variants. Keep changes atomic.
  3. Run: node scripts/ablation_harness.js data/ablation-template.json > summary.json

Script

node scripts/ablation_harness.js data/ablation-template.json > summary.json

See the source at /scripts/ablation_harness.js. Replace callModel with your LLM call.

Why this helps