attacca.ai

/aˈtakːa/

An open-source methodology for building AI agent systems reliable enough to certify and insure.

Specs in. Certified software out.

Reliable Software = Spec Quality × Harness Enforcement × Continuous Evaluation

Read the Framework View on GitHub

By Jhon Moreno

Seamless forward momentum.

The Bottleneck

The model isn’t the problem. The spec is.

AI agents generate code in minutes. But ambiguous specs produce ambiguous software. Agents don’t ask clarifying questions — they assume.

Skills (prompts) get you to 90% reliability. For a 10-step workflow, that’s 6 failures every day.

The bottleneck isn’t the model. It’s the organizational context — the intent, the edge cases, the knowledge that took decades to build — that never makes it into the specification.

Attacca is a methodology that encodes that context into machine-actionable specifications, enforces the process through deterministic rails, and evaluates the output through structured adversarial testing.

The March of Nines

Per-Step Success	10-Step Workflow	Failures / Day (10 runs)
90% (skills only)	65% end-to-end	~6.5
99% (+ harness)	90% end-to-end	~1
99.9% (+ eval)	99% end-to-end	~0.1

The Framework

Eight phases. Every gate enforced.

IDEA

Capture the intent. Classify the risk.

DISCOVER

Map the existing codebase.

brownfield only

SPEC

Write a specification precise enough for autonomous agents.

BUILD

Execute on deterministic rails. Shift-left validation.

TEST

Behavioral scenarios + factorial stress testing.

CERTIFY

Human sign-off at tier-appropriate depth.

DEPLOY

Production setup with verified checklists.

MAINTAIN

Continuous flywheel. Drift detection.

IDEA

Capture the intent. Classify the risk.

DISCOVER

Map the existing codebase.

brownfield only

SPEC

Write a specification precise enough for autonomous agents.

BUILD

Execute on deterministic rails. Shift-left validation.

TEST

Behavioral scenarios + factorial stress testing.

CERTIFY

Human sign-off at tier-appropriate depth.

DEPLOY

Production setup with verified checklists.

MAINTAIN

Continuous flywheel. Drift detection.

Every phase produces artifacts. Every gate has entry requirements. No phase can be skipped.

Trust Architecture

Not everything needs the same rigor. Safety-critical systems need all of it.

Tier 1

Deterministic

Annoyance if wrong. 7 scenarios. Full auto.

Tier 2

Constrained

Wasted resources. Stress test variations. Logging.

Tier 3

Open

Financial/reputational damage. Intent contract. Human oversight.

Tier 4

High-Stakes

Legal/safety/irreversible. Domain expert. Full coverage.

Adversarial Evaluation

Four ways agents fail that standard testing never catches.

Standard evaluations test scenarios once under clean conditions. Factorial stress testing applies controlled variations to expose hidden failures.

FM-1

The Inverted U

Excels at routine, fails at extremes. Aggregate accuracy masks this.

FM-2

Knows But Doesn’t Act

Reasoning identifies the answer. Output contradicts it.

FM-3

Context Hijacks Judgment

Authority pressure shifts the recommendation.

FM-4

Guardrails Fire on Vibes

Triggers on surface language, not actual risk.

Taxonomy grounded in Mount Sinai Health System’s factorial design study (Nature Medicine, 2026) and analysis by Nate Jones.

Open Source

The methodology is free. The rigor is built in.

Attacca Forge

MIT License

The methodology as code. 7 skills across 5 layers.

L1: Specspec-architect, spec-writer

L2: Evalstress-test

L3: Intentintent-spec, intent-audit

L4: Orchestrationbuild-orchestrator

L5: Discoverycodebase-discovery

View on GitHub →

Attacca Claw

MIT License

Secure-by-default local second brain. Built on OpenClaw. Local-only. Privacy-first. No cloud dependency.

View on GitHub →

From the Build

Writing about what we’re learning.

Your AI Agent Knows the Answer (and Recommends the Opposite)→

The Spec Is the Bottleneck(coming March 23)

AI Doesn’t Replace People. It Replaces Excuses for Not Listening to Them.(coming March 26)

The work that shaped this.

No methodology is built in isolation. These are the people, research, and communities whose work directly influenced how Attacca thinks about reliability.

Nate Jones

AI evaluation research & failure mode taxonomy

His analysis of how AI agents fail under real-world conditions — the inverted U, reasoning-action gaps, context hijacking, and guardrail misfires — shaped the adversarial evaluation layer of this methodology.

YouTube→

Mount Sinai Health System

Factorial stress testing design

Their factorial design study on clinical AI agents (Nature Medicine, 2026) provided the empirical foundation for structured variation testing — the idea that you must test under controlled perturbations, not just clean scenarios.

Stripe

Engineering rigor at scale

Stripe’s engineering blog — particularly their writing on building reliable systems and making infrastructure invisible — influenced how we think about methodology as developer tooling, not process overhead.

Stripe Engineering Blog→

The Harness Engineer Community

Practitioner knowledge & field testing

The practitioners building and testing AI agent harnesses in production. Their real-world feedback on what breaks, what scales, and what the documentation never tells you is woven throughout this framework.