Engineering case study

An evolving harness for AI-assisted software engineering.

ArtifactFlow is the real, security-sensitive workload behind a reusable repository workflow for Codex and Claude Code: durable context, tests before behavior, constrained commands, isolated databases, explicit handoffs, adversarial review, and human approval where automation should stop.

Start before code

Product definition and architecture constrain the agents.

The repository begins with a written product boundary: a versioned artifact vault that preserves generated source and lets authenticated teams and approved agents search, preview, run, update, and govern it. AGENTS.md, the README, architecture documents, threat model, operations guide, and roadmap describe what the system is and what it deliberately is not.

That persistent context matters more than a clever one-off prompt. It keeps implementation work aligned with the modular-monolith structure, the two-origin security model, UID identifiers, typed application commands, durable events, authorization rules, and the narrow alpha scope.

Executable expectations

New behavior starts with a failing test.

Feature work and bug fixes add or update a failing test before production code changes. Focused Laravel tests cover use cases, validation, authorization, and database behavior. Browser tests are reserved for interactions and security boundaries that a PHP response assertion cannot prove, including opaque iframe behavior and hostile DOM construction.

The point is not a ceremonial red-green cycle. Tests become a stable handoff artifact: another agent or reviewer can see the intended behavior, reproduce the failure, and verify that the eventual implementation solves the specified problem without weakening a neighboring boundary.

Persistent context

Repository state and explicit handoffs outlive a chat window.

Architecture decisions, threat assumptions, commands, test rules, and release requirements live in versioned files. Work can be split into bounded research, implementation, review, or verification tasks, but the durable result returns to the repository as tests, code, documentation, and a concise handoff rather than undocumented agent memory.

This supports multi-agent work without pretending parallelism is automatically useful. Tasks need clear ownership, non-overlapping write scopes, and an integration point. Security claims are traced back to code and tests rather than accepted from an agent summary.

Tested policy

Codex and Claude Code receive the same conservative guard rules.

Project hooks inspect prompt, command, approval, and file-write events exposed by each agent host. Shared Python policy blocks or escalates high-risk patterns: direct Laravel test runners, secret-bearing file reads, destructive Git operations, recursive deletion, Docker volume removal, command constructions that hide interpreter behavior, and pushes without explicit approval.

make ai-hooks-test exercises the policy corpus. Codex adds an exec-policy file; Claude Code and Codex have separate hook configuration, but both call the shared policy implementation.

Defense in depth, not a sandbox

A command-string denylist cannot understand arbitrary code at runtime. Real secret isolation still requires an OS or container boundary where credentials are absent or unreadable and network access is constrained.

Safe test environments

PHP and browser tests never inherit the development database.

The Laravel wrapper creates a unique temporary PostgreSQL database, injects the testing environment, runs the selected suite, and drops only that database. Direct php artisan test, Pest, or PHPUnit commands are prohibited because they can inherit local application configuration.

The browser wrapper goes further: it creates another isolated database, starts dedicated app and artifact-host services on separate hosts and ports, applies migrations, routes Playwright setup commands into that environment, and tears the database down afterwards. The origin split remains real even during browser tests.

Human authority

Human approval remains the rule for consequential external actions.

Host hooks enforce recognizable commands and events, and the harness tests that behavior, but they cannot prove that every possible execution path will comply.

  • Pushes require approvalThe working agreement requires explicit approval for each push. Hooks reject recognized unapproved push commands, but another execution path can evade that check.
  • Destructive commands require approvalThe policy reserves recursive deletion, worktree-discarding Git commands, Docker volume deletion, and reset targets for explicit approval. Hooks escalate the patterns they can recognize.
  • External account changes are separateRepository settings, deployment mutations, credentials, and third-party publishing remain outside default authority unless specifically authorized.
  • Unexpected database risk requires a stopThe working agreement requires disclosure before further database writes if a test or migration may have touched local development data.
Defense in depth, not hostile containment

These controls are bypassable. They are workflow guardrails for cooperative agent hosts, not a security boundary against a malicious operator or fully privileged process. A process with unrestricted host access can disable hooks, call alternate tools, or run arbitrary code by definition. Their job is to catch mistakes, preserve the ordinary approval flow, and make policy drift testable. Real containment comes from least privilege, absent credentials, isolated environments, OS or container boundaries, and owner review.

Verification

Adversarial review and release gates challenge the implementation.

Security-sensitive changes are reviewed against trust boundaries and triggerability, then exercised with focused regression cases. The full gate set combines style, maximum-level static analysis, type and line coverage, dependency audit, Semgrep rules and fixtures, secret-risk checks, asset build, isolated browser tests, production-image build, and image scanning.

Documentation is part of the same verification loop. Architecture, operations, and threat claims must match what the code and tests currently enforce; a polished claim that outruns implementation is a defect.

Cross-repository contract

Shared guard files carry a versioned drift contract.

harness/ai-hooks-contract.json records a contract version and SHA-256 hashes for the policy files intended to stay shared. Some files must be byte-identical; others permit declared project substitutions such as a repository slug or focused test name before hashing.

A Laravel drift test enforces ArtifactFlow's side without depending on a sibling checkout in CI. A deliberate harness change is coordinated across participating repositories, then bumps the contract version and affected hashes. This is a compact reuse mechanism, not a package manager or a released universal framework.

What transfers

The workflow is reusable; the workload stays specific.

Framework-agnostic

Persistent repository instructions, test-first handoffs, approval boundaries, command/file policy, isolated mutable dependencies, review roles, evidence-backed security audits, and a versioned drift contract can transfer to other stacks.

ArtifactFlow-specific

Laravel command wrappers, PostgreSQL setup, Playwright service topology, the two-origin HTML sandbox, content-scanning rules, MCP authority model, exact gate commands, and the threat corpus belong to this product.

Current status

This is an evolving case study and reusable harness contract proven against ArtifactFlow. It is not presented as a finished, universal multi-agent framework.

Inspect the evidence

Start with the public contract and the security workload.