What Is AgentProof?

AgentProof is a local proof harness for AI-built apps, repositories, and public pull requests. Built by dicnunz and released as an open-source tool, it exists to answer a single, stubborn question: when an AI coding agent claims it finished a feature, did the resulting app actually work?

The tool is deliberately positioned against the most common failure mode of AI coding agents. A coding agent produces a transcript full of confident “done” messages, and a reviewer is left staring at a large diff and still has to judge mergeability and correctness by hand. AgentProof automates that judgment by running the app and verifying outcomes rather than eyeballing code.

Key properties from the official repository:

• Detects the project type automatically and runs safe local checks appropriate to that stack.
• Starts the application and crawls its routes using Playwright.
• Captures screenshots and console errors during the crawl.
• Checks accessibility and broken links.
• Produces a numeric “launch readiness” score.
• Writes a static report to agentproof-report/index.html.

Because everything runs locally, AgentProof makes no paid API calls, needs no API key, requires no hosting, uses no database, sends no telemetry, and never uploads your source code.

Why AI-Generated Diffs Need a New Kind of Verification

The traditional code review workflow was built for human-authored changes. A human writes a diff, and a reviewer reads it, reasons about intent, and approves or requests changes. That model breaks down when the diff is generated by an AI agent.

The core problem is the scale and nature of AI diffs. As the ZeroNoise analysis of verification-first coding agents notes, the core failure is “coming back to a 1000-line diff and still having to judge mergeability and correctness yourself.” When an agent produces a thousand lines of changes in minutes, line-by-line review becomes impractical, and reading a transcript tells you little about whether the software actually runs.

There is also a determinism problem. CI/CD for AI agents behaves differently from CI/CD for traditional software because agents are non-deterministic. As the UnderstandingData analysis of CI/CD agent patterns points out, the same prompt can produce different outputs on each run, which means a test that passed yesterday can fail today for no obvious reason.

The emerging consensus, captured in the “proof beats diffs” thesis, is that teams should stop reviewing code and start verifying outcomes. Instead of asking “does this diff look correct?”, you ask “does the built application work?” That shift is exactly what AgentProof automates.

The market context is enormous. The OpenAI Codex CLI repository has roughly 105,000+ stars as of mid-2026, underscoring how large the AI-coding-tool audience has grown. That audience increasingly needs trust tooling before large AI-generated diffs get merged.

How AgentProof Works: From Project Detection to Launch Score

AgentProof’s verification pipeline is designed to be automatic and hands-off. The workflow follows a defined sequence:

1. Project detection. AgentProof inspects the repository to determine the project type and stack.
2. Safe local checks. It runs checks that are safe to execute locally and appropriate for the detected stack.
3. Application startup. It launches the app so the runtime behavior can be observed.
4. Route crawling with Playwright. It crawls the application’s routes, simulating a user navigating through the UI.
5. Evidence capture. During the crawl it captures screenshots and console errors.
6. Quality checks. It checks accessibility and looks for broken links.
7. Scoring. It aggregates the results into a launch-readiness score.

The output is a static HTML report. This report becomes the durable evidence artifact that a reviewer, a CI system, or a human stakeholder can inspect to judge whether the app actually works.

This runtime-verification approach is philosophically different from simply validating that code compiles or that unit tests pass. AgentProof verifies that the application launches, renders, navigates, and behaves correctly as a running product — which is closer to what a human reviewer actually cares about when an agent claims a feature is done.

AgentProof in CI: GitHub Action, Min-Score Gate, and Badge

The most practical reason to use AgentProof for AI code CI verification is its GitHub Action integration. AgentProof ships a reusable GitHub Action that plugs directly into a workflow.

The key feature is the min-score gate. You can configure a minimum launch-readiness score, for example min-score: '70', and the workflow fails if the app scores below that threshold. This turns a fuzzy “did the AI do a good job?” judgment into a deterministic, repeatable quality gate.

The Action also produces a README badge, so the launch-readiness score becomes a persistent, visible signal on the repository. Path targeting is supported, allowing you to scope the verification to a specific directory, such as examples/demo-app.

This score-gate pattern matters because it addresses the non-determinism of AI agents head-on. When a human reviews every AI diff, the gate is inconsistent and slow. When a CI gate enforces a numeric threshold, the verification is automatic, repeatable, and cheap to run on every pull request.

A typical usage flow is:

1. An AI agent produces a diff and opens a pull request.
2. The GitHub Action runs AgentProof against the PR’s build.
3. AgentProof crawls the app, scores launch readiness, and writes the report.
4. The workflow passes or fails based on the configured min-score.
5. The report and badge give reviewers objective evidence to merge or reject.

Privacy and Local-First Design: No API Key, No Telemetry, No Source Upload

A central selling point of AgentProof is its local-first, privacy-preserving design. The tool is explicitly built to verify AI code without uploading source or trusting a cloud vendor.

The design constraints from the repository are unambiguous:

• No paid API. All verification runs locally with local tooling.
• No API key. There is nothing to configure or leak.
• No hosting. The report is a static local file.
• No database. No server-side state.
• No telemetry. The tool does not phone home.
• No source upload. Your code stays on your machine.

This makes AgentProof attractive for teams with strict privacy and sovereignty requirements. If your organization cannot send proprietary source code to a third-party verification service, a local harness is the only viable option for automated AI-code verification.

For open-source projects, this local design also means anyone can audit the verification itself. There is no opaque cloud service sitting between the code and the verdict — the entire proof is reproducible on any machine.

Pricing and the $149 Mini Audit: When Free Local Proof Isn’t Enough

AgentProof itself is free and open source. The monetization model is a fixed-price professional service for cases where local verification alone is not sufficient.

The paid offering is a Mini Audit at a flat $149 for an asynchronous, written proof packet of one public repository, demo, or pull request. The scope is deliberately narrow: no calls, no secrets, no private-repo access. You get an external, written verification that a public AI-built artifact actually works.

The pricing logic is framed as a break-even trade against the review churn it replaces. At a fully loaded rate of $50/hour, $149 breaks even after roughly 3 hours of saved review time. At $100/hour, it breaks even after 1.5 hours. If you would otherwise spend that long manually reviewing and re-testing a large AI diff, the Mini Audit pays for itself.

The Mini Audit is aimed at a specific situation: when you need outside, independent written proof — for a client deliverable, a pitch, a launch, or a public-facing claim — and running the harness yourself is not sufficient evidence.

AgentProof vs ProofPack: Runtime Verification vs Evidence Bundle

The most direct comparison for AgentProof is ProofPack, another open-source local-only tool in the same space. Although both claim to be “proof” tools for AI coding agents, they solve different problems.

ProofPack positions itself as “receipts for AI coding agents.” It wraps Claude Code, Codex CLI, Hermes, or any command and writes a proof folder containing the prompt, stdout/stderr logs, git before/after state, git.diff, a commands log, test output, a manifest, and checksums.

AgentProof, by contrast, is runtime verification. It does not primarily care about the execution transcript; it cares about whether the resulting application behaves correctly when launched.

In practice the two are complementary. ProofPack answers “what did the agent actually do?” AgentProof answers “does the finished app actually work?” A rigorous team could use both: ProofPack to capture provenance, and AgentProof to verify the runtime outcome.

Pros and Cons

Who Should Use AgentProof (and Who Shouldn’t)

AgentProof is a strong fit if:

• You rely heavily on AI coding agents and merge large generated diffs.
• You need a deterministic CI gate for non-deterministic agent output.
• You cannot or will not send proprietary source to a third-party cloud verification service.
• You want objective launch-readiness evidence for reviews, demos, or client deliverables.
• You maintain open-source projects where a README score badge adds trust.

AgentProof may not be the right tool if:

• Your work is mostly backend logic with no launchable app or browsable routes to crawl.
• You need deep, line-by-line code correctness analysis rather than runtime outcome verification.
• You require verification of code that must never be executed locally in your environment.
• You need independent, third-party written proof of a private repository — that is outside the Mini Audit’s scope.

Verdict and Alternatives

AgentProof is a genuinely useful addition to the AI-verification toolkit. It aligns with the strongest trend in the space — the “proof beats diffs” shift from reviewing code to verifying outcomes — and it does so with a discipline most competitors lack: fully local, no telemetry, no source upload, no paid API. The GitHub Action with a min-score gate turns fuzzy human judgment about AI output into a repeatable CI check, which is precisely what teams merging large agent-generated diffs need.

Its main limitation is scope. Runtime, launch-oriented verification is not a substitute for code correctness analysis, and teams with strict non-execution constraints will find it unusable. The pricing is also worth weighing against recurring spend: a single agent task costs roughly $0.015–0.05 in tokens, a 50-scenario suite runs $0.75–2.50, and 20+ runs a day on every PR can total $15–50 per month — so a $149 Mini Audit is best reserved for cases where local proof is not enough.

Strong alternatives depend on your priority. For diff provenance and execution evidence, ProofPack is the natural counterpart. For teams that want cloud-scale verification and spec-driven loops, the broader verification-first agents ecosystem — exemplified by Cursor’s 10x usage jump when agents can prove their own correctness — points to a future where verification is the core product feature of coding agents, not an afterthought.

For most teams, the sensible starting point is the free, local harness: wire AgentProof into CI, set a reasonable min-score gate, and let the launch score do the reviewing while you focus on the outcomes that matter.

FAQ

What is AgentProof? AgentProof is a free, local-first proof harness that verifies AI-built apps and public pull requests by detecting the project type, running safe checks, launching the app, crawling routes with Playwright, and producing a launch-readiness score in a static report.

How does AgentProof verify AI-generated code diffs in CI? It ships a GitHub Action with a min-score gate (for example min-score: '70'). The Action crawls the built app, scores launch readiness, and fails the workflow if the app scores below the threshold, turning agent verification into a deterministic CI check.

Is AgentProof really free and private? Yes. It requires no paid API, no API key, no hosting, no database, and no telemetry, and it never uploads your source code. Everything runs locally and outputs a static agentproof-report/index.html.

What is the $149 Mini Audit? It is a fixed-price, asynchronous written proof packet for one public repository, demo, or pull request — no calls, no secrets, no private-repo access. It breaks even after roughly 3 hours of saved review time at $50/hour or 1.5 hours at $100/hour.

How does AgentProof compare to ProofPack? AgentProof verifies the runtime outcome — whether the launched app actually works and scores — while ProofPack is an evidence bundle that captures what the agent did (prompt, logs, git.diff, checksums). They are complementary: ProofPack gives provenance, AgentProof gives runtime proof.

What Is a Knowledge Inbox and Why Go Local-First?

A knowledge inbox is the capture stage of a personal knowledge management (PKM) system. Think of it as the “inbox zero” pattern applied to everything you consume: an article you bookmarked, a YouTube video, a podcast episode, a PDF whitepaper, a screenshot, or a WeChat message. Instead of leaving those inputs scattered across apps and browser tabs, a knowledge inbox routes them into one normalized destination where your notes and your AI agents can actually use them.

The “local-first” part is the differentiator. Cloud read-it-later tools like Readwise, Matter, and Instapaper sync your highlights to their servers, where they are subject to their privacy policies, pricing tiers, and eventual shutdown. A local-first knowledge inbox keeps your raw materials and your enriched knowledge cards on your own machine or network. That matters more in 2026 than it did a few years ago, because the inputs you want to preserve are increasingly private: personal research, medical PDFs, financial statements, and sensitive internal documents.

Obsidian’s own ecosystem is a signal of how big this space has grown. The official plugin/community release repository has passed 20,700 GitHub stars, reflecting a large, active community of plugins and tools built around the vault format. A knowledge inbox plugs directly into that ecosystem, which means the structured cards it produces work with the same tools, themes, and graph views you already rely on.

How Local-First Ingestion Works: From Raw Source to Markdown Knowledge Card

The reference knowledge-inbox pipeline is a clean, reproducible model for how local-first ingestion works. Every input flows through the same stages regardless of where it came from:

1. Source Adapter — A connector that pulls content from a specific source, such as a web page, YouTube video, or PDF file.
2. ContentItem — A normalized internal representation of the raw input. Every source produces the same data shape, so downstream stages never care where the content came from.
3. Cleaner / OCR / Whisper / AI — The enrichment stage. It strips boilerplate, extracts text from images and scans, transcribes audio and video, and can hand off to an LLM for summarization and structuring.
4. Classifier / Tags / Knowledge Linker — The intelligence stage. Content is classified, automatically tagged, and linked to related knowledge cards so the result is discoverable rather than dumped.
5. Obsidian Markdown + SQLite — The dual-store output. The human-readable Markdown lives in your vault, while a SQLite index makes the content queryable by your agents.

The dual-store design is worth calling out because it is what separates a knowledge inbox from a simple web clipper. Clippers save a snapshot; a knowledge inbox produces a structured, tagged, cross-linked card that your notes and agents can both use.

The 12-Source Ingestion Layer (Web, Social, Media, PDF, Telegram)

The breadth of source coverage is where a mature knowledge inbox earns its keep. The reference project implements 12 source adapters that normalize content from:

• Web — any URL, article, or page
• WeChat Official Accounts — long-form WeChat posts
• X/Twitter — threads and posts
• YouTube — video, including automatic transcription
• Podcast RSS — episodes and show notes
• Vimeo — video content
• Direct media — uploaded audio and video files
• PDF — documents, including scanned PDFs via OCR
• Images — screenshots and photos
• Telegram — saved messages and channels

The value is not just the number of sources — it is that every one of them produces the same ContentItem shape downstream. Whether you save a YouTube video or a PDF, the enrichment pipeline treats them identically. That normalization is what makes the whole system manageable: one cleaner, one tagger, one linker, one store.

This directly competes with the “send it to Readwise” reflex. A local-first inbox gives you comparable capture breadth, but the output stays on your machine and is immediately usable by your notes and agents rather than living in a proprietary cloud silo.

AI-Enrichment Pipeline: OCR, Whisper, Auto-Tagging, and Knowledge Linking

Raw captured content is only half the job. The other half is making it findable and connected, and that is where the AI-enrichment pipeline does the heavy lifting:

• OCR converts scanned PDFs and images into searchable text. Without it, a photographed whiteboard or a scanned contract is effectively invisible to search.
• Whisper transcribes audio and video. This turns a 45-minute podcast or an hour-long talk into a text transcript that can be chunked, summarized, and searched.
• LLM summarization condenses long documents into digestible knowledge cards, preserving the essence without forcing you to re-read everything.
• Auto-tagging classifies each card so it surfaces in the right searches and vault folders.
• Knowledge linking connects related cards, building the web of connections that makes a vault more than a folder of files.

The practical result is that content becomes queryable the moment it lands. You do not need to transcribe, tag, or summarize anything by hand. The pipeline does it, and the enriched card is ready for both your own reading and your agents’ retrieval.

MCP as the 2026 Integration Standard for AI Agents and Obsidian

The Model Context Protocol (MCP) has become the dominant integration surface between AI agents and local tools, and Obsidian vaults are squarely in that trend. Every notable local-first knowledge tool — engraph, Molio, and swarmvault — ships an MCP server for Obsidian vaults. The reference knowledge-inbox project is harness-neutral, exposing the same adapters and processing service to Hermes, Codex, and OpenClaw via MCP.

MCP matters because it standardizes how agents talk to your data. Rather than each agent tool maintaining its own private API to your notes, an MCP server provides a common protocol: the agent issues standard tool calls, and the server reads and writes the vault. This is why a harness-neutral design is a selling point in 2026 — one ingestion service can feed whichever agent you use today, and you are not locked into a single vendor’s tooling.

For Obsidian specifically, an MCP server means your agents can:

• Retrieve knowledge cards by semantic search over the vault
• Ingest new content directly into the correct location
• Link new cards to related existing notes automatically
• Query the SQLite index for fast, structured lookups

The result is a vault that is not just a note-taking app but a data store your agents operate on natively.

Competitor Landscape Review: Knowledge-Inbox vs Khoj vs Engraph vs Molio vs Swarmvault

The local-first knowledge space has several credible options, each with a different emphasis. Here is how the main players stack up:

The strategic split is between ingestion-led and retrieval-led tools:

• Khoj is the most mature (36,400+ stars) and is retrieval-led. It excels at semantic search and answering questions over your existing docs, with strong Obsidian/Emacs integration and optional offline inference. It is less focused on the structured ingestion pipeline — it assumes your content is already in place.
• knowledge-inbox is ingestion-led. Its strength is the normalization of many raw sources into consistent knowledge cards before they enter your store. If your problem is “I have content everywhere and I want it structured,” this is the gap it fills.
• Engraph (164 stars) is narrowly focused on the retrieval/query layer — a hybrid-search MCP server over Obsidian vaults. It complements, rather than replaces, an ingestion pipeline.
• Molio (181 stars) competes on the knowledge-layer angle with an evolving knowledge graph plus web/WeChat ingestion.
• Swarmvault (654 stars) leans into the “agent memory” and local-first wiki narrative, built on Karpathy’s llm-wiki concept with an MCP server.

The signal across all of them is that MCP is becoming the standard interface and that local-first storage is the shared value proposition. They differ mainly in whether they prioritize getting content in (ingestion) or getting answers out (retrieval).

Key Considerations: Privacy, Cost, Setup Complexity, and Vault Portability

Before you adopt a local-first knowledge inbox, weigh these four factors:

Privacy. Local-first means your content — including sensitive PDFs, private transcripts, and personal research — never leaves your machine unless you choose to send it to an external LLM. Many pipelines let you run OCR and tagging locally, and tools like Khoj support fully offline inference via llama.cpp. If privacy is your top priority, local-first is the clear advantage over cloud read-it-later services.

Cost. A self-hosted pipeline runs on hardware you already own, with no per-seat subscription. The trade-off is that heavy AI enrichment — especially Whisper transcription and LLM summarization — consumes local compute or requires an API key if you offload to a hosted model. Cloud tools have predictable monthly pricing but ongoing subscription costs; local-first trades that for upfront setup and your own compute.

Setup complexity. A local-first ingestion pipeline is more complex to stand up than a hosted app. You are managing Python/FastAPI services, source adapters, an SQLite database, and an MCP server. If you are comfortable with self-hosting, this is manageable; if not, the learning curve is real. Tools like Khoj simplify this by packaging a turnkey self-hosted experience.

Vault portability. Because the output is plain Markdown plus a standard SQLite index, your knowledge cards are not locked into a proprietary format. You can move, export, or back them up with ordinary file tools, and the Markdown works with any tool that reads the format — not just Obsidian. This portability is a genuine long-term advantage over cloud silos.

Who Should Adopt a Local-First Knowledge Inbox in 2026

A local-first knowledge inbox is a strong fit if you:

• Use Obsidian as your primary knowledge base and want your AI agents to read and write the same vault
• Consume a lot of varied media — articles, videos, podcasts, PDFs — and want them normalized into one structure
• Value privacy and prefer not to ship your reading and research through cloud servers
• Are comfortable self-hosting a Python/FastAPI stack and an MCP server
• Want agent integration across multiple tools (Hermes, Codex, OpenClaw) without being locked into one vendor

It is a poorer fit if you want a zero-setup, hosted solution or if you rarely consume content that needs enrichment. For a simple note-taker who never touches PDFs or videos, the pipeline is overkill — a basic clipper and manual tagging will do.

Final Verdict and Recommendations

For 2026, the local-first knowledge inbox is a compelling answer to the “content everywhere” problem, especially for Obsidian users who also run AI agents. The reference knowledge-inbox project shows the right architecture: 12 source adapters → normalized ContentItem → OCR/Whisper/AI enrichment → tagged, linked Markdown cards in a dual Markdown + SQLite store, all exposed to agents through a harness-neutral MCP server.

Our recommendations by use case:

• Capture-led, multi-source, self-hosters — start with a knowledge-inbox-style pipeline. It solves the hardest problem: turning disparate sources into structured, queryable cards.
• Answer-led users who already have rich notes — Khoj is the mature, well-supported choice (36,400+ stars) with strong search and optional offline inference.
• Knowledge-graph-first teams — Engraph or Molio if graph retrieval is your priority; Swarmvault if you are building agent memory on the llm-wiki pattern.

The through-line is unmistakable: local-first storage and MCP integration are the 2026 standard, and the tools that respect your data’s ownership while making it usable by agents will win the workflows of people who take their knowledge seriously.

FAQ

What is a knowledge inbox for Obsidian? A knowledge inbox is a capture pipeline that takes links, PDFs, videos, screenshots, and notes and converts them into structured Markdown knowledge cards inside your Obsidian vault, enriched with OCR, transcription, and auto-tagging.

How is a knowledge inbox different from a read-it-later app like Readwise? Read-it-later apps store your highlights in the cloud, subject to their privacy and pricing. A local-first knowledge inbox keeps everything on your machine, outputs standard Markdown plus a SQLite index, and is directly readable by your notes and AI agents.

Do I need cloud AI to use a local-first knowledge inbox? No. You can run enrichment locally, including OCR, Whisper transcription, and even LLM inference via tools like llama.cpp (as Khoj supports). Offloading to a hosted model is optional and controlled by you.

What is MCP and why does it matter for Obsidian and AI agents? MCP (Model Context Protocol) is the standard protocol that lets AI agents talk to local tools. An MCP server for an Obsidian vault lets agents retrieve, ingest, and link knowledge cards using a common interface, and it works across harnesses like Hermes, Codex, and OpenClaw.

Can I export my knowledge if I leave the tool? Yes. Because the output is plain Markdown and a standard SQLite index, your knowledge cards are portable. You can back up, move, or open them with any tool that reads those formats — there is no proprietary lock-in.