<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Vericoding on RockB</title><link>https://baeseokjae.github.io/tags/vericoding/</link><description>Recent content in Vericoding on RockB</description><image><title>RockB</title><url>https://baeseokjae.github.io/images/og-default.png</url><link>https://baeseokjae.github.io/images/og-default.png</link></image><generator>Hugo</generator><language>en-us</language><lastBuildDate>Mon, 15 Jun 2026 19:10:54 +0000</lastBuildDate><atom:link href="https://baeseokjae.github.io/tags/vericoding/index.xml" rel="self" type="application/rss+xml"/><item><title>Vericoding AI Formal Verification Code Correctness: How AI Proves Its Own Code Is Correct (2026)</title><link>https://baeseokjae.github.io/posts/vericoding-ai-code-verification-guide-2026/</link><pubDate>Mon, 15 Jun 2026 19:10:54 +0000</pubDate><guid>https://baeseokjae.github.io/posts/vericoding-ai-code-verification-guide-2026/</guid><description>A practical 2026 guide to vericoding, AI formal verification, and machine-checked code correctness.</description><content:encoded><![CDATA[<p>Vericoding is AI-assisted software development where code is generated with formal specifications and machine-checked correctness proofs, not only tests or review. In 2026, it matters because AI coding is common, but trust in “almost right” generated code is the limiting factor for serious production use.</p>
<h2 id="what-does-vericoding-mean-in-2026">What Does Vericoding Mean in 2026?</h2>
<p>Vericoding is the practice of using AI to produce code together with a formal specification and a machine-checkable proof that the implementation satisfies that specification. The largest public vericoding benchmark reports 12,504 formal specifications across Dafny, Verus/Rust, and Lean, including 6,174 unseen problems, which makes the term more than a branding exercise. In practical terms, vericoding changes the deliverable from “the model wrote code that looks plausible” to “the model produced code that a verifier accepted under explicit rules.” The verifier may be Dafny, Lean 4, Verus, SPARK, Coq/Rocq, an SMT solver, or a model checker. The AI can still hallucinate candidate programs and proof attempts, but invalid proofs are rejected by the checker instead of being trusted by a reviewer. The core takeaway: vericoding is AI coding with correctness evidence attached.</p>
<p>The most important shift is not that AI becomes magically reliable. The shift is that the reliability claim moves from model confidence to a separate tool with deterministic semantics. A language model can propose a binary search implementation, a loop invariant, and a proof outline. Dafny can then reject the proof if the invariant fails for an edge case. That feedback is concrete enough for an agent to repair the implementation or split the proof into smaller lemmas.</p>
<p>For senior engineers, the useful mental model is “compiler plus proof checker plus coding agent.” The model writes, the verifier judges, and the loop repeats until the artifact either verifies or exposes that the specification is wrong, incomplete, or too expensive to prove.</p>
<h2 id="how-is-vericoding-different-from-vibe-coding-and-traditional-testing">How Is Vericoding Different From Vibe Coding and Traditional Testing?</h2>
<p>Vericoding differs from vibe coding because it requires explicit correctness properties and machine acceptance, while vibe coding relies on natural-language prompting, manual inspection, and runtime experiments. The contrast matters because the 2025 Stack Overflow Developer Survey found that 84% of respondents use or plan to use AI tools, yet the top frustration was “almost right, but not quite” answers at 66%. Traditional testing samples behavior through selected examples; vericoding proves behavior over a specified input space. A test can show that <code>sort([3,1,2])</code> returns <code>[1,2,3]</code>; a formal proof can show that for every valid input list, the output is ordered and contains exactly the original elements. The tradeoff is cost: writing the right specification and invariants takes discipline. The clear takeaway: tests find evidence, while vericoding tries to establish proof under stated assumptions.</p>
<table>
  <thead>
      <tr>
          <th>Approach</th>
          <th>Main input</th>
          <th>Main output</th>
          <th>Strength</th>
          <th>Common failure mode</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Vibe coding</td>
          <td>Prompt and examples</td>
          <td>Plausible code</td>
          <td>Fast exploration</td>
          <td>Hidden bugs and hallucinated APIs</td>
      </tr>
      <tr>
          <td>Traditional testing</td>
          <td>Test cases</td>
          <td>Pass/fail signals</td>
          <td>Regression protection</td>
          <td>Untested edge cases</td>
      </tr>
      <tr>
          <td>Static analysis</td>
          <td>Source code and rules</td>
          <td>Warnings or guarantees</td>
          <td>Scales across codebases</td>
          <td>False positives or shallow properties</td>
      </tr>
      <tr>
          <td>Vericoding</td>
          <td>Formal spec plus code</td>
          <td>Machine-checked proof</td>
          <td>Strong correctness evidence</td>
          <td>Wrong or incomplete specification</td>
      </tr>
  </tbody>
</table>
<h3 id="what-does-vibe-coding-still-do-well">What does vibe coding still do well?</h3>
<p>Vibe coding is useful for prototypes, throwaway scripts, UI scaffolding, and domains where the cost of a wrong answer is low. A developer can ask an AI assistant for a React component, run it locally, and iterate from screenshots. That workflow is fast because it avoids specification work. It becomes dangerous when teams mistake speed for assurance. The code may pass a happy-path demo while mishandling authorization, concurrency, rounding, or malformed input.</p>
<h3 id="why-are-tests-still-necessary">Why are tests still necessary?</h3>
<p>Tests remain necessary because verified code still runs inside messy systems with I/O, versioned dependencies, users, timeouts, and deployment configuration. A verified parser can still be wired to the wrong queue. A proved payment calculation can still receive stale exchange rates. Use tests to validate integration, performance, migration behavior, and operational contracts. Use vericoding where the property is crisp enough to specify.</p>
<h2 id="how-does-ai-prove-code-correct-with-specs-proofs-and-checkers">How Does AI Prove Code Correct With Specs, Proofs, and Checkers?</h2>
<p>AI proves code correct by generating or repairing formal artifacts that an independent checker can validate, usually a specification, an implementation, invariants, lemmas, and proof steps. In the public vericoding benchmark, off-the-shelf LLMs reached 82% success in Dafny, 44% in Verus/Rust, and 27% in Lean, showing that the checker, language, and proof burden strongly affect results. The AI does not “prove” correctness by sounding convincing. It proposes text in a formal language, and a proof engine decides whether the proof follows from accepted rules. For program verification, the specification may include preconditions, postconditions, loop invariants, ownership rules, termination measures, or refinement claims. For theorem proving, the goal may be a mathematical statement about the program. The key takeaway: AI contributes search and repair, but the checker supplies the trust boundary.</p>
<p>A small Dafny-style example makes the workflow concrete. Suppose the goal is an absolute value function. The specification says the result must be non-negative and equal to either <code>x</code> or <code>-x</code>. The implementation is trivial for a human, but the verifier still requires the branch conditions to imply the postconditions. For loops, the proof burden grows quickly. A function that sums an array may need an invariant relating the loop index, accumulator, and mathematical sum of the processed prefix.</p>
<p>AI helps because proof engineering is repetitive and syntax-heavy. It can infer missing invariants, propose helper lemmas, and translate a natural-language intent into a first formal draft. The hard engineering question is whether the formal intent is actually the business intent.</p>
<h2 id="what-is-the-vericoding-workflow-propose-check-repair-decompose">What Is the Vericoding Workflow: Propose, Check, Repair, Decompose?</h2>
<p>The vericoding workflow is an agentic loop where an AI proposes code and proof artifacts, runs a verifier, reads errors or counterexamples, repairs the artifact, and decomposes hard goals into smaller lemmas. AlphaVerus demonstrates this pattern by using verifier feedback, translation, tree search refinement, and filtering to bootstrap formally verified code generation without human intervention or model fine-tuning. In day-to-day engineering, the loop looks less exotic: generate a candidate, run <code>dafny verify</code> or a Lean build, capture the failing obligation, ask the model to explain the missing invariant, and try a narrower proof. When the checker returns a counterexample, the model can decide whether the code is wrong or the specification is too weak. The takeaway: vericoding works best as a tight feedback loop, not as one-shot code generation.</p>
<h3 id="what-should-the-agent-do-after-a-verifier-failure">What should the agent do after a verifier failure?</h3>
<p>The agent should classify the failure before editing. A syntax error needs a mechanical fix. A failed postcondition may require a stronger branch condition, a missing lemma, or a different implementation. A failed loop invariant may mean the invariant is too weak on preservation or too strong on initialization. A timeout may need decomposition rather than a larger prompt. Blind retries waste tokens and often produce proof churn.</p>
<h3 id="why-does-decomposition-matter">Why does decomposition matter?</h3>
<p>Decomposition matters because proof search degrades when a goal combines too many facts. A sorting proof can be split into permutation preservation, ordering, bounds safety, and termination. Each lemma gives the checker a smaller target and gives the AI a clearer repair surface. In my experience, the difference between a stuck proof and a verified proof is often one named lemma that captures the missing idea.</p>
<h2 id="which-tools-and-languages-matter-dafny-lean-verusrust-spark-and-coqrocq">Which Tools and Languages Matter: Dafny, Lean, Verus/Rust, SPARK, and Coq/Rocq?</h2>
<p>Vericoding tools matter because each language encodes a different compromise between automation, expressiveness, runtime integration, and proof ergonomics. Dafny currently appears automation-friendly in benchmarks, with one reported pure Dafny verification improvement from 68% to 96% over the prior year, while Lean remains central for expressive theorem proving and broader mathematical infrastructure. Verus brings verification ideas into Rust-style systems programming, SPARK applies formal methods to safety- and security-critical Ada code, and Coq/Rocq has a long history in certified compilers and proof-heavy systems. The right tool depends on the property. A data-structure invariant may fit Dafny. A protocol proof may fit TLA+ or a model checker. A Rust memory-safety-adjacent proof may fit Verus. The takeaway: choose the verifier for the property, not because a model demo looked impressive.</p>
<table>
  <thead>
      <tr>
          <th>Tool or language</th>
          <th>Best fit</th>
          <th>Why AI helps</th>
          <th>Watch out for</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Dafny</td>
          <td>Algorithms, contracts, loop invariants</td>
          <td>Strong automation and readable specs</td>
          <td>Solver timeouts and brittle invariants</td>
      </tr>
      <tr>
          <td>Lean 4</td>
          <td>Theorems, deep specifications, proof libraries</td>
          <td>Tactic search and lemma discovery</td>
          <td>Higher proof-engineering cost</td>
      </tr>
      <tr>
          <td>Verus/Rust</td>
          <td>Systems code with Rust-like ownership</td>
          <td>Translating code intent into specs</td>
          <td>Smaller ecosystem than mainstream Rust</td>
      </tr>
      <tr>
          <td>SPARK/Ada</td>
          <td>Safety-critical embedded and defense software</td>
          <td>Drafting contracts and proof fixes</td>
          <td>Requires disciplined Ada/SPARK workflow</td>
      </tr>
      <tr>
          <td>Coq/Rocq</td>
          <td>Certified systems and foundational proofs</td>
          <td>Proof script repair and lemma search</td>
          <td>Steep learning curve</td>
      </tr>
  </tbody>
</table>
<h3 id="is-lean-better-than-dafny-for-vericoding">Is Lean better than Dafny for vericoding?</h3>
<p>Lean is not simply better than Dafny; it is better for different proof shapes. Lean is powerful when the target is mathematical precision, reusable theorem libraries, and deep reasoning. Dafny is often more direct for program verification with contracts, loops, arrays, and SMT-backed automation. If your team wants to verify utility functions, parsers, and algorithmic kernels, Dafny may produce results sooner. If your team needs rich theorem development, Lean deserves serious attention.</p>
<h2 id="what-do-the-latest-vericoding-benchmarks-actually-show">What Do the Latest Vericoding Benchmarks Actually Show?</h2>
<p>The latest vericoding benchmarks show fast progress but uneven reliability across languages and task types. One benchmark reports 12,504 formal specifications and success rates of 82% in Dafny, 44% in Verus/Rust, and 27% in Lean for off-the-shelf LLMs, while VeriBench evaluates 140 Lean 4 tasks spanning HumanEval, algorithms, security-critical programs, and Python standard library programs. VeriBench reports current limitations sharply: Claude 3.7 Sonnet achieved 35.0% compilation success, theorem accuracy was 0.615% under an LLM-judge metric, and a trace-based self-debug agent reached 49.3% compilation success. Those numbers are not a reason to dismiss the field. They are a map of where engineering work remains. The takeaway: vericoding is real, but benchmark success is highly sensitive to language, task design, and evaluation criteria.</p>
<p>The practical lesson is to avoid headline-driven adoption. A high Dafny success rate on benchmark tasks does not mean your service authorization layer can be verified next sprint. A low Lean theorem-accuracy score does not mean Lean is unsuitable for all AI-assisted proof work. Benchmarks compress multiple problems into one number: synthesis, specification understanding, proof search, library knowledge, compiler compatibility, and repair behavior.</p>
<p>For teams, benchmark results should drive pilot scope. Start with pure functions, serialization rules, validation logic, state-machine transitions, or algorithmic kernels. Track verification pass rate, human repair time, proof churn, and escaped defects. The useful internal metric is not “can the model verify benchmark tasks?” It is “can this workflow reduce review burden and production risk on properties we actually care about?”</p>
<h2 id="where-does-vericoding-work-today">Where Does Vericoding Work Today?</h2>
<p>Vericoding works today where correctness properties are narrow, explicit, and stable enough to encode formally. Good candidates include pure functions, financial rounding rules, parsers, bounded state machines, cryptographic helper routines, authorization predicates, serialization/deserialization invariants, and protocol transition logic. The reason is simple: these domains have crisp properties such as “the balance never goes negative,” “decoded output re-encodes to the same bytes,” or “only an admin can perform this transition.” In safety-critical contexts, SPARK/Ada workflows already show how proof of absence of runtime errors and functional correctness can expose AI-generated corner cases that tests miss. AI improves the economics by drafting contracts and proof repairs, but the property must still be precise. The takeaway: vericoding is most valuable on compact code where a wrong edge case is expensive.</p>
<h3 id="what-is-a-realistic-first-team-project">What is a realistic first team project?</h3>
<p>A realistic first project is a small library with stable rules and painful edge cases. Examples include currency normalization, date range overlap, access-control predicates, retry-state transitions, or a parser for an internal configuration format. Do not start with the whole application. Start with one module where the specification can fit on a page and the business owner can confirm the property in plain English.</p>
<h3 id="how-does-this-help-code-review">How does this help code review?</h3>
<p>Vericoding helps code review by changing what reviewers inspect. Instead of arguing over every branch, reviewers can focus on whether the specification says the right thing and whether assumptions are acceptable. That is still hard work, but it is higher-leverage work. The verifier handles many mechanical paths. Reviewers handle intent, boundaries, and integration risk.</p>
<h2 id="where-does-vericoding-still-fail">Where Does Vericoding Still Fail?</h2>
<p>Vericoding still fails when the specification is wrong, incomplete, ambiguous, or disconnected from the real system. A machine-checked proof can show that code satisfies “discount is at most 20%,” but it cannot know the business changed the enterprise discount cap to 25% unless someone updates the specification. Integration bugs remain another hard boundary: a verified function can be called with the wrong units, stale data, incorrect permissions, or a malformed external response. Scale also hurts. Large systems involve concurrency, databases, network failures, feature flags, latency budgets, migrations, and human workflows that are difficult to capture in one proof. The biggest risk is false confidence: teams may treat a verified artifact as proof that the product behavior is correct. The takeaway: vericoding proves stated properties, not unstated intent.</p>
<p>This is the same old formal-methods warning, but AI makes it easier to forget. If the model writes both the code and the spec from the same vague prompt, it may produce a beautifully verified implementation of the wrong requirement. For example, a refund function can prove that it never refunds more than the original charge while omitting a fraud-hold rule that lives in a policy document. The verifier is doing its job; the engineering process failed.</p>
<p>The mitigation is independent specification review. Ask domain owners to approve the property. Add examples that should and should not satisfy the spec. Keep a trace from requirement to formal contract. Treat proof artifacts as code: review them, version them, and test their integration assumptions.</p>
<h2 id="how-should-teams-adopt-vericoding-safely">How Should Teams Adopt Vericoding Safely?</h2>
<p>Teams should adopt vericoding safely by starting with bounded, high-value properties, assigning specification ownership, and measuring human repair effort instead of only verifier pass rates. A good 2026 adoption plan begins with one repository module, one verifier, and one property class, such as input validation or state-machine safety. The process should require a human-reviewed natural-language requirement before formalization, generated code and proof artifacts in version control, CI verification on every change, and ordinary tests around integration behavior. Track how many proof failures were code bugs, spec bugs, missing lemmas, or tool limitations. That taxonomy prevents the team from blaming the model for every issue or trusting it blindly. The takeaway: vericoding adoption is a software engineering process change, not just another AI tool rollout.</p>
<h3 id="what-should-go-into-ci">What should go into CI?</h3>
<p>CI should run the verifier, ordinary unit tests, and at least one negative check that proves the spec is not vacuous. If a postcondition can be weakened until any code passes, the proof is not useful. Keep verifier commands deterministic where possible, pin tool versions, and make timeouts visible. A flaky proof build will lose developer trust faster than a normal flaky test because the failure mode is harder to interpret.</p>
<h3 id="who-owns-the-specification">Who owns the specification?</h3>
<p>The team owns the specification, not the AI. For product logic, that may mean an engineer pairs with a product owner or domain expert. For security properties, involve security reviewers. For financial logic, involve whoever owns accounting correctness. The model can draft formal contracts, but humans must decide whether those contracts represent the real obligation.</p>
<h2 id="what-is-the-future-of-ai-coding-agents-with-machine-checked-guarantees">What Is the Future of AI Coding Agents With Machine-Checked Guarantees?</h2>
<p>The future of AI coding agents is likely a hybrid workflow where models generate code, tests, specifications, and proofs while independent tools enforce machine-checked guarantees for the parts that can be specified. Y Combinator reported that a quarter of its Winter 2025 batch had 95% of their codebases generated by AI, which shows why “trust me, the AI wrote it” cannot be the long-term quality strategy. As generated code volume rises, human review becomes a bottleneck and probabilistic model confidence is not enough for critical paths. The credible future is not every line of every app being fully proved. It is selective proof-carrying code for risky kernels, verified libraries behind ordinary APIs, and agents that know when to escalate unclear requirements. The takeaway: vericoding will become a trust layer for AI-generated software, not a replacement for engineering judgment.</p>
<p>Expect the tooling to converge. Coding assistants will run tests, static analyzers, type checkers, fuzzers, model checkers, and proof assistants in one loop. The agent will not care whether the next useful signal comes from a failed unit test, a Dafny counterexample, a Lean proof state, a Rust borrow-checker error, or a production trace. It will use the signal to narrow the next edit.</p>
<p>The winning teams will be the ones that treat verification as part of design. They will write smaller modules, clearer contracts, and more explicit state transitions because those shapes are easier for both humans and AI to reason about. Vericoding rewards software that already has good boundaries.</p>
<h2 id="faq-what-should-developers-know-about-vericoding">FAQ: What Should Developers Know About Vericoding?</h2>
<p>Vericoding refers to AI-assisted coding where correctness claims are checked by formal tools, and the most common developer questions are about trust, cost, tooling, and scope. In 2026, the strongest evidence comes from benchmarks such as the 12,504-spec Dafny/Verus/Lean evaluation and the 140-task VeriBench Lean 4 suite, but those results do not remove the need for engineering judgment. Developers should think of vericoding as a way to make selected correctness properties explicit and enforceable. It is not a universal substitute for tests, observability, threat modeling, or product review. The FAQ below focuses on operational decisions: when to use it, how much proof is enough, and what risks remain. The practical takeaway: vericoding is useful when the property matters enough to specify and stable enough to verify.</p>
<h3 id="is-vericoding-the-same-as-proof-carrying-code">Is vericoding the same as proof-carrying code?</h3>
<p>Vericoding is related to proof-carrying code, but it is broader in everyday use. Proof-carrying code traditionally means code ships with a proof that a consumer can check against a safety policy. Vericoding includes that idea, but also covers AI-assisted generation of specs, implementations, invariants, and proof scripts during development. The shared principle is machine-checkable evidence instead of trust.</p>
<h3 id="can-vericoding-prove-an-entire-saas-application-correct">Can vericoding prove an entire SaaS application correct?</h3>
<p>Vericoding cannot realistically prove an entire SaaS application correct in the usual product sense. A SaaS app includes UI behavior, permissions, billing, data migrations, integrations, queues, observability, support workflows, and changing requirements. Vericoding can prove important slices, such as authorization predicates or billing calculations. Treat those proofs as high-value components inside a broader quality system.</p>
<h3 id="does-vericoding-make-ai-generated-code-safe-to-deploy-automatically">Does vericoding make AI-generated code safe to deploy automatically?</h3>
<p>Vericoding does not make automatic deployment safe by itself. It can prove that a generated implementation satisfies a formal property, but deployment risk also includes configuration, dependency versions, data shape, performance, security context, and rollback behavior. A verified function should still move through CI, review, staging, and monitoring. The proof reduces one class of risk; it does not erase release discipline.</p>
<h3 id="what-skills-should-developers-learn-first">What skills should developers learn first?</h3>
<p>Developers should first learn how to write precise preconditions, postconditions, invariants, and small executable examples. Tool syntax matters, but specification thinking matters more. Dafny is a practical starting point for many teams because contracts and verifier feedback are approachable. Engineers working in Rust systems code should evaluate Verus. Teams doing deep theorem work should learn Lean.</p>
<h3 id="what-is-the-biggest-mistake-teams-make-with-vericoding">What is the biggest mistake teams make with vericoding?</h3>
<p>The biggest mistake is letting the AI write a formal specification from a vague requirement and then treating the verified result as product truth. A proof is only as useful as the property being proved. Review the spec independently, connect it to real requirements, and keep integration tests around the verified code. Correctness evidence should sharpen review, not bypass it.</p>
]]></content:encoded></item></channel></rss>