Best-of-N Coding Agents With LLM-as-a-Verifier in Oh My Pi
Best-of-N coding agents run the same task N times in isolated workspaces, then use an LLM-as-a-verifier to score every full trajectory and select the winning patch. In Oh My Pi, the omp-best-of plugin does this with /best-of --n 5 --apply "Fix the failing authentication test", defaulting to 3 candidates and selection-only mode. That is the short answer. The rest of this guide is the part that decides whether the technique saves you money or burns it: what selection can and cannot fix, why the maintainers’ own benchmark numbers are much smaller than the headline claims circulating about them, and how to choose N without paying for a fourth opinion nobody needed. ...