LLM-as-Judge

What Is OctopusGarden? — The Open-Source Dark Factory OctopusGarden is an open-source autonomous software development system that operates as a “dark factory” — it takes specifications written in markdown and scenarios defined in YAML, then builds the software without human intervention. Released by foundatron on GitHub under the MIT License, OctopusGarden represents one of the first fully open-source implementations of what Dan Shapiro calls Level 5 AI coding maturity: a system where humans define intent and review outcomes, but never touch the code during generation. The system is built in Go, requires Docker, and supports both Anthropic and OpenAI APIs. ...

Agent CI/CD in 2026 requires five evaluation gates that don’t exist in traditional pipelines: golden dataset offline eval, regression blocks, cost gates, shadow evaluation against production traces, and canary rollout with auto-rollback. If you’re shipping agent updates against only lint and unit tests, you’re shipping blind — 89% of production agent teams run observability but only 52% run evals, and that 37-point gap is where quality silently decays (LangChain State of Agent Engineering Survey, 2026). ...