Conceptual checklist for similarity cutoffs: Constraint query, then Similarity score, then Threshold decision. Check whether a cutoff hides a required constraint.

Agent Memory Cutoffs Can Hide Constraints: Test Before You Tune

Your memory pipeline ends with a score gate: keep rows at or above 0.7 cosine similarity, drop the rest. You tuned that number on topical questions and it looked fine. Then someone asks “May I deploy to production on Friday?” and the agent retrieves nothing — not the wrong thing, nothing — because the one stored sentence that should have stopped the deploy scored 0.660. That is the measurement in this article, scoped to what was actually run. On a 12-fact synthetic fixture with two local Ollama embedders, the prohibition “Deployment to production on Fridays is prohibited for all teams.” scored 0.660 against “May I deploy to production on Friday?” under all-minilm:latest. The two descriptive facts in the same fixture scored 0.764 and 0.773 for their own questions. A 0.7 gate therefore keeps the descriptions and deletes the rule, and the ordering reverses on the second embedder — the same prohibition scores 0.949 under snowflake-arctic-embed:33m. The safe cutoff is a property of the encoder, not of your application. ...

November 14, 2026 · 11 min · baeseokjae
Conceptual checklist for context compaction: Original transcript, then Compressed summary, then Fact retention probes. Compare retained facts, not just summary fluency.

Agent Context Compaction: What Gets Lost—and How to Test It

An agent works for two hours. The harness compacts the transcript, the run continues, and somewhere in the next twenty turns it re-runs a migration it has already run, asks the user for a value the user typed an hour ago, or refactors a file it was explicitly told not to touch. Nothing in the run log marks the turn where that fact disappeared. The summary reads fluently, the task still finishes, and the only trace of the loss is work the agent repeats. ...

November 2, 2026 · 11 min · baeseokjae
Google Agent Quality Flywheel Skill Guide 2026

Google Agent Quality Flywheel Skill Guide 2026: Testing and Grading Agents

A Google agent quality flywheel skill is a reusable workflow that turns agent traces into graded eval cases, runs them locally and in CI, then feeds production failures back into the test suite. For coding agents, the goal is simple: stop trusting “done” and start checking behavior, tool use, and real state changes. What is a Google agent quality flywheel skill? The phrase “Google agent quality flywheel skill” is not an official Google product name. I use it as a practical pattern for teams building agents with Google Agent Development Kit (ADK), Vertex AI Gen AI Evaluation, and a coding agent that can maintain its own eval suite. ...

April 13, 2026 · 15 min · baeseokjae