The system behind this note is an Android app that watches a child's chat apps on the phone itself. It sends only suspicious slices of conversation to an LLM, and the LLM decides whether an adult is grooming the child. When the LLM decides that an adult is, the app tells the parent or locks that one chat. This note covers how the app's evaluation is set up to catch the case where the team has fit its own test data.
What is the anchor set in this eval design?
The eval design uses 4 splits. The rule covered here involves two of them, the anchor set and the dev set. The anchor set contains at least 50 trajectories. It is hash-committed and locked, and it is opened once per release. Three uses of it are explicitly forbidden:
- prompt iteration
- few-shot examples
- debugging
Why keep a held-out set out of prompt iteration, few-shot examples and debugging?
Each of these uses can carry information from evaluation data back into the system being evaluated. In prompt iteration, every change that is kept because it moved a number becomes a small fit to the data behind that number. Few-shot examples are a more direct route, because they put eval content inside the prompt itself.
Debugging is the least obvious of the three. When an engineer reads a failed trajectory, they learn something from it. That lesson can shape the next prompt change even though no data was copied anywhere. Forbidding all three uses narrows the ways the anchor set can influence development. It does not prove that no leakage of any kind is possible.
What does hash-committing an eval set actually buy you?
In general, a hash commitment fixes the contents of a set at a point in time. In principle, anyone who holds the hash can later confirm that the set being scored is the same set that was committed. That makes it detectable if cases are swapped, added or removed after results are seen. The claim says only that the anchor set is hash-committed. It does not say who holds the hash or who can verify it.
The hash and the usage rules solve different problems. The hash shows that the set did not change. The usage rules address whether anyone learned from it.
What is the preregistered failure rule?
Anchor recall is compared with dev recall. If the two diverge by more than 10 percentage points, the dev set is declared overfit, and that declaration appears publicly in the release note. Because the rule is preregistered, the threshold and the consequence are both fixed before the comparison is made.
Why write the overfit declaration before you see the number?
A gap between a dev metric and a held-out metric is easy to explain away after it appears. The usual explanations are distribution shift, a handful of odd cases, or noise in a small set. Any of these can be true in a particular instance. The problem is that every one of them is available every time, and the person choosing among them has a stake in the outcome.
Preregistration removes that choice. The threshold is set while nobody knows which side of it the result will land on. Putting the consequence in a public release note, rather than in an internal document, gives the rule a visible cost. Everyone knows ahead of time the sentence they will have to publish if the gap is too wide.
How would you adopt this pattern in your own eval pipeline?
The suggestions below are adoption advice for your own pipeline. They do not describe the order in which this project did things.
- Reserve one set as an anchor that development work never touches.
- Hash-commit that set, ideally before tuning begins, so that later checks have a fixed reference.
- Write down the forbidden uses explicitly, and include debugging as well as tuning.
- Decide how often the set is opened, for example once per release.
- Choose the metric, the divergence threshold and the public consequence before you look at any anchor result.
Set the threshold according to how noisy a set of your size is. If it is too tight, ordinary sampling variation will trigger false declarations. If it is too loose, real overfitting will get through. Pick the threshold for your own data, and then hold to it.
What does this design not tell you?
The rule is a commitment, not a result. This note does not report how often the threshold has been crossed, what anchor or dev recall has been, how the 10-point figure was chosen, or what the other splits contain. Readers should not infer any of those details. The design states in advance what will be published if the gap exceeds the margin. Whether that commitment is kept in practice can only be judged from the release notes themselves.