Engineering log

Things we measured while building Rubicon, written up with the numbers in them.

Rubicon is a trivia game where you answer in your own words and an AI judge marks what you meant rather than how you spelled it. Building that turned up questions with no published answers, so we measured our own and wrote them down.

• We measured when an embedding can replace an LLM judge — Grading a typed answer with an LLM costs about as much as writing the question. We measured how much of that work a cheap embedding comparison can take over, on real player answers rather than a benchmark. 22 September 2026