>
eval_set: refund_qa · 240 examples
metric: rubric · pass-rate
OPTIMIZING PROMPT...
. . ✶ . . . ✶ ✶ ✶ . . ✶ ✶ ✶ ✶ . . ✶ ✶ ✶ . . . ✶ . .
v3 -
v5 -
v7 -
Latest Writing
Thoughts on software development, AI, and building systems
Closing the Loop on Harness Engineering
My exploration of harness engineering, and the primitives that allow the agents to do work on loop even when designing harnesses
Building a Harness: How I Got Agents to Verify Their Own Code
I wanted to see how far I could push autonomous code generation. Not by writing better prompts, but by building a system where agents could implement, verify, and fix their own work without me watching. A DOCX editor built from behavioral specs and pixel diffs was the testbed.
Prompt Optimization Shouldn't Require Rewriting Your App
How come we have advanved agent tracing products with minimal setup, but prompt optimization itself needing rewriting your product in new frameworks in order to work. My attempt on showing that it does not need to be the case