PromptWake vs Braintrust

you need to measure and improve the quality of an AI system you are building, with datasets, scoring and experiments.

you need to reconstruct how AI-written code came to exist, months after the fact.

Braintrust is an evaluation platform: define datasets, write scorers, run experiments, compare versions, bring humans in where the automatic score is not enough. It exists because 'the demo looked fine' does not survive contact with production, and it is aimed at teams whose product is the AI.

We do not evaluate anything. We record. If you asked us whether last Tuesday's Cursor session produced good code, we could show you exactly what was asked, what was answered and which lines changed — and then you would have to decide, because we have no opinion.

Evaluation is forward-looking and sampled. You choose cases that represent the behaviour you care about, score them, and use the result to make the next version better. That is the correct shape for improving a system, and it is structurally unable to answer a question about one specific past event — which is what a provenance question always is.

An evaluation tells you the system is right 94% of the time. An incident is about the one change that was in the other 6%, and evaluation has nothing to say about that particular one.

The reverse is also true, and worth stating so the comparison stays honest: our record cannot tell you whether your AI feature is improving. It is a ledger of what happened, not a measurement of whether what happened was good.

Evaluation becomes urgent when you are about to ship an AI feature to customers. Provenance becomes urgent at four specific moments: a reviewer facing a diff with no statement of intent, an incident where the question is why a change looked correct at the time, a new engineer inheriting a codebase whose reasoning was never written down, and a customer's security questionnaire asking how AI-generated code is retained and reviewed.

The second list arrives on someone else's schedule, and the record has to already exist when it does. That is the only real argument for starting now rather than later — not that provenance matters more than quality, but that it cannot be reconstructed retroactively once the transcripts are deleted.