Illustration benchmark
Runs are read from runs/. Labels save to runs/<run>/review.json.
Summary by approach and model
No runs yet. Run pnpm pipeline --profile <id> --approach B --model pro.
Acceptance rate = accepted / reviewed scene images. Attempts per accepted = all scene calls (including refusals and errors) / accepted. Cost per accepted = total run cost including the character sheet / accepted scenes. Reused sheets cost $0 in the run that reuses them.
Refusals by step (all runs)
| Step | Calls | Safety refusals | Refusal rate | Other errors |
|---|---|---|---|---|
| describe-photo | 0 | 0 | – | 0 |
| character-sheet | 0 | 0 | – | 0 |
| describe-sheet | 0 | 0 | – | 0 |
| describe-sheet-minimal | 0 | 0 | – | 0 |
| scene | 0 | 0 | – | 0 |
| repair | 0 | 0 | – | 0 |
Fallbacks (per child): photo description unavailable 0×; sheet description needed the minimal retry 0×; scenes used the photo description 0×. describe-sheet-minimal only runs after a describe-sheet refusal.
Runs
| Run | Approach | Model | Profile | Status | Descriptions | Images | Reviewed | Cost | Created |
|---|