The blind tests
Five tasks from real use, with names and project details taken out: a product caption, book typesetting, selling a car, moving an app off Vercel and tidying a Downloads folder. Each test rewrote them in several ways and had a judge score the results without knowing which was which. deeper 1.1 was judged best on 4 of 5.
Why there's a 1.1
deeper 1.0 made prompts more expert, but it also grew them. In a sample of 40 of my own past runs, 33 asked for at least one deliverable I hadn't. Version 1.1 adds a rule: the prompt asks for the same deliverables as the task, and anything else goes on one line, "Worth knowing, not in this task", for you to choose.
The first wording of that rule kept scope tight but thinned the methods and checks. The final wording keeps every check that tests what you asked for, however much work it takes, and only cuts things you'd be left to keep or maintain.
How the current test ran
- The same five tasks were rewritten four ways: with no skill, with deeper 1.0, with the first 1.1 draft and with the final 1.1.
- A script shuffled the four prompts for each task into A to D and kept the answer key in a separate file.
- One judge on a different model scored all of them in a single pass, so its quirks hit every version equally. It also listed anything each prompt added beyond the task.
The writers ran on Claude Opus 5.5 and the judge on Claude Sonnet 5.5. Every prompt, key and score is in examples/raw/v1.1.
Averages
| Out of 5 | No skill | deeper 1.0 | First 1.1 draft | deeper 1.1 |
|---|---|---|---|---|
| Expert method | 2.6 | 4.6 | 4.2 | 5.0 |
| Starts from raw material | 1.8 | 4.8 | 5.0 | 5.0 |
| Check outside the model, with a pass mark | 1.0 | 3.8 | 3.4 | 4.8 |
| Kept to the original scope | 2.2 | 2.8 | 4.4 | 4.0 |
| Extra deliverables per task | 3.2 | 3.6 | 0.6 | 1.8 |
Task by task
T1 Honey caption
Write an Instagram caption for a jar of borage honey.
| Version | Expert method | Starts from | Check outside | Kept to | Added |
|---|---|---|---|---|---|
| No skill | 3 | 2 | 1 | 3 | two alternative opening lines |
| deeper 1.0 | 4 | 5 | 3 | 2 | second A/B variant; week-later A/B test process; source list for each claim |
| First 1.1 draft | 4 | 5 | 3 | 4 | list of facts with sources |
| deeper 1.1 | 5 | 5 | 4 | 3 | post-publication performance comparison process; pollen analysis check; Insights-based hashtag research |
Best: First 1.1 draft. First 1.1 draft stays closest to the one-caption scope while demanding primary material, a no-invention rule and a test with a real buyer; deeper 1.1 has stronger external checks but adds a post-publication measurement loop.
T2 Book typesetting
Evaluate the typesetting of my book manuscript and improve it. The inner margins look exceedingly large.
| Version | Expert method | Starts from | Check outside | Kept to | Added |
|---|---|---|---|---|---|
| No skill | 3 | 2 | 1 | 3 | prioritised change list; tool-specific setting instructions |
| deeper 1.0 | 5 | 5 | 4 | 4 | preflight report; measurement report with numbers |
| First 1.1 draft | 4 | 5 | 3 | 5 | Nothing |
| deeper 1.1 | 5 | 5 | 5 | 4 | before/after measurement table; optional designer or prepress review |
Best: deeper 1.1. deeper 1.1 pairs spread-based judging and measured targets with printer-profile preflight, a printed proof and a pass/fail bar, adding only a before/after table.
T3 Selling a car
I need to sell my car, a VW Polo, and buy a new one, perhaps a Golf.
| Version | Expert method | Starts from | Check outside | Kept to | Added |
|---|---|---|---|---|---|
| No skill | 2 | 2 | 1 | 2 | viewing checklist; scam-avoidance guide; alternative car list; negotiation guidance; generic guide |
| deeper 1.0 | 5 | 5 | 4 | 3 | valuation with sources; sale plan; TCO table; shortlist; sale preparation pack |
| First 1.1 draft | 4 | 5 | 3 | 4 | cost spreadsheet |
| deeper 1.1 | 5 | 5 | 5 | 4 | TCO spreadsheet; standard written sale contracts |
Best: deeper 1.1. deeper 1.1 uses binding instant-buy offers and DAT/Schwacke valuations as external anchors, requires an independent inspection with clear fail conditions, and still delivers only the sale and the purchase.
T4 Leaving Vercel
Can we move our Next.js app off Vercel to Cloudflare?
| Version | Expert method | Starts from | Check outside | Kept to | Added |
|---|---|---|---|---|---|
| No skill | 3 | 2 | 1 | 2 | feature-by-feature mapping; cost and effort comparison; CI/CD guidance; step-by-step plan |
| deeper 1.0 | 5 | 5 | 4 | 2 | cost comparison; load test; parallel-run cut-over plan; rollback plan; approval gate |
| First 1.1 draft | 5 | 5 | 4 | 4 | migration plan |
| deeper 1.1 | 5 | 5 | 5 | 5 | Nothing |
Best: deeper 1.1. deeper 1.1 proves feasibility by actually building and running it on Workers with route-by-route diffs and bundle-size checks, and delivers exactly the yes/no/not-yet answer the question asked for.
T5 Tidy Downloads
Tidy my Downloads folder.
| Version | Expert method | Starts from | Check outside | Kept to | Added |
|---|---|---|---|---|---|
| No skill | 2 | 1 | 1 | 1 | maintenance system; folder structure; optional sorting script; how-to guide |
| deeper 1.0 | 4 | 4 | 4 | 3 | retention rule; full move log; plan for approval |
| First 1.1 draft | 4 | 5 | 4 | 5 | Nothing |
| deeper 1.1 | 5 | 5 | 5 | 4 | before/after size and count; spot-check by the user |
Best: deeper 1.1. deeper 1.1 uses content hashes, Finder/Trash-only rules, a source-of-download check and a count that must reconcile, plus a user spot-check, all aimed at tidying the one folder.
The first test
Before 1.1, deeper 1.0 was tested against a plain rewrite alone, by a judge who wasn't asked to count extra deliverables. deeper won 5 of 5, and scope was already its weak spot. Its numbers, out of 5: expert method 2.6 to 4.8, raw material 1.8 to 4.8, outside check 1.2 to 4.8, scope 4.2 to 4.0. The files are in examples/raw.
Limits
Five tasks is a small sample, and each round is one run. Scores for the same prompts moved by up to 0.6 between the two judging passes, which is why the current test judged every version together. The judge is a model, though a different one from the writers, and the writers knew they were in a test. A person judging the shuffled prompts blind would be a stronger check.
The skill's own rule applies here too: a model grading model output doesn't count as proof. Treat these numbers as a reason to try it, not as a verdict.