How do you measure puzzle level difficulty in a mobile game?

Why is difficulty measured against perfect play misleading?

Screwsmith is a screw-removal puzzle: you tap a reachable screw and it goes either to an open crate of its colour or onto the holding rail (five slots on 161 of the 200 levels, four on 39). Three screws fill and seal a crate, and the next one comes off the queue. A part losing screws settles and uncovers the layer beneath, or comes away entirely. The level is lost when the rail is full and the arriving screw has nowhere to go.

The obvious metric is the minimum number of rail slots needed to finish. Our solver computes it; in the code 2 means a warm-up and 5 means no margin at all. The trouble is that almost every level survives perfect play. That number says nothing about a player who cannot see the crate queue three moves ahead.

So SolveReport carries two numbers: MinRailSlotsRequired for perfect play, and HumanFailRate for human play, plus AveragePeakRail, the worst rail occupancy averaged over trials. The curve is tuned against the second; the first tells a broken level from a merely hard one.

What did separating the rules engine from Unity buy us?

Game logic lives in its own assembly, Screwsmith.Core, whose asmdef sets noEngineReferences: true. GameSim has no UnityEngine type; even positions use our own Vec3. LevelData is engine-free too, so the solver can evaluate thousands of candidates without opening a scene.

Randomness is xorshift32, not System.Random: _state ^= _state << 13; _state ^= _state >> 17; _state ^= _state << 5;. Identical output on every platform and Unity version means a level number is the same board for every player. The crate queue is shuffled from the level seed with it.

The split paid off three ways:

• Level generation and measurement run headless from the command line with -batchmode, no editor window.
• The solver’s depth-first search backtracks with the game’s own undo, which means the solver is only as trustworthy as undo.
• On the phone the Daily Bench is built off the main thread with Task.Run. That is safe because Core never touches the engine; the first version did the work inline and the app froze between the button tap and the board appearing.

How does the solver measure a level, step by step?

LevelSolver.Solve runs in three stages, cheapest first.

First it looks for a permanently buried screw: one whose blocking part can neither settle into a later pose nor detach. That is a design bug, not a hard level, and since it is cheap to detect it runs before anything else.

Then it measures human play. It runs 24 trials, each with its own seed (level.Seed + trial * 7919 + 13). The strategy is deliberately shallow: among reachable, legal screws prefer the ones some crate will accept, and pick at random when several qualify. The comment in the code reads: most players do this much; almost nobody plans further than that. A single winning trial counts as proof the level can be finished, and the expensive search is skipped.

Only when all 24 trials lose does a depth-first search run, from one rail slot upward, capped at 120,000 nodes. Hitting the cap means “not proven”, never “impossible”; treating the two alike silently threw away good hard levels, so SearchExhausted is its own flag.

We also had a state-key bug. The key stored only whether each screw was removed, so the first tap on a rusted screw hashed like the state before it and the search abandoned the branch. No rusty screw reached its second tap, and every rusty level a human trial lost was dropped at bake time as unsolvable. Each screw now writes three characters: removed, taps taken, colour known.

How is the difficulty score computed, and which candidate wins?

DifficultyScore is an integer from 1 to 10. The formula is short:

• a base of 1
• HumanFailRate * 11, the dominant term
• max(0, AveragePeakRail - 1) * 1.6, so two levels that never lose still rank by rail pressure
• +0.6 when the move count exceeds 45
• +1.5 when there is only one solution

An unsolvable level scores 10 outright. Because a rusted screw takes two taps, the move estimate counts taps, not screws; otherwise the pacing curve read every rusty level as shorter than it plays.

GenerateTuned tries 12 board shapes per target band: the band’s own, its neighbours and several screw-density variants. The bake tool uses 16 seeds per shape, so up to 192 candidates per level. The seed budget used to be divided across shapes, so adding a shape silently shrank every other shape’s search; seeds are now counted per shape.

At equal difficulty, the candidate that pushes the rail higher wins. As the code puts it, an easy level must be easy because the rescue always comes, not because the rail sits empty; an empty rail is an invisible game. A candidate on target with an average rail peak of at least 3 ends the search, and the winner gets the full proof search before it ships.

The curve: 35 gentle levels, a wall at 36, then a rising floor

TargetFor maps each level number to a target band: 1–5 band 1, 6–15 band 2, 16–25 band 3, 26–35 band 4, and level 36 a deliberate wall at band 8. Then a six-level rhythm rides on a floor: -1, -1, 0, +2, +2, 0. The easy levels earn currency back; the spikes spend it.

At first the rhythm was a bare cycle (3,3,4,7,7,4) repeated to level 200, so level 197 asked what level 37 had. Against a competent player the win rate across the four fifties was 94%, 94%, 88%, 88%: a fifty-level game played four times. The fix was a rising floor, 4 + (level - 37) * 4 / 163: 4 at level 37, 8 at 200.

The bands were not a ladder either: band 8 was a 12-panel board and band 7 a 20-panel one. Now layers grow first (one to four), then colours (three to six), and the rail tightens last (five slots to four, three in the top band).

The baked levels.json records the outcome: a 0.8% average human loss rate over levels 1–35, 37.5% at level 36 (9 of 24 trials), 34% over 101–200. A player who keeps losing also gets an invisible assist: after 3 losses the rarest colour is recoloured to the most common. It runs on a copy; the first version edited the cached baked level, so every retry folded away another colour.

Why did readability become the ceiling on difficulty?

Half of the readability gate is a question about people, and only a play test answers it. The other half is pixel facts, and ReadabilityAudit keeps those from ever reaching a play test.

The touch target must be at least 8.5% of screen width, roughly 4.7 mm: below the ~9 mm Apple and Google recommend, because a screw is round and sits among neighbours, and 8.5% is the floor below which mis-taps rise sharply. The audit measures the collider, not the visible head, since taps resolve by raycast. It renders the board to a RenderTexture at real device resolution and measures at every state of a playthrough, because boards get crowded as parts settle.

It caught two real defects. Level 178 had shipped three screws at 0.084 of screen width against a 0.085 gate. On level 153 two top-face screws sat exactly on top of each other, one of them untappable. The first changed the design: 24-panel boards do not fit the screen legibly, so the top two bands stop growing and take rail slots away instead.

For colour it uses CIE76 ΔE: at least 11 for normal vision, at least 9 after a Brettel/Viénot red-green colour-blindness simulation. Blocked screws are drawn dimmed, so the palette must separate when dimmed too. At six colours the closest pair holds ΔE 19.7 dimmed and 29.4 bright; a seventh broke the dimmed floor. Six colours is a readability ceiling, not a difficulty one.

Why does the undo audit have to prove it can fail?

Undo is the screwdriver tool, and its contract is strict: the board, every part’s pose included, returns exactly to its pre-move state. The solver backtracks with the same undo, so a bug here corrupts measurements too.

It was broken for a while. Undo restored crates by position, assuming the crates a move pulled were the last entries of the active list. That fails when a crate pulled during a move is filled by screws draining off the rail and sealed later in the same move: a pre-existing crate was deleted, a nonexistent one took its place, and the id counter walked back over live ids. Nobody noticed, because nothing compared the board before a move with the board after undoing it. The fix records pulled crate ids and the slot each sealed crate vacated.

UndoAudit.Run is that comparison. On each of the 200 levels it plays up to 60 random legal moves, arming the wrench 15% of the time to cover bolts. Each move gets a snapshot of move count, screws, parts, crates and rail, then tap, undo, compare.

We wanted to see it fail before trusting it. GameSim.UseLegacyCrateUndo restores the old broken restore, and corruptSnapshot drops crate ids from the snapshot. An audit that stays clean against a broken sim is not looking at enough state.

How does a serverless Daily Bench give everyone the same board?

Beyond the 200 baked levels there is one Daily Bench a day, the same machine for everyone, with no server, account or download: the board is generated from the date. The bench number is days since the game’s epoch, 26 August 2026 (UTC), plus one, and the machine rotates through all 16 by that number.

It is generated rather than baked because a baked set would have to be reissued daily. Generation is deterministic, so the same date yields the same screws on every device, and “23 moves” becomes a number worth sharing.

The risk: an unlucky seed ships an impossible board to everyone at once. So the flow is:

• The target band is fixed at 5: a real puzzle, but not the level-36 wall.
• 12 seeds are tried and the first one proven solvable is used.
• If none holds, it steps down through bands 3, 2 and 1, six seeds each, still proving every candidate.
• Only the very last resort is an unproven board in the easiest shape.

Past level 200 the game replays the second half of the baked set instead of generating live: a 128-level loop, two passes through the 16 machines at four levels each. Proving a board takes seconds on a phone’s main thread, and iOS kills an app that is slow to launch.

If you are building a similar puzzle game: a checklist

Rules from Screwsmith that carry over to other kinds of puzzles:

• Keep the rules engine in an engine-free assembly the compiler enforces (noEngineReferences).
• Use your own platform-independent RNG; a level number must be the same board for everyone.
• Measure two numbers: perfect play catches broken levels, a shallow human simulation measures difficulty. Tune against the second.
• A search that hits its cap has not proven anything impossible.
• Put everything that makes two states play differently into the state key; one missing field can erase a mechanic.
• Check that bands form a real ladder, and that win rate keeps falling slice by slice.
• Pay for generation at build time, and store baked levels with their measurements.
• Measure touch targets, colour distance and HUD overlap automatically, before any play test.
• Write an undo audit that compares before-move and after-undo boards, and watch it fail against a broken build.
• Prove anything that hands everyone the same board, like a daily level, before it ships.

At AMG Software Studio we build data-driven, measurable mobile puzzle games like this in Unity.