The judge

The judge checks a set of guides the way a player would meet them. It answers three questions about a leveling chain:

• Can a player actually do every line as written, with nothing given for free?
• Where does the route waste time: quests walked past, trips back to a place the player just left, long detours to a trainer?
• Did a change make things worse than v1 (the shipped guides) somewhere?

It lives in the forge-judge crate (rust/crates/forge-judge) and runs as forge judge. forge build runs it as its last stage (report) on every written start. See Polestar Forge for the whole pipeline, The writer for how the guides it judges are written, and The time model for the simulator that supplies the levels and hours.

All hours in this page are model hours: time as the simulator prices it, not a stopwatch on a real character.

The pieces#

The library builds without the file system too (--no-default-features, for the browser planner); the native feature adds the folder readers and the CLI.

Engine replay#

engine is a line-by-line copy of the addon's Core/Engine.lua rules for which step is current and when a goal is done:

• only (class, race, faction; !x excludes) and when decide which steps apply.
• Each verb has its done rule. optional and note goals never hold a step.
• Unavailable quests are marked (closed, prerequisite never coming, level gates), along steps are passed over and listed under later steps, skipto jumps.
• When a guide ends, next hands over: the one option, else the recommended one.

Replay plays a character through the guides against a lenient stub world: any quest can be taken, objectives finish when acted on, items come from nowhere. Each loop it completes the first open goal that is not optional. A goal that stays open after the act is ticked if it is manual; otherwise the play is stuck there, and either stops or skips the step (OnStuck::Stop / Skip). The clock runs like the addon's test stub (0.6 s per act), so goals finish in the same order as in a real Lua play.

The replay finds guides the addon cannot play at all (stuck goals, goals with no location). Because the world is lenient, it does not find guides that are merely wrong: that is what the honest playtest is for.

There is no command for the replay by itself. It runs inside the seed tests, and its parity test against the real addon is tests/engine_parity.rs.

Honest playtest#

The honest playtest plays the compiled guides from their lines and the game's data, with nothing free. The character gains a quest, an item, a flight path or a place only through a line that gives it. Each rule a line breaks is a finding. The line is then played as if it had worked, so one cause gives one finding, not a cascade.

Who plays#

A chain starts at each guide that no next leads into and follows each guide's first next option without a when. Each chain is played by a set of personas (one race, faction and class): every race the first guide is for, each class an only in the guide names, and one class no only names. With --class you pick the classes instead.

The rules#

Default mode runs R1-R10. It merges equal findings across personas and chains, plays each start's chain through the shared guides, and lets buy lines cover buy steps. It also adds a far-walk check to R1: a plain goto into another zone more than 4,000 yd away is a walk across zones, not a way in.

Reading a finding#

<rule> <severity> <file>:<line> (<race> <class>) step <n> "<step title>": <reason>
  repair: <what to change>

That is the rule, severity, file and line, the persona, the step number and title, why the line fails, and a suggested fix. With --json, each finding also carries every persona that met it and, for R1, where the player stood, the flight paths they knew and where they were bound. The writer's travel repair reads those fields (see The writer).

The walk, passed by and avoidable returns#

The walk lists the chain's goals in the order a player meets them. The chain is the race's first leveling guide, then each next. Each goal gets a level and a time from the simulator's evaluator (levels::SimLevels), using the sim folder's world and params.json. --levels FILE replays the levels from a saved judge dump instead. With neither, the walk is skipped.

forge judge report, forge judge check and forge build's report price the walk as the routes stage prices an order (SimLevels::route): the average route profile on the sim folder's walk store, so a quest with no kill or escort work is taken without first grinding to its level, items that drop from more than 20 kinds of mob come from the bags, and every placed line with no quest (a goto, getfp, sethome, a vendor) is traveled to, so a written detour costs its travel. The judge's own walk (forge judge) and the JS-parity tests keep the gate's model (SimLevels::new).

Guide-text metrics#

guidetext::score reads a chain as a player follows it and gives the numbers the old quality gate tracked. The key names are kept exactly, so old and new results can be compared:

It needs the simulator's run and its exported routes as input, so it has no command of its own. It runs in the seeded-defect test and in tests/guidetext_parity.rs.

Seed tests#

The seed tests are checks written from player reports against v1's guides: "the arrow sent me onto the hill above the mine", "no guide sends you to the refugees for Stop the Spread". Each pins the fix: a quest taken on the walk that passes its giver, a turn-in before a follow-up, a cave mouth before a target inside a cave, and so on. They are the old Node tests (tests/guides-reported.test.js, tests/guides-caves.test.js) ported to Rust, with the same names. That way a written guide tree reports pass or fail for each test, exactly as the Node run would.

Tests that play a guide run on the engine replay above. Each test compiles the guides it needs with the game's data, and records failed assertions instead of stopping. One failure does not hide the others.

A failing seed test on a written tree means the route broke a fix a player asked for, or moved it to a place the test does not expect. Read the test's name and its failure lines, then the guide lines it names.

forge judge seed-tests --guides out/guides/Orc --root out/.forge/root --lint
forge judge seed-tests --guides guides --filter caves

--guides is a folder holding forever/. --data defaults to <root>/data/forever. --filter TEXT runs only tests whose name contains TEXT. --lint adds the compiler's error and warning counts and the first errors. --json prints { tests: [...], lint }. The command exits 1 when a test fails.

Running the judge#

forge judge --help
forge judge report --help
forge judge check --help
forge judge seed-tests --help
forge judge route-status --help
forge judge feedback --help

One pack: forge judge#

forge judge guides --faction Horde --race Orc --class Warrior
forge judge out/guides/Orc --faction Horde --race Orc --root out/.forge/root --sim out/.forge/sim --to 30 --json

The first argument is a guides folder (holding forever/). --faction and --race are required. Other flags:

Text output is every finding (two lines each), then a count per rule and severity, then one walk line:

walk: <guides> guides, <goals> goals, level <end level>, <hours> h; passed by <n> (<never> never, <later> later); avoidable returns <n>, left behind <n>, same mob <n> (<ms> ms)

v1 against the written guides: forge judge report#

forge judge report --routes out/routes --guides out/guides --root out/.forge/root --sim out/.forge/sim --out out/review
forge judge report --route out/routes/Orc.json --guides out/guides --root out/.forge/root --sim out/.forge/sim --json

forge build writes the same two files to the top of its --out folder. After a build, <out>/.forge/root and <out>/.forge/sim are a consistent root and sim folder for these commands.

The route gate: forge judge check#

forge judge check out/guides
forge judge check out/guides --start Orc --json

The fast check of a build's written guides, independent of the search and the writer. It reads the build's files beside the guides folder (../routes, ../.forge/root, ../.forge/sim, ../fixes.json) and the deliberate orders in the route fixes (--route-fixes, else $POLESTAR_ROUTE_FIXES, else this checkout's data/route-fixes.json). It prints one line per broken rule:

<start> <file>:<line> <rule> blocker|warn: <message> [<classes>]

The dashboard's route status: forge judge route-status#

forge judge route-status --report out/report.json --out data/local/dashboard/routes.json

Reads a judge report and writes, per start, the base and new hours to 30 and 60, passed-by quests and avoidable returns to 60, honest fails, the seed tests and lint, and the routes stage's skips (<start>.skips.json beside the route: those for missing information, and all). The file is written whole and then renamed, so the dashboard never reads half of it. Without --out it prints to stdout.

Player reports into the route: forge judge feedback#

forge judge feedback --out out/route-fixes.json --report out/feedback.json
forge judge feedback --out out/route-fixes.json --offline

Turns data/local/research/RouteFeedback.json (insert, drop and constraint inputs mined from player reports) and the GitHub issues labeled guide into route-fixes override lines: include, skip and pairs, each with its source (F01, #647) and whether the sim already honors it. The issues come from gh api and are cached in data/local/research/guide-issues.json; --offline reads only the cache, so the output is the same for the same inputs. An issue can carry a fenced route-override JSON block, read as given. Specs the reader does not understand are listed as manual, and issues no feedback row cites as untriaged; neither is guessed. --out is data/route-fixes.json with those overrides added; point the sim at it with POLESTAR_ROUTE_FIXES.

Reading report.md and report.json#

report.md begins with a Sources line: the current guides' folder, the data folder and when it was built, the sim folder, the root, and each faction world's dataHash (the data it was built from). If two runs differ, compare this line first.

Then one table with four rows per start: guides and written, each cut at 30 and at 60. The to column shows 30 and 61; 61 means "everything below 61", the whole 1-60 walk.

What the judge cannot see#

We test the checks by seeding one defect at a time into the shipped Orc and Human chains (tests/seeded.rs). About 7 in 10 seeded defects are caught, almost always by the rule written for them. These still pass every check:

• back-and-forth inside one zone (R1 only looks at zone and continent changes): a talk to an escort NPC after the escort, a hearth followed by a walk back;
• a trainer or flight master that is not the nearest one on the route;
• required kills of rare or wandering mobs (only lint knows elites);
• a dropped class quest chain (the passed-by list only covers quests any class can do);
• a tip copied from another step (R7 reads only numbers);
• a repeated buy, collect or train class;
• the cost of deliberate deaths (no check models resurrection sickness).

A player review is still the final check for these.

When something goes wrong#

Tests#

cd rust && cargo test --release -p forge-judge -- --test-threads=2

Parity tests compare against saved JavaScript references and skip, with a note, when their inputs are absent. The seeded-defect test accepts POLESTAR_SEEDED_N=<sites per class> and POLESTAR_SEEDED_ONLY=<class,...>.