FIELD NOTES
How we test a browser game, and the bugs that got through
A candid look at the test harness behind every Riftspinner release, the bugs it catches, and the ones it did not.
PLAY FREE IN YOUR BROWSERRiftspinner is a canvas game with no engine, no build framework and no QA department. It is written in plain JavaScript, with a small PHP backend, and published by a very small studio. Every release still goes through the same process before it reaches anyone: a battery of automated tests, a stress build, random-input fuzzing and a final check of the package that gets uploaded. This article describes that process honestly, including the places where it has failed us, because the failures are the useful part.
The basic harness
The tests use Playwright driving a headless Chromium browser, against either the file system or a small local web server. Each test is a script that plays part of the game and asserts something about the result. A regression run executes about thirty such scripts, covering things like: the tutorial can be completed; the leaderboard accepts and returns scores, including new content identifiers; every mission and daily task type can actually be finished; the game responds correctly to keyboard, mouse and a mocked gamepad; the offline cache is populated; and each recent feature has its own checks.
To let tests reach into the game, we make a copy of the site and append one line to the copy's main script that exposes the game's internals on a debug object attached to the page: the state object, the saved data, and functions such as the one that spawns an enemy. A test can then say "put a Mythical weapon on the floor next to the player" instead of playing for an hour to find one. The packaging step searches for that line and refuses to ship if it finds it, so the hook exists only in test copies.
The test copy is also adjusted in other ways that make results trustworthy. The animated background is pinned to its cheapest setting, because headless software rendering is slow and the game deliberately limits how much time a single frame may advance. A heavy background would make game time run slower than wall-clock time and make every timing-based test flaky. Server rate limits and host checks are relaxed in the copy only, so tests can post many scores quickly.
Stress builds and fuzzing
Most of the worst bugs in a game live in rare situations, and rare situations are hard to reach by playing. So we cheat: a stress build patches spawn chances and timers to "always", so that features that normally appear once in nine minutes appear constantly, and then we fuzz the result. Fuzzing means a script that presses random keys, moves and clicks the mouse at random, changes settings mid-run and wiggles a mock gamepad, while the harness listens for any uncaught page error or any report from the game's own fault logger. A forced-spawn fuzz run on four rigs, two gun and two melee, is part of the checklist for every new feature that can appear during play.
We also write bots, simple scripted players, for questions about feel. Is the rare treasure enemy catchable by a chasing player with every rig? A bot answers that faster than a person can. The catch is that a bot has to play sensibly first. If you conclude a feature is unfair before checking that the bot is not simply running into walls, you have measured the bot.
The last layer is edge-case probes, which deliberately do strange things: spawn something in a corner, spawn it twice, open a screen while it is already open, equip an item on a character with a maxed-out stat, pass the wrong type of argument to a function, hand the game a deliberately broken object. These are the cases a human tester would never think of and a player eventually will.
What the game does when a test fails in public
Because no test suite finds everything, the game has a safety net. The update, draw and audio steps of the main loop each run inside their own error handler, so a fault in one cannot take down the others. A fault logger de-duplicates errors and sends one report per distinct problem, with the build number, wave, health, browser, the stack trace and the last 25 events leading up to the crash. If errors repeat several times within a few seconds, a fatal screen appears instead of a frozen game. Those reports have been the single most useful source of fixes, because a stack trace names the exact file and line. They cannot see everything: a tab that crashes outright, a freeze or a page that fails to load sends nothing, so those only reach us if someone tells us what they saw.
Bugs that got through
These are real, and each one produced a permanent rule.
The reward that crashed on pickup. A mission reward spawned a power-up by calling a function with a third argument of the wrong type: the function expected a power-up object and was given a true flag. The reward appeared on the floor and looked fine. The moment a player walked over it, the pickup code threw an error on every frame until the fatal screen appeared. It hit real players three times before we found the cause. Our tests had checked that the reward spawned, and never that it could be collected. The rules: check argument types when calling an existing function, remove a pickup before applying it, wrap pickups in error handling, and test the whole chain, down to the pickup.
The ids the server did not know. When we added new melee rigs, we added them to the game and not to the server's list of valid rig identifiers, so a melee score would have been saved as the default rig. The rule: adding any content means searching for its id list in the client, the server and the documentation, then testing the round trip.
The health bonus that vanished. A rig's extra health was lost when the player equipped armour, because maximum health was recomputed from scratch without including every source. The rule: any recomputed stat must include all sources, and the test is "equip an item after spawning with a bonus".
The keyboard trap. Pressing Escape did nothing while a text box had focus. For a keyboard-only player that is a trap. The rule: Escape must always work, even inside inputs.
A boss shield that did nothing. The Rift Weaver's cocoon was meant to reduce damage taken to 20%, but the reduction was applied before a per-hit cap instead of after, so large hits ignored it. We found it by measuring the damage actually taken, not by checking that the cocoon appeared. The story is in the Weaver article.
Rare, but not rare enough. A new top-tier weapon passed every technical test and then turned out to drop about ten times too often. No script can tell you that a number feels wrong; see the drop odds article.
Words that went stale. Documentation drifts. Our FAQ kept telling players to clear their browser data to reset, after we had added a Reset button. And while writing the articles in this section we found that two guide pages still quoted old ability cooldowns, and that the Ash Market total had been left out of date when an upgrade was added. We have corrected both. The rule: whenever a feature changes, search every guide, FAQ entry and in-game briefing for statements it has just made false.
A page destroyed by a replace. A script that edited one of our legal pages used a slice with its indexes reversed, which returned an empty string, and replacing an empty string with new text inserts it at the very start of the page. A later edit then removed two sections. Since then, every scripted edit asserts that its search text appears exactly the expected number of times, and every build validates that each page begins with a doctype and has its sections.
The package check
Before anything is uploaded we validate the folder statically: every page starts with a doctype, every link and image path resolves, no test hook remains, no leftover or secret files are included, advertising code appears only on the pages meant to carry it, and every script passes a syntax check. Then we unzip the finished package into a clean location, serve it, and run a smoke test: the menu loads, a run starts, every content page loads, the server endpoints answer and the error reporting works. Testing the unzipped package, not the working folder, has caught a bad archive before it was uploaded.
What testing cannot tell you
We say this on every release, because it is true. Automated tests run in a desktop browser with software rendering. They have not told us how the game runs on a particular phone's GPU, how it feels under a real thumb on a real touch screen, how a screen reader sounds with a real person using it, or whether an advertising programme will approve a site. The numbers we publish about performance come from software rendering, which is slower than most real hardware, so they are pessimistic. If something misbehaves for you, tell us what you saw, with your browser and the wave you were on. It is the one kind of information the harness cannot produce, and it is how most of the real bugs above were finally understood.