Measure first: making a TV app stop feeling slow

The brief for bili-webos this round was one sentence: the user should notice loading as little as possible. Scrolling to the next page, thumbnails filling in, the player starting — all of it should feel attached to the remote rather than to the network.

The instruction that made the work honest came with it: build the observation scheme first, optimise second. That ordering sounds like process ceremony. It is not. It is the only reason the numbers below are believable, and — more usefully — the only reason an optimisation that would have made the app dramatically worse never shipped.

An instrument that costs nothing to carry

The target hardware sets the rules. My development TV is a 2024 OLED: four cores, two gigabytes of reported device memory, a recent Chromium. Plenty of users are on older sets running Chromium 68 or 79. Anything I measure on a desktop is fiction, and anything that relies on a modern performance API silently returns nothing on half the install base.

So the instrument had three constraints. It records at explicit code points using nothing but a timestamp, so old and new engines both produce data. It stores fixed-size buckets of plain numbers — no object trees, no string building, no serialising. And it never grows: a bounded number of samples per metric, a few kilobytes total, flat over hours of use.

The metrics are deliberately phrased as waits a person can feel, not as internals: key press to focus drawn, scroll trigger to new cards drawn, image decided-to-load to visible, player open to first frame, comments open to list drawn.

The measurement traps were worth more than the results

Five of them, each of which produced a confidently wrong answer before I caught it.

• Stashing the changes to get a baseline removes the instrument too. My first before/after comparison had an empty "before" column — obviously, since the timers lived in the same diff. Every optimisation had to move behind a runtime flag so both runs use one build, one set of timers, and only the switch changes.
• A single run is not a measurement. One network retry stretched a first-frame number to 12.7 seconds and I nearly wrote it down. Everything became a median of repeated runs.
• Opening the player repeatedly cannot measure engine load cost. Only the first open pays for the module import; the rest hit the module cache. The harness now reloads the page per round and measures only the cold first open.
• The throttled desktop simulator cannot test lazy loading. Desktop Chrome cheerfully loads an entire screen of images at once, so anything image-related has to be confirmed on the television.
• Caching quietly contaminated a comparison. Once the on-device proxy started forwarding cache headers, whichever variant ran first warmed the cache for the second. Every round now clears the HTTP cache — which also matches what a user's first visit actually feels like.

The instrument even broke itself once. A single shared ring buffer of 200 samples sounded fine until one screenful of a hundred and seventy-three thumbnails evicted every pagination sample, and the report announced "no data" for the metric I most wanted. Per-metric buckets fixed it. An observability system that can be starved by its own busiest signal is not observability.

The optimisation that made everything worse

This is the part I would keep if I had to throw the rest away.

Prefetching the next page and the next row of thumbnails is the obvious fix for "scrolling stutters", and in isolation it works: pagination dropped by an order of magnitude. Then I read the rest of the table.

The middle column is a disaster wearing a success's clothing. Images twice as slow, the player 2.3 times slower to show a frame, ten times as much main-thread blocking. On a two-gigabyte television, speculative work is not free — it competes with exactly the work the user is waiting for. Without the instrument I would have shipped that, because the one number I had gone looking for did improve.

The fix is not less prefetching but disciplined prefetching: it waits until scrolling settles, never runs while a key press is being serviced, keeps at most one request in flight, and stays bounded. Prefetching is something you do when idle, not something you do eagerly. Adding the settle delay alone recovered the responsiveness that an earlier, greedier version had quietly taken.

What actually worked

Measured on the television, cold cache, variants alternated within the same session:

The single largest win was not clever. The on-device proxy was dropping the upstream cache-control, etag and last-modified headers, so the browser cached nothing — scrolling back re-downloaded every thumbnail, and, best of all, images that had been prefetched were downloaded a second time when they were actually displayed. The prefetch was doing negative work. Passing four headers through turned a year-long cache back on.

After that: request thumbnails at the size the card actually renders instead of shipping 720-pixel images into 420-pixel slots; move focus and scroll updates off React and straight onto the DOM; let cards opt out of layout while off-screen. And the balance requirement stayed in force throughout — none of this buys speed with memory. Heap usage before and after is the same. On this hardware, "cache everything" trades one stall for another.

The list of things that did nothing

Reported with the same weight as the wins, because a performance write-up without them is advertising:

• Shrinking thumbnails further, 420px to 336px: 457 ms versus 458 ms. Bytes were not the constraint.
• Prefetching two rows of images instead of one: no difference.
• Unloading off-screen images to save memory: measurably slower.
• Rendering a new page in chunks: no improvement.
• Taking scrolling off React: 1–2 ms, inside the noise. I kept the change because it is structurally correct, but it is not counted as a win.

Knowing when to stop

Two findings ended the project more decisively than any remaining idea could have continued it.

First, the responsiveness metric hit the floor of its own yardstick. Measuring "drawn" honestly requires waiting two animation frames, which on this display costs about 33 ms by itself — and the measured figure was in the twenties to low thirties. Key-press-to-focus was already free; I had been measuring the ruler.

Second, a CPU profile of seven and a half seconds of scrolling on the TV: the main thread is 66% idle, application JavaScript accounts for roughly 250 ms, and the engine's own layout, paint and image decode account for 2169 ms. There is no JavaScript left to optimise. Further gains would mean virtualising the grid — touching scroll geometry that has historically been a bug source — for a theoretical saving smaller than the measurement floor. The right call is to stop.

Then take the instrument back out

Shipping the measurement code to users would be a small ongoing tax on people who get nothing from it, so release builds now compile it out: a build-time constant makes every recording function's body dead code, the bundler deletes it, and no performance observer is installed on a user's television. Instrumented builds are one environment variable away for the next investigation.

One last discipline problem: how do you verify that removing your measurement code did not change performance, using measurement code you just removed? With a different ruler — one injected from the debugger rather than compiled into the app, identical across both builds. The answer was 3.3 ms versus 3.2 ms. No difference.

Which is the honest conclusion, and I wrote it into the commit message: stripping the instrument is not a performance improvement. It never cost anything. The reason to remove it is that users should not be running your test equipment.

If there is one transferable idea here, it is that the instrument's job is not to confirm your optimisation. It is to catch the one that makes everything else worse — and, eventually, to tell you that you are finished.