Giving a TV remote a mouse pointer

The showroom app’s Map tab puts every building on one map you move around freely: no rail, no lists, from the whole city down to street level. On a laptop that means a mouse and a scroll wheel. On the showroom TV it means a remote with four arrows and OK, and an air mouse that wobbles. (The TV’s other constraints are in Building for a 2 GB TV.) Claude Code wrote most of the code; my job was deciding how it should feel.

Move the pointer, not the map

The arrows move a pointer over the map, the way a mouse would, and the map stays still. Push the pointer against the edge of the screen and the map scrolls, so it can still travel the whole city. That gives the remote what a mouse has: a place you are pointing at.

Four arrows that feel like a hand

• A tap nudges it. Holding an arrow starts it gently, then speeds it up.
• Two arrows held together go diagonally.
• Turns curve. Some remotes only press one arrow at a time. Switching from one arrow to another curves round instead of turning a sharp corner, so even those remotes can go diagonally.

The curve falls out of smoothing each axis on its own. Speed follows a held arrow quickly, lets go of a released one slowly while another is still held, and stops briskly once nothing is:

// every frame, for x and y separately
const target = held(axis) ? dir * speed(timeHeld) : 0;   // gentle start, then faster
const lag = held(axis) ? QUICK : anyArrowHeld() ? SLOW : BRISK;
v[axis] += (target - v[axis]) * (1 - Math.exp(-dt / lag));

And at the end of a move, let go right beside a building and the pointer settles onto it. Nobody has to land exactly on a dot.

Finding a building without aiming

• The building at the pointer turns gold, and a card shows its photo, area, homes and plans. The card sits in a corner and moves across when the pointer heads into it.
• Buildings are named as you come close: the ones with AHA homes first, then every building, never overlapping and never under the card or the key hints.
• On a near tie, an AHA home wins. It’s what visitors come to see.
• OK always does something. It opens the building once it stands clear or the map is close in. Before that, it goes one level closer around the pointer.
• BACK retraces. It steps out one level at a time to the whole city, then closes the map. Open a building and come back, and you’re at exactly the same spot and zoom.

The air mouse wobbles

The TV also has an air mouse, so the map takes a drag (a flick carries on), a scroll to zoom, a hover for a building’s card and a click to open it.

An air mouse moves a little as its button is pressed, so a click can turn into a drag by accident. On the TV, a press has to travel three times as far as on a laptop before it counts as a drag. The map moves only while the button is actually held down, so a button-up the browser never passed on can’t leave the map stuck to the pointer. Put the air mouse down and pick up the arrows, and the pointer carries on from where the cursor was.

Keeping it crisp

The camera is driven by the app, not by Google’s own gestures, which misread the double-size map the app draws for sharpness. Every step in or out is a whole zoom level from the opening view, so the map stays crisp at each stop.

What I took from it

1. Model the hand, not the keys. Ease in, curve, settle.
2. Make every press useful. OK zooms until it can open.
3. Give hardware slack. An air mouse is not a mouse.
4. BACK should retrace. One level at a time, back to the same spot.