Cactus in a browser tab: Mirage, Whistle and Needle on my lab page

ai · 2026-11-15 · 5 min

cactus compute builds small models for phones and wearables, and ships the whole engine as a 1.2 MB WebAssembly binary. That combination is the first time a vision model has been cheap enough to put inside a lab card on this site: Mirage is 21.6 MB at 2 bits, Whistle is 16.9 MB of speech recognition, Needle 3 is 35.3 MB of tool calling. No account, no API key, no request leaving the tab.

What the card does

The demo is called Cactus and it has its own page at /cactus. Three tabs, one worker:

  • Mirage reads an image. Start the camera or upload a file, and it returns a scene description, the objects with boxes and scores, and any text it can read. The boxes are drawn over the preview at their real position.
  • Whistle turns speech into text with word timings. Record from the microphone or drop an audio file; it resamples to 16 kHz mono and transcribes.
  • Needle takes tool definitions as JSON plus one sentence, and returns the call it would make.

Everything I measured below is from that card, not from a notebook.

Why the model runs in a worker

Inference is 1-3 seconds of saturated CPU. On the main thread that is a page that stops scrolling, a settings button that does not open, a tap that lands on nothing. The worker also gives the download somewhere to live: the model is fetched with a reader, progress goes back to the UI as it arrives, and the bytes land in Cache Storage so the second visit never touches the network.

The engine glue is a classic script that assigns a global, so the worker imports it through a blob. And because a worker has no document.currentScript, the glue cannot guess where the .wasm sits: hand it the bytes instead of a URL, which is also one round trip fewer.

const glue = await (await fetch(`${ENGINE}/cactus.js`)).text();
importScripts(URL.createObjectURL(new Blob([glue], { type: "text/javascript" })));
const wasm = new Uint8Array(await (await fetch(`${ENGINE}/cactus.wasm`)).arrayBuffer());
const M = await createCactus({ wasmBinary: wasm });

After that the whole API is four calls in the C header: cactus_load takes the .cact bytes and a length, cactus_describe takes packed RGB and a JSON options string, cactus_transcribe takes float32 PCM, and cactus_init plus cactus_complete cover tool calling. One detail that cost me ten minutes: the length argument of cactus_load is an unsigned long long, so in JavaScript it is a BigInt. A plain number throws before anything else can go wrong.

What Mirage actually returns

One photo, four objects, one line of text, about 500 bytes of JSON:

{
  "scene": { "setting": "roadside", "lighting": "daylight", "weather": "clear", "sky": "blue" },
  "objects": [
    { "id": 2, "name": "stop sign", "bbox": [378, 237, 645, 437], "score": 0.643 },
    { "id": 3, "name": "pole", "bbox": [485, 406, 534, 984], "score": 0.555 }
  ],
  "text": [{ "string": "ARRÉT", "bbox": [364, 296, 660, 379], "score": 0.856 }]
}

Boxes use a 0-1000 scale relative to the image, so drawing them is a percentage: left: bbox[0] / 10%, and so on. The model also emits segmentation masks by default; I ask for fields: ["bbox", "score"] and skip them, because polygons are four times the payload and the card has no use for them.

Numbers, from this laptop

Three runs, headless Chromium, single-threaded wasm, no GPU, no warm-up. Times are inference only: download and load are the progress bar, not this number.

model weights first result output
Mirage 21.6 MB 2.9 s description, 5 objects, 1 text line
Whistle 16.9 MB 1.1 s transcript, 14 word timings, 35 tok/s
Needle 3 35.3 MB 1.5 s one call, correct arguments

A desktop CPU at 2-3 seconds per image is not the 74 ms on a phone NPU that Cactus advertises, and I am not going to pretend otherwise. What matters for a website is that the work is bounded and on the right thread: the page stays at 60 fps while the model chews, and the answer arrives with a timestamp.

The honest limits

  • 74 MB if you try all three. Cached after the first visit, but still a real download on a phone, which is why each tab starts with nothing loaded and the progress bar is explicit.
  • Cache Storage is not permanent. A browser under pressure evicts it, and the next visit downloads 21 MB again. The card works either way, just slower once.
  • One model in memory at a time. Switching tabs unloads the previous weights, so the tab does not hold 74 MB of wasm heap.
  • No streaming. This build returns the finished JSON; there is no token-by-token render to animate. A spinner with a language model would be dishonest theatre.
  • Pinned versions. The engine is release 4.1.1 and the weights come from the Hugging Face repos as they are today. The wasm ABI and the .cact format move in lockstep, so an unpinned build would break quietly the day either changes.
  • Not a product. It is a page of its own at /cactus, a demo that loads weights on demand and asks for the camera only when you press the button. It started as experiment 18 on the lab page and outgrew the card: a 16:10 box was the wrong shape for a camera, a transcript and a JSON panel at once.

Weights and engine are Apache-2.0. The worker, the client and the UI are in src/components/cactus/ if you want to read the whole thing; it is under 500 lines including the comments that explain the two traps above.

Try it: /cactus. Start with the stop sign photo, then point the camera at your desk.