The Goal: Frictionless Interactive Segmentation​

When an annotator uses Intel® Geti™ to segment an object, their mental model should be instantaneous: click an object, get an accurate polygon mask, annotate, and move on. They should never have to think about where inference is running.

To deliver that speed, our guiding philosophy was simple: keep the interaction local. Network round-trips for every mouse click introduce latency jitter that destroys the tactile feel of an interactive annotation tool.

Achieving that in production, however, required rethinking how we partition vision transformer workloads between native hardware and web runtimes.

1. The Initial Architecture (And Why It Made Sense)​

In our first implementation, we took the "keep it local" philosophy to its logical extreme: we ran the entire Segment Anything (MobileSAM) pipeline directly inside the client runtime.

Whether running in a desktop browser or our standalone Tauri desktop application, the browser engine owned the entire lifecycle:

To make this viable in a web UI, our frontend team engineered a multi-threaded client pipeline:

• Off-Main-Thread Execution: A shared Web Worker hosted ONNX Runtime Web (ort-web) for both the encoder and decoder.
• Comlink Bridging: Maintained a responsive UI thread by wrapping worker communications in asynchronous RPC-style Comlink calls.
• In-Memory Cache: Once an image was encoded into an embedding vector, users could click multiple objects without ever re-running the encoder or pinging a server.

2. The Wall We Hit (The Webview Reality Check)​

On paper, this was an elegant client-only architecture. In production, we hit a hard ceiling dictated by client runtime fragmentation.

While Web Workers prevent UI freezing, they do not reduce the raw computational cost of a Vision Transformer. Geti targets two distinct environments: collaborative web browsers and packaged Tauri desktop apps. The performance gap between them was staggering:

The best-case browser run was great, but the worst-case was completely unacceptable. We needed native execution speeds, but sending every prompt click over a network connection to a cloud server would destroy interactivity.

3. The Pivot (Image Work vs. Prompt Work)​

The breakthrough came when we stopped viewing the pipeline as a binary choice between frontend and backend, and instead categorized the workload by its interaction frequency:

The new split pipeline:

Because images already reside on the backend (or in local desktop storage), the UI does not upload raw pixels. It simply requests an embedding by image ID.

Crucially, in Geti's Tauri desktop application, the "backend" is simply a local native sidecar process. The architecture is identical whether running against a cloud server or a local desktop binary: native silicon execution outside the webview sandbox.

4. How We Built It (Frontend & Backend in Tandem)​

Splitting a neural network across process boundaries required both engineering teams to solve hard systems problems simultaneously.

The Backend: Flexing Intel® Silicon with OpenVINO™​

Our backend and AI engineers built a native inference service optimized for Intel platforms:

• Dynamic Hardware Targeting: Using OpenVINO's unified API and AUTO device plugin, the backend seamlessly targets Intel Core™ CPUs, integrated Arc™ GPUs, or dedicated NPUs without code branching.
• Hybrid Core Scheduling: Modern Intel CPUs feature a mix of Performance cores (P-cores) and Efficient cores (E-cores). By setting OpenVINO’s performance mode to LATENCY, the engine optimizes thread scheduling so heavy ViT encoding finishes rapidly without starving companion OS tasks.
• Lean Memory Lifecycle: In a desktop sidecar or containerized environment, models cannot hoard VRAM. We utilized OpenVINO Compiled-Model Caching (ov::cache_dir) to store compiled binaries on disk, keeping model weights ready while creating and releasing individual ov::InferRequest instances per job to return execution memory immediately to the OS.

The Frontend: High-Performance Client Orchestration​

Meanwhile, our frontend team rebuilt the client runtime to ingest and manage external embeddings without dropping frames:

• Zero-Copy Memory Pipelines: Receiving multi-megabyte binary payloads can trigger severe garbage collection churn. The frontend streams raw safetensors buffers and transfers them into Comlink Web Workers via transferable ArrayBuffer objects, completely bypassing structured-clone overhead.
• Dual-Tier State Machines: Using TanStack Query, the client separates execution lifecycles from data lifecycles. Decoder workers stay alive across project navigation, while heavy 4 MiB embedding tensors are evicted from memory after 60 seconds of inactivity.
• Canvas Coordinate Parity: OpenCV WebAssembly handles polygon generation and letterbox aspect-ratio math directly on the client canvas thread, ensuring generated masks lock precisely to user cursor clicks.

The Contract: Geometry and Precision​

To prevent spatial drift between two different runtimes, the backend serializes the embedding with strict geometry headers (original dimensions, resized dimensions, and padding offsets).

While quantizing to Float16 would have halved the payload from 4 MiB down to 2 MiB, early evaluations revealed numerical instability in MobileSAM's boundary precision under FP16. We prioritized deterministic mask quality and kept the contract at Float32, using gzip/brotli transport compression to minimize wire transfer times.

5. The Results & The Takeaway​

Moving the encoder to native OpenVINO eliminated the wild performance variance of browser-side ViT execution:

Note: Benchmarks represent pure MobileSAM encoder execution time on representative testbeds, excluding I/O, serialization, and network transfer.

Put next to the worst-case webview behavior from before, the difference is hard to miss:

On Tiger Lake CPU, native execution delivers rock-solid consistency matching the best-case browser runs. On next-generation Panther Lake integrated graphics, encoding drops to ~51 ms—fast enough to feel instantaneous.

Once that single embedding reaches the client, every click prompt, negative point, and boundary edit runs locally inside the browser with zero network lag.

The Architectural Lesson​

High-performance AI web applications do not need to be 100% cloud-hosted or 100% client-side. The key is knowing what kind of work you are scheduling:

Move the expensive preparation closer to native compute.
Keep the repeated interaction close to the user.

Expect posts on model optimization, OpenVINO integration, browser and native runtimes, data workflows, and the engineering tradeoffs that shape Geti. Each post will focus on the problem, the approach, and what we learned in production.