AI voice prototyping for Lucid Assistant

Summary

An internal AI voice prototyping platform that lets designers and researchers wire interactive UI concepts to a conversational AI agent, with no custom engineering required. I designed, built, and shipped it end-to-end: the interaction logic, the prompt system, the architecture connecting LLM responses to prototype behavior, and the workflow that lets designers stand up working voice prototypes in days instead of weeks.

Why it didn’t exist

Voice-first prototyping at most companies requires a designer to write a concept document, hand it to an engineer, wait weeks for a working demo, then iterate slowly. The bottleneck means voice gets under-explored, even at companies that say AI is a priority. Designers test what they can prototype; they don’t prototype what they can’t test.

The bet I made was that a researcher with enough fluency in LLMs and prototyping tools could collapse that cycle from weeks to hours. Not by becoming an engineer, but by building the connective tissue between two existing systems: a conversational AI model on one side, an interaction-design prototype on the other.

What I built

The platform takes natural language input, routes it through an LLM, parses the response into structured actions, and triggers those actions inside an interactive prototype. A designer can describe a feature concept (a climate adjustment, a navigation interaction, a media control), define test-case scenarios, and watch a working voice interaction emerge without writing code.

The work involved decisions I made independently across multiple layers: interaction grammar between LLM responses and prototype state, model selection, prompt schemas, the message routing layer, and the workflow that designers actually use. Parts of the code were later handed to production engineers on the assistant team for downstream use. The platform itself sits inside the design team and continues to support voice-first prototyping across multiple feature areas.

How designers use it

The typical use case looks like this. A designer wants to test how a climate control conversation feels when the user interrupts the assistant mid-response. Before the platform, that test would have taken an engineer two days to mock up. With the platform, the designer prototypes it in an afternoon, tests it with users the next day, iterates by the end of the week. Multiple designers across the org now build their own voice prototypes this way.

The platform also became a foundation for benchmarking. I used it to evaluate the prototype against ten production in-car voice systems using a five-pillar benchmarking framework (see Methods). The benchmarking surfaced a specific advantage of the prototype’s interaction model: a task prioritization layer that ranked vehicle and safety interactions above entertainment, behavior several production systems didn’t replicate.

Recognition

The platform was the basis for my 2025 Lucid Extra Mile Award, received at the company-wide town hall in November 2025.

Sometimes the most useful thing is to build it yourself

I think the line between research and building is thinner than it used to be. Not because researchers should become engineers, but because the gap between an idea and a testable prototype is now small enough that crossing it yourself is sometimes faster than handing off. The voice prototyping platform is one example of what that looks like in practice. The Screen Obscuration Analysis method is another (see Methods). Both are research infrastructure built by researchers, used by designers and other researchers, and shaped by the same instinct: the most useful thing you can sometimes do is build the thing that lets the team test the thing.

That instinct shows up across most of the work on this site. It’s the through-line.