Smashing Magazine: Matching AI modality to user intent

Victor Yocco, a behavioral researcher and UX designer, opens with an observation that cuts through a lot of current product thinking: the design community has defaulted nearly every AI capability into a text chat interface, and the result is interfaces that ask users to adapt to the system rather than the other way around.

His frame is “adaptation load” — the cognitive cost users pay when they have to translate their natural thinking into prompt language, wait for a response, and then parse that response into something usable. Chat adds this cost on top of whatever task the user was already trying to do.

The alternative Yocco describes is deliberate modality selection. Before designing any AI interface, he recommends conducting an evidence-based task audit: observe users in their actual working environments and document four constraints. Input constraints are about physical reality — can users type, or are their hands occupied? Output constraints ask whether users can safely view a screen in that moment. Social constraints cover whether audio output would be disruptive. Cognitive load constraints assess how much mental effort the primary task already demands.

From that audit, teams can apply what Yocco calls an Input/Output Alignment Matrix — a mapping of user intent to modality combinations. A field technician with hands occupied near equipment needs voice input and audio output, not a text box. A data analyst building forecasts at a desk can work with a dashboard that surfaces structured recommendations. The modality follows the context, not the other way around.

The article also makes a practical accessibility argument. When an AI interface offers only one modality, it excludes everyone whose physical, sensory, or cognitive situation doesn’t match that assumption. Designing for multiple input and output paths is not an edge case exercise — it benefits all users by reducing the conditions under which they have to struggle with the interface.

Yocco does not argue that chat is always wrong. He argues that it is one option, not the default, and that choosing it without justification is a design decision made by omission. The piece is most useful for product teams building AI features who have not yet asked the question: given who uses this and where, what modality actually fits?