Most teams choose ai model input and output modalities the same way they choose lunch: by impulse, by what is in front of them, by what sounds appealing. That works until the feature ships and turns out to be too expensive, too slow, or unable to do the thing the model was never going to do. A repeatable framework replaces impulse with a decision you can explain and defend.
The method below is one I call HAVE-NEED-BRIDGE. It has three stages, applied in order, and each stage answers one question that constrains the next. The result is the smallest, cheapest, most reliable modality mix that actually solves the user's problem. It is reusable across features, which means once your team internalizes it, modality decisions stop being debates and become routine.
This is not a rigid procedure that replaces judgment; it is a scaffold that focuses judgment on the questions that matter. Use it to structure the conversation, not to shut it down.
The Three Stages at a Glance
The framework moves from the user inward to the model:
- HAVE: What input does the user already possess?
- NEED: What output does the user actually require?
- BRIDGE: What is the smallest modality path connecting the two?
Each stage gates the next. You cannot choose a bridge until you know both endpoints, and the endpoints come from the user, not from the model's feature list. That ordering is the whole trick, because it stops you from starting with "the model can do X" and working backward into a feature nobody needs.
Stage One: HAVE
Start by cataloging what the user already has in hand when they arrive at your feature. Not what they could produce with effort; what they actually possess right now.
Inventory honestly
A user might have a photo on their phone, a voice memo, a typed sentence, a PDF, or some combination. Write down the real artifacts, including their quality. "A photo" and "a blurry photo taken in poor light" are different inputs with different reliability, and the worse one is what you must design for. This is where the worst-case-input discipline enters the framework.
Output of this stage
A concrete list of input modalities and their realistic quality. This list is fixed; you do not get to wish the user had cleaner inputs than they do.
Stage Two: NEED
Now define what the user genuinely requires as output. Again, be precise about the real job, not a flashy version of it.
Distinguish the consumer
The single most important question here is who or what consumes the output. If software consumes it, the need is structured data. If a human reads it directly, the need is prose. This distinction, drawn from our best-practices guide, determines the output modality more than anything else.
Output of this stage
A concrete output modality and its consumer. "A summary a person reads" and "fields a system stores" lead to different designs even when the underlying task feels similar.
Stage Three: BRIDGE
With both endpoints fixed, design the smallest modality path connecting them. This is where you choose the model, the fusion approach, and the validation.
Apply three filters
- Minimalism: Use the fewest modalities that connect HAVE to NEED. Reject anything extra.
- Cost and latency: Budget each modality on the path; cap rich inputs and background slow outputs.
- Reliability: Confirm the model supports every modality on the path, and plan validation for the worst input from stage one.
Output of this stage
A defended modality design: specific inputs, specific output, a confirmed model, and a validation plan. Because the endpoints were fixed first, the bridge is constrained rather than open-ended, which is exactly why this stops the over-engineering that plagues unstructured decisions.
A Worked Pass Through the Framework
Consider a team building a feature that turns a photographed whiteboard into a shareable meeting summary. Running HAVE-NEED-BRIDGE keeps the decision honest.
HAVE, NEED, and BRIDGE in sequence
The HAVE stage records the real input: a phone photo of a whiteboard, often glare-streaked and shot at an angle. That imperfect quality is the input the design must survive, not a clean studio capture.
The NEED stage asks who consumes the summary. If a person reads it in a chat channel, the need is prose. If it populates a project tool's fields, the need is structured data. Suppose the answer is both, a readable summary plus extracted action items, which means two outputs with two consumers.
The BRIDGE stage then applies its filters. Minimalism says image input is required but audio is not, so audio is cut. Cost-and-latency budgeting caps the photo resolution, since image cost scales with pixels. Reliability confirms the model reads images and emits structured output, and plans validation around the glare-streaked worst case from HAVE. The result is a defensible design reached in minutes, not a debate. The execution order this implies, confirm capabilities, test the hard input, lock the schema, is exactly the sequence our step-by-step process follows.
When to Apply Each Stage
HAVE-NEED-BRIDGE runs at the start of any feature that touches more than text. But the stages also serve as a diagnostic for features already in trouble. A feature that is too expensive usually failed the cost filter in BRIDGE. A feature that does something impressive but useless usually skipped NEED and started from the model's capabilities. A feature that breaks on real inputs skipped the honest inventory in HAVE.
Run the framework forward to design, and run it backward to diagnose. Either way, the discipline of fixing the endpoints before choosing the bridge keeps you anchored to the user's actual job. For concrete walkthroughs of the framework in action, our real-world examples trace each scenario through these same three stages, and the definitive guide supplies the underlying mechanics each filter relies on.
Frequently Asked Questions
Why fix the endpoints before choosing the bridge?
Because starting from the model's capabilities leads to features that are impressive but unwanted. Fixing what the user has and needs first constrains the design to the actual job, which prevents over-engineering and keeps the modality mix minimal.
What if the user could produce a better input with effort?
Design for what they actually have, not what they could produce. Requiring users to clean up inputs adds friction that most will not tolerate. The honest inventory in HAVE deliberately captures real, imperfect artifacts so your design survives contact with reality.
How does the framework decide between structured output and prose?
Through the consumer question in NEED. If software consumes the output, the need is structured data; if a human reads it directly, the need is prose. Identifying the consumer settles the output modality more reliably than any other factor.
Can I use HAVE-NEED-BRIDGE to fix an existing feature?
Yes. Run it backward as a diagnostic. Cost problems usually trace to a skipped cost filter, useless-but-impressive features to a skipped NEED stage, and input failures to a dishonest HAVE inventory. The framework localizes where the decision went wrong.
Does the framework replace technical judgment?
No, it focuses it. The stages structure the conversation around the questions that matter and keep the design anchored to the user, but choosing the specific model, schema, and validation still requires judgment. The framework scaffolds that judgment rather than substituting for it.
Key Takeaways
- HAVE-NEED-BRIDGE selects modalities in three ordered stages: what the user has, what they need, and the smallest path between.
- Fixing the endpoints before the bridge prevents features that are impressive but unwanted.
- HAVE demands an honest inventory of real, imperfect inputs you must design for.
- NEED hinges on the consumer: software needs structured data, humans need prose.
- BRIDGE applies minimalism, cost-and-latency budgeting, and reliability filters to produce a defensible design.