The autonomy conversation keeps asking whether a human is in the loop. The design question is more specific. What does the interface need to do as consequences, time pressure, and scale change? Those conditions don't produce one answer.
Operators don't defer to a system or reject it outright. Their behavior shifts with the situation, and the interface has to account for that movement.
TL:DR
Presentation can move trust beyond what the underlying data supports.
Authority needs to be defined across a range of actions. A single approval point isn't enough.
The amount of friction should rise with the consequence of an action.
In highly autonomous systems, operators spend more time monitoring system health than approving individual decisions.
Time pressure and scale can make deference necessary. The design has to account for that.
The interface patterns for meaningful explainability are still underdeveloped.
I was designing a land navigation device with a very small screen. Its default view had to display the most critical data points at once, but the points didn't have equal importance. I led a card sort of almost sixty attributes to understand what information people needed most and what they needed to see immediately. Across the research, the most important information was a soldier's current MGRS grid location.
We built two prototypes. One gave each data point similar visual weight. The other made the grid location more prominent. In user research with ground infantry and special forces, the scaled hierarchy was preferred. The data didn't change. Its presentation did, and that made the information easier to act on.
That example shaped how I think about trust. I haven't seen operators express broad doubts about whether system data is accurate. In my experience, they generally expect the data to be reliable and the system to flag information that's stale or wrong. My concern increases when a system starts making recommendations. That's where the interface can change how much a person defers.
Presentation and trust
A contemporary interface can move a person's trust further than the underlying data deserves. In defense, improving presentation is still a meaningful near-term opportunity. Much of the software I encounter is flat, dense, and difficult to parse. A designer can make it clearer and easier to use. But confidence can move along with comprehension, even when the evidence behind that confidence hasn't changed.
I notice the same effect in myself. When Claude or ChatGPT gives me a few sentences about what it's doing, my trust goes up. Those sentences are a simple mechanism, and they can make me feel more confident than they should.
A 2026 study in the Journal of Conflict Resolution examined a targeting decision-support system with Israeli military personnel. The study reported that participants approved fewer strikes when the recommendation came from AI, and that hesitation increased as projected collateral damage rose. A brief explanation then reduced that hesitation. The authors interpret this as more considered evaluation. They may be right. The part I keep returning to as a designer is that a short explanation can also make someone feel that the system is working, including in areas they haven't examined.
That doesn't mean explanations should be withheld. It means an explanation can't stand in for access to the evidence behind a recommendation. The appropriate level of detail depends on what's happening and what's at stake.
Authority on a scale
The line between an interface that informs a decision and one that effectively makes the decision can get crossed quietly. Teams often start with capability, meaning what the system can do. The question of how authority is divided can seem secondary until the team works through an actual decision.
I'd rather define authority as a scale for each system. What can only a person do? What does a person need to be informed about? What can a person observe without intervening? A simple maneuver by an autonomous agent might be observable. A plan the agent proposes might need to be visible without requiring approval in every case. When a plan reaches execution, especially in a kill chain, a human almost certainly needs to approve it. The system also needs to establish who has that authority, in what role, and under what conditions.
The system and the person will sometimes disagree. The design and the operating rules should establish in advance whose decision stands and when. That question shouldn't be left for the interface to resolve in the moment.
I've seen stakeholders begin with a binary understanding of human involvement. When we discuss the actual decisions the system will make, the nuance becomes apparent quickly. A recent Air Force demonstration offers one public example. An F-22 pilot issued tactical directives to a YFQ-44A Fury, which then flew maneuvers and simulated combat tasks. The pilot was neither in the loop nor out of it and I want to understand what those directives looked like, because that's where the pilot exercised authority.
Trigger guards sized to consequence
I use "trigger guard" to describe an interface intervention that interrupts routine deference long enough for a person to exercise judgment. The guard should be sized to the consequence. A simple, reversible maneuver needs a light touch. An action with lethal consequences needs a much heavier one.
Effector selection is one consequential point in an air-defense workflow. Kinetic and electronic warfare options can have very different effects on an adversary and on the surrounding environment. Some of that selection is automated, but I believe a human should approve it. Permission to fire is a more severe decision and needs a stronger guard. The appropriate interaction can depend on what has already happened. If a person has worked through the full decision chain, the interface needs less friction than when the system itself is recommending the action.
The simplified version of this interaction is a prompt. Something is happening, do you want to eliminate it? I don't think it should be that simple. At the same time, there's a complication. The less an operator understands the data in front of them, the more likely they are to defer to the system. For many routine decisions, I think that's acceptable. For consequential decisions, it's a problem the interface can't solve by adding a confirmation button.
In a highly autonomous environment, an operator's attention isn't best spent approving every decision. That would work against the purpose of the system. The operator's time goes to monitoring whether it's operating as intended and recognizing when it isn't. Their agency shifts from each individual decision to judging whether the system should continue making decisions at all.
Trust should track system health. As health degrades, confidence should degrade with it, potentially to the point where the system is shut off. Health indicators and safeguards need to be visible. If operators can't tell whether the system is functioning as intended, that's a reason to withhold trust.
When the Air Force evaluated portable ground stations for commanding collaborative combat aircraft, it considered weight, setup time, connection time, time to command, and packed size. Those are real constraints. I'd add another, which is whether the operator can understand the system's health. In an autonomous environment, I'd expect monitoring that health to be much of the work.
Time and scale
Time and scale make the spectrum harder to manage. As consequences rise, operators are likely to become more cautious. In my air-defense work, tracks averaged fourteen seconds to target. Under that kind of time pressure, the operator can't investigate every recommendation before acting.
Scale creates a similar pressure. The question of one person managing “2,000 tracks” keeps coming up. At some point, the volume overwhelms what a person can assess individually, and they have to defer. The interface can't prevent every instance of deference without also getting in the way of action. The mechanisms meant to discourage routine deference need to adapt to the scenario.
I also hear the expectation that anyone should be able to sit down and use a system after twenty minutes of training. I understand the motivation. Reducing training burden is a central part of my work. But these are professional tools, and some of the decisions they support are complicated. An untrained person can't make an informed decision to fire in fourteen seconds because the interface is clear. Training gives an operator the knowledge to question a recommendation, much as an airline pilot still needs to know how to fly when automation fails.
Design can reduce the amount of training a tool requires. It can't replace the understanding that makes an operator's judgment meaningful.
Opacity and accountability
I want to be direct about opacity. The documentary NAZA presents testimony from Israeli officers about AI-assisted targeting in Gaza. The IDF disputes that testimony. I can't independently verify the accounts, but the system they describe raises serious design and oversight questions. Officers who couldn't trace how a target recommendation was produced, who didn't understand how collateral-damage thresholds were weighed against target priority, and a unit culture in which some interviewees said they lost their moral bearings.
I find that deeply upsetting. If a decision process can't be understood in real time or reconstructed afterward, it isn't adequate for oversight or accountability. The claim that a system is too complex for people to understand isn't an acceptable answer to that problem.
I don't think the designer is the first line of accountability when trust fails. The interface has a responsibility to communicate the information an operator needs to make the best decision available to them. Accountability for what happens next is a larger policy question. The documentary also reinforces something I believe about experience design, which is that the interface is only part of the system. Decisions are shaped by peers, chain of command, training, and what the unit rewards. Those conditions matter whether or not anyone treats them as design.
The patterns still to be designed
The public design patterns I've found for explainable AI go little beyond natural-language summaries. Anduril says explainability is built into Lattice. I think that's a useful start, but I suspect it isn't enough. That's a hypothesis, since I haven't seen the underlying design to evaluate it.
My hypothesis is that explainability will need to be tailored to each product. The available data, the decision, and its consequences vary. I'd expect an interface to need progressive disclosure down to raw data, explanations that deepen as the stakes rise, health indicators that can't be hidden, and trigger guards that adapt to context. Friction is one part of that design, and on its own it's a small part.
Bringing those elements together in a pattern library is work still ahead. The work I'd do here would be new, and I'd want operators involved in shaping and testing it.
Human agency in the age of autonomy won't have one answer. It will look different across domains and situations. That's the question I'm still working through.
Further reading
Hoff and Bashir, "Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust," Human Factors, 2015 (paywalled). https://journals.sagepub.com/doi/10.1177/0018720814547570
Vasconcelos et al., "Explanations Can Reduce Overreliance on AI Systems During Decision-Making." https://arxiv.org/abs/2212.06823
Biswas, "Modeling AI Overreliance as a Complex Adaptive System," 2026. https://arxiv.org/abs/2608.19616
Kahn, Horowitz, and Samotin, "What is Human in Judgment?" 2026. https://arxiv.org/abs/2604.04333