The Recognition-Exposure Gap: Why Upgrading Intent Detection Can Reduce Tool Availability in LLM Agents
识别与曝光差距：为何升级意图检测机制会降低大型语言模型智能体的工具可用性
Abstract
Tool-rich LLM agents increasingly gate tool visibility by intent-detection confidence: a narrow tool set is exposed when the agent is confident about the user’s intent, and a broad fallback set is exposed otherwise. This “confidence-tiered exposure” is widely assumed to be monotone in detector quality—a better intent detector should make the correct tool more available. We report a counterintuitive failure that violates this assumption. In a production desktop agent (WeClaw, 119 registered tools), replacing a rule-based intent detector with an LLM-based one raised recommendation accuracy to 100% (17/17 seed queries) yet lowered real tool-exposure coverage to 52.9% (9/17): in 47.1% of cases the detector named the correct tool but the tool never entered the function-calling schema set, so the model could not call it on the first attempt. We name this failure the Recognition-Exposure Gap and trace it to a structural cause—confidence-to-tier mapping and recommended-tool-to-exposure-set construction are two decoupled stages, so a detector upgrade that shifts the confidence distribution can silently drop correctly recognized tools. Critically, the gap is not a harmless offline artifact: an execution-level replay that faithfully drives the production failure-escalation ladder shows the gap costs 16 failed round-trips and 8 tier escalations across the affected queries, inflates the exposed tool set by 136% (mean final-tier exposure 8.9 → 21.0), and adds ≥31.2 s of cumulative interaction latency—a “correct, but late and expensive” recovery. A zero-training co-design fix that merges recognized tools into the exposure set closes the gap completely (first-attempt reachability 52.9% → 100%, failed round-trips 16 → 0, escalations 8 → 0), and reduces rather than increases exposure. We position the Recognition-Exposure Gap as the first quantified tool-exposure instance of the “coupling bottleneck” between model and harness, and abstract a general failure condition—confidence-tiered exposure plus decoupled recommended-tool injection—that is detector- and framework-independent.
摘要
功能丰富的大型语言模型（LLM）智能体正日益根据意图检测置信度来控制工具的可见性：当智能体对用户意图具有较高置信度时，会展示一个狭窄的工具集；反之，则展示一个广域的备用工具集。这种“基于置信度的分层暴露机制”通常被认为与意图检测器的质量呈单调关系——即更优质的意图检测器应使正确的工具更易于被调用。然而，我们发现了一起违背这一假设的反直觉案例。在一款实际部署的桌面端智能体（WeClaw，共119个注册工具）中，将基于规则的意图检测器替换为基于LLM的检测器后，推荐准确率提升至100%（17/17个基准查询），但实际工具暴露覆盖率却降至52.9%（9/17）：在47.1%的案例中，检测器识别出了正确的工具，但该工具从未被纳入函数调用模式集，因此模型无法在首次尝试时调用该工具。我们将这一现象称为“识别-暴露差距”，并将其归因于一个结构性原因——即“置信度-层级映射”与“推荐工具-暴露集构建”这两个阶段是相互独立的；因此，对意图检测器进行升级（从而改变置信度分布）可能会悄然排除那些被正确识别的工具。至关重要的是，这一差距并非无害的离线伪影：通过执行层面的重放操作——该操作能准确模拟生产环境中的故障升级链路——可发现：该差距导致受影响查询中出现 16 次往返通信失败和 8 次层级故障升级；使暴露的工具集规模膨胀了 136%（平均最终层级暴露度从 8.9 升至 21.0）；并增加了 ≥31.2 秒的累计交互延迟——这属于一种“正确但延迟且代价高昂”的恢复过程。通过一种零训练的协同设计修复方案（将已识别的工具整合至暴露集之中），可完全消除该差距（首次尝试可达性从 52.9% 升至 100%，往返通信失败次数从 16 次降至 0，故障升级次数从 8 次降至 0），且该方案并未增加而是减少了工具暴露度。我们将“识别-暴露度差距”定义为首个可量化表征模型与运行环境之间“耦合瓶颈”的工具-暴露度实例，并抽象出一种通用的故障条件——即基于置信度的层级化暴露度叠加解耦式推荐工具注入机制——该条件与检测器及框架均无关。
关键词：tool exposure · LLM agents · function calling · intent detection · agent harness · recognition-exposure gap
浏览 61 · 下载 9 · 归档于 Zenodo