TSDS Framework Optimizes Edge LLM Agents with Deferral
A new framework called Think Short, Defer Smart (TSDS) enables edge-based LLM agents to manage reasoning budgets while deferring uncertain actions to cloud models. TSDS integrates a lightweight convergence probe that halts on-device reasoning once an action stabilizes, and a perplexity-based deferral rule for escalation. Both mechanisms are calibrated via a multi-objective Learn-Then-Test (LTT) procedure, providing finite-sample guarantees on expected episode reward and cloud-call rate. The framework was evaluated on four ReAct benchmarks spanning arithmetic, multi-hop QA, code generation, and physical AI control. The paper is available on arXiv under ID 2607.26865.
Key facts
- TSDS stands for Think Short, Defer Smart.
- The framework is designed for edge LLM agents following the ReAct paradigm.
- It uses a convergence probe to halt on-device reasoning when action stabilizes.
- A perplexity-based deferral rule escalates uncertain actions to a cloud model.
- Calibration is done via a multi-objective Learn-Then-Test (LTT) procedure.
- Finite-sample guarantees are provided on expected episode reward and cloud-call rate.
- Evaluated on four ReAct benchmarks including arithmetic and multi-hop QA.
- Paper available on arXiv with ID 2607.26865.
Entities
Institutions
- arXiv