Budget-Aware Routing for Long Clinical Text

A recent study introduces RCD, a monotone submodular objective aimed at budgeted context selection for large language models handling lengthy clinical texts. This approach tackles constraints related to token costs and latency by choosing a limited number of document units within a defined budget. The researchers evaluate various unitization methods, including sentence, section, window, and cluster-based approaches, and present a routing heuristic that adjusts according to budgetary conditions. Findings from experiments conducted on MIMIC discharge notes, Cochrane abstracts, and L-Eval indicate that the most effective strategies vary depending on the evaluation context.