Linguistic Rules as Efficient Prompt Compressors for LLMs

A recent study published on arXiv suggests that linguistic rules can serve as standalone prompt compressors for large language models, removing the necessity for LM-based scoring in the compression process. This technique employs offline evolutionary searches across lexical, syntactic, semantic, and discourse seeds to identify effective combinations of rules. During deployment, the compressor operates without requiring an LM forward pass, relying solely on CPU processing, which lowers inference expenses. The evaluation of this method utilizes a dual-path protocol that maintains a balance between the quality of compression and the fidelity of reconstruction for both short passages and multi-document tasks.