Test-Time Capability Transfer via Harnesses Nearly Doubles Model Performance
An arXiv paper (2608.12307) investigates the potential for transferring capabilities from a stronger model to a weaker one during testing, without modifying parameters. The authors introduce a technique in which a more robust 'builder' model creates inference-time harnesses that support a less capable 'target' model. This process involves using 5% of the data as a validation set for the builder model to iteratively enhance its harness across several rounds before testing on the complete dataset. This method significantly improves the average performance of the target model from 0.49 to 0.91. The findings suggest that the improvements largely stem from transferring erratic behaviors to the harness. The paper, titled 'AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses,' is noted as a cross-type submission on arXiv.
Key facts
- Paper titled 'AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses'
- arXiv ID: 2608.12307
- Announcement type: cross
- Explores test-time capability transfer without parameter updates
- Builder model constructs inference-time harnesses for weaker target model
- Uses four Theory-of-Mind benchmarks
- Builder uses 5% of data as validation set for iterative refinement
- Average target-model performance nearly doubled from 0.49 to 0.91
- Gains primarily from offloading unstable model behaviors
Entities
Institutions
- arXiv