TTCov: Test-Time Data Curation for Deployed AI
The newly introduced approach known as TTCov (Test-Time Coverage) tackles the issue of preparing training data for AI systems in situations where the test distribution diverges from the training set. Unlike conventional data curation techniques that prioritize training-side metrics, TTCov leverages test-side insights prior to training to enhance deployment alignment. It breaks down the curation process into coverage and distribution components. For coverage, it creates a task Atlas, which consists of LLM-based atomic propositions (APs) that represent concepts relevant to deployment, drawing from open task knowledge and incorporating unmatched APs from unlabeled deployment data. For distribution, it develops a Knowledge Atlas (K-Atlas) by matching deployment APs with their respective frequencies. This method is detailed in a paper available on arXiv (2607.22697), aiming to refine data curation by focusing on the deployment context.
Key facts
- TTCov stands for Test-Time Coverage.
- It is a data-level test-conditioned curation method.
- It uses test-side information before training, not during inference.
- Deployment-conditioned curation is decomposed into coverage and distribution.
- Coverage is represented by a task Atlas of LLM-based atomic propositions (APs).
- The task Atlas is seeded from open task knowledge and expanded with unmatched APs from unlabeled deployment samples.
- Distribution is represented by a Knowledge Atlas (K-Atlas) with frequencies of matched APs.
- The paper is available on arXiv with ID 2607.22697.
Entities
Institutions
- arXiv