TokenPrint: New Method for Language-Model Provenance
A recent study presents TokenPrint, a method for fingerprinting that does not require training, aimed at determining the provenance of language models, including their foundational checkpoints and possible training distribution overlaps. This technique employs 250 fixed knowledge probes to extract late hidden states and assesses top-k vocabulary projections through Jaccard overlap on decoded token strings. Tested on 32 open-weight models from nine different families (ranging from 0.6B to 32B) with established relationships, the findings illustrate a similarity hierarchy related to model connections. The paper, which can be found on arXiv under the identifier 2608.08139 and categorized as 'new', offers a solution to governance issues that metadata alone cannot address, potentially aiding in the verification of model origins and training data overlaps.
Key facts
- TokenPrint is a training-free fingerprinting method for language-model provenance.
- It uses 250 fixed knowledge probes and compares top-k vocabulary projections via Jaccard overlap.
- Evaluated on 32 open-weight models from nine families (0.6B–32B).
- Independently trained models on identical data score 0.48 raw (0.35 vocabulary-corrected).
- Shared-base fine-tunes score 0.39/0.33.
- Same-developer relatives score 0.38/0.28.
- Models with no documented relationship score 0.22/0.17.
- The signal persists across three organizations, two tokenizer families, and two architecture classes.
- The method emerges within the first few layers of the models.
- Paper available on arXiv with ID 2608.08139.
Entities
Institutions
- arXiv