Bias Inheritance in LLM-Based Data Augmentation
A new study from arXiv introduces the concept of 'bias inheritance' in large language models (LLMs), where biases present in training data are propagated and amplified when models are fine-tuned on synthetic datasets generated by LLMs. The research systematically investigates this phenomenon across 10 classification and generation tasks, analyzing six different types of biases. By varying the proportion of augmented data in combined real and synthetic datasets, the study demonstrates that bias inheritance negatively impacts downstream task performance, particularly in bias-related classification and generation. The work aims to understand and mitigate these biases to improve fairness and robustness in LLMs.
Key facts
- arXiv paper 2502.04419v3 introduces bias inheritance in LLMs
- Bias inheritance refers to propagation and amplification of biases from synthetic data
- Study fine-tunes LLMs with combined real and LLM-augmented data
- Experiments cover 10 classification and generation tasks
- Six different types of biases are analyzed
- Bias inheritance harms performance on bias-related tasks
- Systematic investigation aims to understand and mitigate bias inheritance
- Published on arXiv as a replace-cross announcement
Entities
Institutions
- arXiv