Active Learning with Noisy Oracles: Real-World Annotation Study
A new study on arXiv (2604.23290) analyzes active learning algorithms using real-world crowd-sourced text annotations, addressing the challenge of imperfect or noisy labeling oracles. Traditional active learning assumes oracles are infallible, but real-world applications often involve errors. Previous research simulated noisy oracles with machine learning models, which may not capture real-world annotation nuances. This research collects actual human annotations to study algorithm performance under realistic conditions.
Key facts
- arXiv paper 2604.23290 analyzes active learning algorithms
- Focuses on real-world crowd-sourced text annotations
- Addresses imperfect/noisy labeling oracles
- Traditional active learning assumes infallible oracles
- Prior research used ML models to simulate noisy oracles
- Real-world annotation challenges are more nuanced
- Study collects actual human annotations
- Aims to reduce human annotation effort in machine learning
Entities
Institutions
- arXiv