SAHM Benchmark Introduced for Arabic Financial and Shari'ah-Compliant NLP Tasks
A new standard known as SAHM has been introduced to enhance Arabic financial natural language processing and Shari'ah-compliant reasoning, filling a void in computational linguistics. This dataset comprises 14,380 instances, all verified by experts, across seven different tasks, including QA based on AAOIFI standards, fatwa-based QA, accounting examinations, financial sentiment analysis, extractive summarization, and event-cause reasoning. Sourced from credible materials, researchers assessed 19 large language models, revealing that proficiency in Arabic does not guarantee sound financial reasoning based on evidence. While there has been significant progress in English financial NLP, Arabic financial NLP is still relatively underdeveloped, despite a growing need for dependable finance assistants. The SAHM benchmark is designed to evaluate and fine-tune Arabic financial applications, addressing the demand for Shari'ah-compliant financial support.
Key facts
- SAHM is a benchmark for Arabic financial NLP and Shari'ah-compliant reasoning
- Contains 14,380 expert-verified instances across seven tasks
- Tasks include AAOIFI standards QA, fatwa-based QA/MCQ, accounting exams, financial sentiment analysis
- Materials curated from authentic regulatory, juristic, and corporate sources
- Researchers evaluated 19 open and proprietary LLMs
- Arabic fluency doesn't reliably translate to evidence-grounded financial reasoning
- English financial NLP has progressed rapidly while Arabic financial NLP remains under-explored
- Practical demand exists for trustworthy finance and Islamic-finance assistants
Entities
Institutions
- AAOIFI