Benchmark-Based Comparative Assessment of Indian Foundation Models

A recent study published on arXiv (2608.11891) offers a detailed, benchmark-oriented comparison of Indian foundation models with global leading and similarly scaled models. This research arises from government initiatives aimed at enhancing domestic AI capabilities, promoting digital sovereignty, and advancing multilingual computing. It assesses models across eight domains, including general-purpose reasoning, software engineering, agentic AI, cybersecurity, visual understanding, video and multimodal comprehension, scientific inquiry, and capabilities in Indic languages. The findings reveal that Indian models perform well on benchmarks like MMLU and MATH-500, though these benchmarks have seen widespread regression. To tackle issues related to inconsistent reporting and proprietary evaluation methods, the paper introduces a framework for capability and evaluation maturity. The study is also cross-listed on arXiv.