OpenAI Launches IndQA Benchmark Based on Indian Languages and Culture

Exam-Oriented Key Points

1. Overview and Objective

• OpenAI launched “IndQA” on November 5, 2025 , as a new benchmark to evaluate how well AI systems understand Indian languages, cultures, and contexts .

• The initiative focuses on enhancing AI’s multilingual and multicultural understanding , starting with India — one of the most linguistically diverse countries in the world.

• Project name: IndQA benchmark.
• Developer: OpenAI.
• Release date: 5 November 2025.
• Size of dataset: 2,278 questions.
• Languages included: 12 Indian languages.
• Cultural areas covered: 10 domains.
• Contributors: 261 subject experts.
• Assessment method: rubric-based scoring (not multiple choice).
• Models evaluated: GPT-4o, GPT-4.5, GPT-5 and OpenAI o3.
• Purpose: strengthen AI’s understanding of India’s languages and cultural context.

2. Development and Structure

• Developed in collaboration with 261 domain experts across India.

• The dataset includes 2,278 natively written questions (not translations) across 12 Indian languages and 10 cultural domains .

• Domains include Literature, History, Spirituality, Law & Ethics, Food, Arts & Culture, and Everyday Life .

• IndQA ensures authentic phrasing and context , unlike conventional benchmarks such as MMMLU or MGSM .

3. Evaluation Method

• Uses a rubric-based evaluation system instead of multiple-choice questions.

• Each question contains:

  • A culturally contextual prompt in an Indian language,

  • English translation,

  • Expert-designed rubric,

  • Ideal model answer.

• AI responses are graded for nuance, reasoning, and cultural correctness , ensuring higher accuracy in real-world context understanding.

4. Language and Model Testing

• Covers 12 languages — Bengali, Hindi, English, Hinglish, Kannada, Marathi, Odia, Telugu, Gujarati, Malayalam, Punjabi, and Tamil.

• Tested using GPT-4o, GPT-4.5, GPT-5, and OpenAI o3 models .

5. Significance and Future Plans

• Aims to make