An open-source 1.2-billion-parameter language model for Hindi and its 37 dialects, served on 5th Gen Intel® Xeon® processors with Bud Runtime — delivering GPU-class inference performance on CPUs, without the GPU.
Bud Ecosystem, Intel, and Tech Mahindra collaborated to benchmark Project Indus — an innovative open-source language model designed specifically for Hindi and its dialects. Focused on the Indian linguistic landscape, Project Indus aims to advance natural language generation and processing for languages that mainstream models routinely under-serve.
The study profiles the model across the metrics that matter for production inference: Time to First Token (TTFT), inter-token delay, input and output prompt length, and total throughput in tokens per second. By evaluating these parameters under varying numbers of concurrent requests, the team built a detailed performance picture of the Indus LLM on Intel® AI hardware — surfacing the model's effectiveness, scalability, and practical deployment economics.
India is a linguistically diverse country, and the largest language models built by leading firms still struggle to respond reliably across its many dialects. Hindi alone has over 600 million speakers and encompasses numerous dialects, several spoken by more than a million people each. Closing this gap requires a foundational model built in Hindi that comprehensively includes its dialects — not an afterthought bolted onto an English-first model.
Beyond coverage, large language models carry a significant carbon footprint — for both training and inference — making them environmentally costly to run at scale. A CPU-first inference path directly addresses that cost, on hardware enterprises already own.
Makers Lab — the innovation team at Tech Mahindra — developed Project Indus, a large language model purpose-built for Hindi and its 37 dialects. It began with an extensive data-collection effort that produced the world's largest dataset for these languages: 100 GB of text.
The model was trained on CDAC GPUs — a cluster of 48 NVIDIA A100 machines. For inference, the fine-tuned Indus model runs on a platform powered by 5th Gen Intel® Xeon® processors. Intel collaborated with Bud Ecosystem to develop a custom build of the model, integrating Bud Runtime and an inference stack to extend its functionality.
The build was validated and benchmarked using the LLMPerf library, running load tests that measure inter-token delay and generation throughput for both individual and concurrent queries.
Intel collaborated with Bud Ecosystem to develop a custom version of the model, integrating Bud Runtime and an inference stack to enhance functionality. Five capabilities carry the solution.
Supports portable, production-ready LLM applications with high throughput and concurrency — proven on Intel® Xeon® processors across the 4th, 5th, and 6th generations for efficient large-scale handling.
Advanced optimization methods enhance both performance and security, targeting the best total cost of ownership (TCO) and return on investment (ROI) for performance-driven applications.
Uses Intel® Advanced Matrix Extensions (AMX) and Intel® Advanced Vector Extensions 512 (AVX-512) to improve inference-serving performance, optimizing computational efficiency and data processing.
Supports 'Across the Box Scale-Out' and 'In-Box Scale-Out' approaches with Sub-NUMA clusters, addressing memory-performance issues and enabling efficient parallel application processing.
Provides a single API set and toolchain for seamless integration and deployment, streamlining the development and management of LLM applications.
Throughput, latency, time-to-first-token and scalability — measured on Intel® Xeon® with the LLMPerf load-test suite.
The Indus LLM was benchmarked extensively on the Intel platform across the critical inference metrics — TTFT, inter-token latency, prompt lengths, and throughput — at varying levels of concurrency.
| Metric | Condition | Value |
|---|---|---|
| Average throughput | steady state | 33.998 tok/s |
| End-to-end latency | average | 3.07 s |
| Total response time | 22–167 tokens | 0.249–4.27 s |
| Time to First Token | 2 NUMA nodes · 100 concurrent | 8.03 s |
| Time to First Token | p90 · 200 concurrent | 17.18 s |
The performance of the Indus LLM is comparable to GPU inference — confirming that a CPU-first path on Intel® Xeon® can serve a production Hindi LLM without conceding throughput.
Overall, the Indus LLM proves to be a robust, versatile, and efficient model for natural language processing — demonstrating strong scalability and performance on the Intel platform, and a viable, sustainable route to serving India's languages at scale.