Bud Case Study
Bud × Intel × Tech Mahindra · LLM Benchmarking

Benchmarking Project Indus on Intel® AI Hardware

An open-source 1.2-billion-parameter language model for Hindi and its 37 dialects, served on 5th Gen Intel® Xeon® processors with Bud Runtime — delivering GPU-class inference performance on CPUs, without the GPU.

~34 tok/s
Average throughput
3.07s
Avg end-to-end latency
37
Hindi dialects covered
≈ GPU
Inference parity on CPU
Prepared byBud Ecosystem Inc.
PartnersIntel · Tech Mahindra (Makers Lab)
Platform5th Gen Intel® Xeon® · Bud Runtime
DateNovember 2024
Benchmarking Project Indus on Intel® AI Hardware Bud
Overview

GPU-class language AI, served on Intel CPUs.

Bud Ecosystem, Intel, and Tech Mahindra collaborated to benchmark Project Indus — an innovative open-source language model designed specifically for Hindi and its dialects. Focused on the Indian linguistic landscape, Project Indus aims to advance natural language generation and processing for languages that mainstream models routinely under-serve.

The study profiles the model across the metrics that matter for production inference: Time to First Token (TTFT), inter-token delay, input and output prompt length, and total throughput in tokens per second. By evaluating these parameters under varying numbers of concurrent requests, the team built a detailed performance picture of the Indus LLM on Intel® AI hardware — surfacing the model's effectiveness, scalability, and practical deployment economics.

The problem

India is a linguistically diverse country, and the largest language models built by leading firms still struggle to respond reliably across its many dialects. Hindi alone has over 600 million speakers and encompasses numerous dialects, several spoken by more than a million people each. Closing this gap requires a foundational model built in Hindi that comprehensively includes its dialects — not an afterthought bolted onto an English-first model.

1,645
Dialects across India
27
Official languages
600M+
Hindi speakers
37
Hindi dialects targeted
The sustainability dimension

Beyond coverage, large language models carry a significant carbon footprint — for both training and inference — making them environmentally costly to run at scale. A CPU-first inference path directly addresses that cost, on hardware enterprises already own.

Bud Ecosystem · Case Study02
Benchmarking Project Indus on Intel® AI Hardware Bud
The Solution

Project Indus, optimized for Intel.

Makers Lab — the innovation team at Tech Mahindra — developed Project Indus, a large language model purpose-built for Hindi and its 37 dialects. It began with an extensive data-collection effort that produced the world's largest dataset for these languages: 100 GB of text.

1.2B
Model parameters
22B
Training tokens
100 GB
Largest Hindi dataset
$150K
Build budget · 15-person team

The model was trained on CDAC GPUs — a cluster of 48 NVIDIA A100 machines. For inference, the fine-tuned Indus model runs on a platform powered by 5th Gen Intel® Xeon® processors. Intel collaborated with Bud Ecosystem to develop a custom build of the model, integrating Bud Runtime and an inference stack to extend its functionality.

Model
Project Indus — fine-tuned Hindi LLM
1.2B parameters · 37 dialects · trained on 22B tokens.
Inference layer
Bud Runtime + inference stack
Custom Intel × Bud build for portable, production-ready serving.
Intel® AMXIntel® AVX-512Sub-NUMA scale-out
Hardware
5th Gen Intel® Xeon® processors
NUMA-aware memory layout for efficient parallel processing.
Figure 1. The Indus inference stack. The solution is optimized to leverage Intel® AMX and Intel® AVX-512, with NUMA-architecture considerations incorporated to resolve memory-performance bottlenecks.

The build was validated and benchmarked using the LLMPerf library, running load tests that measure inter-token delay and generation throughput for both individual and concurrent queries.

Bud Ecosystem · Case Study03
Benchmarking Project Indus on Intel® AI Hardware Bud
The Engine

What Bud Runtime brings.

Intel collaborated with Bud Ecosystem to develop a custom version of the model, integrating Bud Runtime and an inference stack to enhance functionality. Five capabilities carry the solution.

1

High throughput & concurrency

Supports portable, production-ready LLM applications with high throughput and concurrency — proven on Intel® Xeon® processors across the 4th, 5th, and 6th generations for efficient large-scale handling.

2

State-of-the-art optimization

Advanced optimization methods enhance both performance and security, targeting the best total cost of ownership (TCO) and return on investment (ROI) for performance-driven applications.

3

Enhanced performance

Uses Intel® Advanced Matrix Extensions (AMX) and Intel® Advanced Vector Extensions 512 (AVX-512) to improve inference-serving performance, optimizing computational efficiency and data processing.

4

Scalable architecture

Supports 'Across the Box Scale-Out' and 'In-Box Scale-Out' approaches with Sub-NUMA clusters, addressing memory-performance issues and enabling efficient parallel application processing.

5

Unified API & toolchain

Provides a single API set and toolchain for seamless integration and deployment, streamlining the development and management of LLM applications.

Bud Ecosystem · Case Study04
Benchmarking Project Indus on Intel® AI Hardware Bud
The Evidence

Benchmark Results

Throughput, latency, time-to-first-token and scalability — measured on Intel® Xeon® with the LLMPerf load-test suite.

Benchmarking Project Indus on Intel® AI Hardware Bud
Key Results

Robust, versatile, efficient.

The Indus LLM was benchmarked extensively on the Intel platform across the critical inference metrics — TTFT, inter-token latency, prompt lengths, and throughput — at varying levels of concurrency.

33.998
Avg throughput (tok/s)
3.07s
Avg end-to-end latency
0.249–4.27s
Response, 22–167 tokens
≈ GPU
Inference parity

Latency profile

MetricConditionValue
Average throughputsteady state33.998 tok/s
End-to-end latencyaverage3.07 s
Total response time22–167 tokens0.249–4.27 s
Time to First Token2 NUMA nodes · 100 concurrent8.03 s
Time to First Tokenp90 · 200 concurrent17.18 s
Bud Ecosystem · Case Study05
Benchmarking Project Indus on Intel® AI Hardware Bud

Performance comparison

Indus vs. GPU

The performance of the Indus LLM is comparable to GPU inference — confirming that a CPU-first path on Intel® Xeon® can serve a production Hindi LLM without conceding throughput.

Scalability & responsiveness

  • Scalability. Performance scales effectively with the number of NUMA nodes, maintaining high throughput even as concurrent requests increase.
  • Responsiveness. The model handles diverse linguistic tasks with low inter-token latency and consistent performance across varied input and output prompt lengths.
Bottom line

Overall, the Indus LLM proves to be a robust, versatile, and efficient model for natural language processing — demonstrating strong scalability and performance on the Intel platform, and a viable, sustainable route to serving India's languages at scale.

Bud Ecosystem · Case Study06