Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI Perspective
Rakshit Aralimatti, Syed Abdul Gaffar Shakhadri, Kruthika KR, Kartik Basavaraj Angadi
2025-03-06
Summary
This paper talks about Shakti, a series of small AI language models designed to work efficiently on devices like smartphones and IoT systems, while still performing well on tasks in areas like healthcare, finance, and law.
What's the problem?
Large AI language models require too much computational power, energy, and memory to run on smaller devices like phones or smart appliances. They also raise privacy concerns because they often need to process data in the cloud instead of on the device itself.
What's the solution?
The researchers developed Shakti models with fewer parameters and optimized them for edge devices. They used techniques like quantization, which reduces the memory and energy needed to run the models, and fine-tuned them for specific tasks. Despite being smaller, these models perform as well as larger ones in many areas by focusing on efficiency and accuracy.
Why it matters?
This matters because it makes advanced AI tools more accessible and practical for everyday use on personal devices. It also ensures better privacy since the AI can work directly on the device without sending data to external servers. This could lead to smarter apps and tools that are faster, safer, and more energy-efficient.
Abstract
Deploying large scale language models on edge devices faces inherent challenges such as high computational demands, energy consumption, and potential data privacy risks. This paper introduces the Shakti Small Language Models (SLMs) Shakti-100M, Shakti-250M, and Shakti-500M which target these constraints headon. By combining efficient architectures, quantization techniques, and responsible AI principles, the Shakti series enables on-device intelligence for smartphones, smart appliances, IoT systems, and beyond. We provide comprehensive insights into their design philosophy, training pipelines, and benchmark performance on both general tasks (e.g., MMLU, Hellaswag) and specialized domains (healthcare, finance, and legal). Our findings illustrate that compact models, when carefully engineered and fine-tuned, can meet and often exceed expectations in real-world edge-AI scenarios.