What is a small language model?
A small language model is a language model with fewer parameters than the large models in full-size chat apps. It can be cheaper or able to run on modest hardware. It is often weaker on messy instructions. Parameters are the model's learned settings, not knobs you tune like temperature.
Also called: local LLM, parameters
In plain English
Smaller can mean faster and less spend. It can also mean it drops a constraint you wrote. Local LLM usually means this kind of model running on hardware you control. Size and license are separate questions. Judge it on your messages, not on a parameter count from an ad. Smaller can be the right tool for a narrow job: sort these ten labels, extract these four fields, run on a machine you control. It will be worse at open-ended advice. Try it on your real notes, not on a leaderboard. If it misses the weird messages, keep the larger model for those and the small one for the repetitive sort.
How an owner would use it
If a task is a tight classification with a few labels, a smaller model may be enough. If the task is a nuanced customer reply, compare it with the model you already trust on the same ten messages. Do not switch the whole shop because a number was lower.
Common mistakes
Buying on parameter count is astrology. A local model does not automatically keep data safer if the laptop is shared and unencrypted. Do not put a weaker model on the path that quotes prices. Do not assume small means unlimited use. The host still has limits.
A worked example
An ice cream shop
Flavors can be labeled small, complaints cannot
This is an illustration, not a customer result. The owner tests a smaller model on sorting inbox notes into catering versus daily hours. It matches the lead's sort on a short stack. On a complaint about an event, it misses the tone. Complaints stay with the lead. The labeler is only a sort.
Related terms
Prompts and guides
Questions owners ask
What is a small language model?
It is a language model built to be lighter than the largest chat models, often cheaper or able to run on smaller hardware, and often less reliable on messy tasks.
How do I tell if a smaller model is good enough?
Run the same real messages through it and through your current model. Keep the smaller one only where the outputs match what a person would accept.
What are parameters?
Parameters are the internal settings learned in training. More of them does not give you a dial for your prices. Temperature and your prompt are the controls you can actually set.
Product guides are on AI apps for business.