Inference
The act of actually running a model to produce an answer. Training builds the model once; inference is every time you use it afterward.
Inference is what you pay for per use and what determines speed. It is the running cost of AI.
Frequently asked questions
What does inference mean when people talk about AI?
Inference is simply the act of running a finished model to get an answer, which is what happens every single time you use an AI. It is separate from training, the one-time process that built the model in the first place.
Is inference what I am paying for?
Yes. Training is a one-time cost the model maker paid, while inference is the running cost charged each time you send a request and get a response. It is also what determines how fast an answer comes back.
Why are some AI responses slower than others?
Inference speed depends on the model and how much work it does per answer, so a bigger or reasoning-focused model that thinks more will take longer than a lean one. There is a real trade-off between speed and depth of thinking.
New to all this? Start with what an AI agent really is, browse the full glossary, or explore the learning hub.
← Back to the glossary