Skip to content

LLM

πŸ“ Short definition

AI model trained on very large text corpora, able to understand and generate text.

πŸ“– Detailed definition

Modern LLMs mostly use the Transformer architecture. Once trained, they mainly serve inference: on-demand text generation from a prompt. Practical local performance depends as much on memory hardware as on model size and quantization.

πŸ’‘ Why it matters for on-prem AI

Central stack component: all sizing (RAM/VRAM, throughput, latency) follows from it, especially for local inference β€” rarely for training from scratch.

⚠️ Common pitfalls

  • Assuming a β€œbigger model” is always better for every use case.
  • Forgetting performance depends as much on machine memory as on the model.

πŸ“š Go deeper

Want to see what an LLM does when you talk to it?

  1. 🧠 The Journey of a Prompt (tokenisation, prefill, decoding β€” full cycle)
  2. LLM inference (day-to-day use of a trained LLM)
  3. πŸ—œοΈ 4-bit & 8-bit quantization (shrinking a model to fit memory)

πŸ”— See also

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.