Skip to content

πŸš€ Zero to Hero Index

Welcome to the Local AI β€œZero to Hero” guide. This vault is designed as a digital garden: don’t read it linearly. Follow your needs, click on concepts you haven’t mastered yet, and build your expertise step by step.


πŸ”’ Why run AI on-premises?

You may already use ChatGPT or Claude daily. Why bother hosting a model yourself?

1. Data sovereignty Every prompt sent to a cloud service is processed on third-party servers, often outside the European Union. In medical practices, legal departments, public administration, or industrial companies, some data cannot legally leave your premises β€” GDPR, professional secrecy, classification. On-premise AI is the only technical answer to that constraint. For a 6-criteria audit grid and the GDPR/AI Act framework, see πŸ”’ Sovereignty & Privacy.

2. Predictable cost at scale Cloud APIs bill per request. At low volume, that’s negligible. With 50 employees querying an assistant all day, the monthly bill can exceed the amortized cost of a dedicated machine within a few months.

3. Full control: model, behavior, availability On-premises, you choose the model, customize it, run without an Internet connection, and availability does not depend on a third party’s pricing or terms of service.


πŸ—ΊοΈ Vault contents

πŸ“ 01 - Foundations (The physics of AI)

Understand why a €4,000 computer can still be too slow for AI.

πŸ“ 02 - Hardware

A catalog of architectures.

πŸ“ 03 - Software stack (The engines)

How to bring the silicon to life.

πŸ“ 04 - Architecture blueprints (Scenarios)

Ready-to-propose configurations for your clients.

πŸ“ 05 - On-Premise Agents & Assistants

The application layer: AI that knows you, and AI that acts for you.

πŸ“ 06 - Practical implementation

How to test and decide before deploying.


🧭 Which blueprint should you choose?

Have a concrete need? This table points you in the right direction in 30 seconds.

Concurrent usersModel sizeHardware budget→ Blueprint
1 (dev / test)8–14B€1,500 to €3,700 (Oct. 2026; > €6,000 with a new RTX 5090)πŸ› οΈ A β€” Dev Lab
1 (dev / test)70B with offloading€1,500 to €2,500 (used 24 GB GPU, Oct. 2026)πŸ› οΈ A β€” Dev Lab (limited performance)
2–20 (SME, team of 10–50 people)70B€3,900 to over €8,000 incl. VAT (Oct. 2026, rising)🏒 B β€” SME Appliance
2–20 (SME, team)200B+ / MoE€10,000–22,000πŸ–₯️ C β€” Desktop Cluster
50+ (production)70B–400B> €280,000🏭 D β€” Datacenter
SLA constraint < 500 msAnyβ€”πŸ­ D β€” Datacenter

🚢 I’m new here: where do I start?

First visit? Follow this path before exploring freely:

  1. LLM (what is a large language model?)
  2. LLM inference (how you use it day to day)
  3. 🧠 The Journey of a Prompt (what really happens when you talk to it)
  4. 🏎️ Memory Bandwidth (why your machine can feel slow)
  5. πŸ› οΈ Scenario A: Dev Lab (your first concrete blueprint)

πŸ“– Dictionary & concepts

Quick definitions for all technical terms. πŸ‘‰ AI Glossary

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.