Skip to content

πŸ—οΈ Possible Architectures

Not all local AI applications do the same thing. Before choosing a tool, it helps to understand which architectural category it belongs to β€” and what that category implies for hardware, complexity, and sovereignty.


πŸ—‚οΈ Taxonomy: three main families

1. Personal Assistant (β€œAI that knows you”)

Definition: A personal assistant is an interactive system. You ask questions; it answers using its memory (your documents, notes, past conversations).

Characteristics:

  • Primary interaction: real-time text dialogue
  • Memory: persistent, centered on your context (notes, files, past conversations)
  • Trigger: a human asks a question
  • Autonomy: low β€” it responds, it does not act

Examples: Open WebUI, Jan.ai, Khoj, AnythingLLM (OpenHuman, now an agent harness, belongs to the hybrid category)

Analogy: a colleague very well informed on your cases, available 24/7, but who waits for you to speak.


2. Custodian Agent (β€œAI that acts for you”)

Definition: A custodian agent runs tasks autonomously on a trigger. It does not answer questions β€” it acts: reads files, detects issues, generates proposals, creates Git branches, waits for human validation.

Characteristics:

  • Primary interaction: scheduled or event-driven trigger, then report
  • Memory: task context (the vault, the repo, error logs)
  • Trigger: cron, webhook, Git event, CLI command
  • Autonomy: high for read/analysis, always human-in-the-loop for irreversible actions

Examples: OpenHands Agent Canvas (scheduled or event-driven automations, self-hosted), a Cursor CLI + systemd script pipeline; Aider remains usable but is frozen (see its page)1

Analogy: a junior research assistant who works overnight, leaves proposals on your desk in the morning, and signs nothing without your approval.


3. Hybrid (β€œAI that knows you and acts for you”)

Definition: A combination of both. The assistant remembers your context and can trigger actions β€” web search, file updates, notifications β€” with or without validation depending on action risk level.

Characteristics:

  • Can both answer and act
  • Requires fine-grained permission and autonomy level management
  • Higher complexity; risk of unwanted side effects if misconfigured

Examples: Open WebUI with tools and sub-agents (v0.11, July 2026)2, OpenHuman, Khoj (agent mode enabled), interactive conversations in OpenHands Agent Canvas

Warning: hybrid complexity is real. A poorly designed implementation can give the AI the ability to modify files, send email, or run commands without sufficient guardrails. Prefer an explicit architecture (assistant or custodian) to start.

Most of these tools expose their actions through the Model Context Protocol (MCP). Its July 28, 2026 revision is detailed in 🧩 RAG & Agents: check that the MCP servers and clients in your stack follow the same revision before stacking permissions.


πŸ“Š Comparison table of the three patterns

CriterionPersonal AssistantCustodian AgentHybrid
Interaction modeReal-time dialogueBatch / event-drivenBoth
TriggerHumanCron / webhookHuman or automatic
Action autonomyLow (responses)High (tasks)Variable
Memory requiredLong, personalShort, task contextBoth
LLM modelLarge (response quality)Small OK (routing) + 14B+ coder (file editing)Both
Minimum VRAM8–24 GB (7–14B model)12–16 GB (14B coder model minimum)24+ GB
Install complexityLow to mediumMedium to highHigh
Side-effect riskLowMedium (without guardrails)High without guardrails
SovereigntyVaries by toolControllable if open stackControllable if well architected

πŸ”— Relationship between the two tracks

The two tracks in this section are not competitors β€” they are complementary and can coexist on the same infrastructure.

flowchart TB
    subgraph Machine["Your machine (or on-premise server)"]
        A["**Track A β€” Personal Assistant**\nβ€’ Knows you\nβ€’ Answers your questions\nβ€’ Long memory"] -->|"feeds"| B["**Track B β€” Custodian Agent**\nβ€’ Maintains your vault\nβ€’ Proposes fixes\nβ€’ Creates branches/PRs\nβ€’ Notifies you"]
        A --> ENG["**Inference engine**\n(Ollama / vLLM)"]
        B --> ENG
    end

How they feed each other:

  • The custodian agent keeps the vault up to date β†’ the personal assistant has a fresh knowledge base to query.
  • The personal assistant identifies unclear areas in your notes β†’ the custodian agent can be triggered to enrich them.
  • Both share the same inference engine β†’ one Ollama or vLLM server is enough for both tracks.
  • Since June 2026, Agent Canvas (OpenHands) can also orchestrate third-party agents via the Agent Client Protocol and assign a local model (vLLM, Ollama) per automation: Track B becomes an orchestration layer rather than a single agent1.

🧭 Which architecture for which need?

Your situationRecommended architecture
You want a ChatGPT that knows your documentsPersonal Assistant β†’ Track A
You want to automate vault maintenanceCustodian Agent β†’ Track B
You are starting out, hardware budget < €3,500Blueprint A + a simple assistant (Jan.ai or Open WebUI)
SME, 5–20 concurrent usersBlueprint B + Open WebUI or AnythingLLM
You want both (knows + acts)Start with Track A, add Track B after validation
Production, multi-site, strict SLABlueprint D + controlled hybrid architecture

πŸ“ Hardware sizing

Both tracks share the same inference engine but do not have the same requirements.

TrackTypical LLM modelMinimum VRAMComment
Personal Assistant (dialogue quality)14B–70B16–48 GBResponse quality matters β€” avoid < 7B
Custodian Agent (routing + editing)7B for routing only, 14B coder model minimum for editing files, 24–32B recommended (models: target architecture)12–24 GBRouting does not need a large model; file editing does
Hybrid14B–70B24–48 GBCompromise between the two

For detailed sizing, see Blueprints A–D.


πŸ’» Getting started with code (external resources)

This guide covers architecture theory. To move to practice, here are the recommended entry points for each track:

Track A β€” Personal Assistant

ToolStarting point
Open WebUIOfficial documentation β€” Docker install in 5 minutes, Ollama connection
AnythingLLMAnythingLLM GitHub β€” full local RAG, multi-model interface
KhojKhoj self-hosting setup β€” personal memory + local file access (maintenance slowed since March 2026)3

Track B β€” Custodian Agent

ToolStarting point
AiderAider quickstart β€” local coding agent, Ollama-compatible; frozen project (see the Aider page), to be used only with full awareness
OpenHands Agent CanvasAgent Canvas (README, Docker) β€” self-hosted control center for conversations and automations of coding agents (OpenHands, or Claude Code / Codex / Gemini CLI via ACP); the SDK and the agent live in software-agent-sdk1
LiteLLM + OllamaLiteLLM proxy quickstart β€” unified routing to a local model; require version β‰₯ 1.106.9 / 1.108.6 / 1.109.1 depending on your line (security advisory: see the LiteLLM page)
SmolAgentsSmolAgents cookbook β€” minimal agent framework, HuggingFace (reduced activity since May 2026)4
LangGraphLangGraph β€œlocal agent” tutorial β€” agent orchestration with state graphs

πŸ”— See also

πŸ“š Sources and References

Footnotes

  1. Aider-AI, aider (GitHub repository: last commit on 2026-05-22, latest release v0.86.0 of 2025-08-09), accessed 2026-10-10; OpenHands, Introducing Agent Canvas (2026-06-16) and Use any coding agent in OpenHands with ACP (2026-06-18). https://github.com/Aider-AI/aider Β· https://www.openhands.dev/blog/introducing-agent-canvas Β· https://www.openhands.dev/blog/use-any-coding-agent-in-openhands-with-acp ↩ ↩2 ↩3

  2. Open WebUI, Release v0.11.0 (sub-agents, LDAP group synchronization), 2026-07-27. https://github.com/open-webui/open-webui/releases/tag/v0.11.0 ↩

  3. Khoj, Self-Host (Docker / pip installation), read on 2026-10-10; khoj-ai, khoj β€” Releases (latest version 2.0.0-beta.28 of 2026-03-26). https://docs.khoj.dev/get-started/setup Β· https://github.com/khoj-ai/khoj/releases ↩

  4. Hugging Face, smolagents β€” Releases (latest version v1.26.0 of 2026-05-29, repository not archived), accessed 2026-10-10. https://github.com/huggingface/smolagents/releases ↩

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.