ποΈ Possible Architectures
Not all local AI applications do the same thing. Before choosing a tool, it helps to understand which architectural category it belongs to β and what that category implies for hardware, complexity, and sovereignty.
ποΈ Taxonomy: three main families
1. Personal Assistant (βAI that knows youβ)
Definition: A personal assistant is an interactive system. You ask questions; it answers using its memory (your documents, notes, past conversations).
Characteristics:
- Primary interaction: real-time text dialogue
- Memory: persistent, centered on your context (notes, files, past conversations)
- Trigger: a human asks a question
- Autonomy: low β it responds, it does not act
Examples: Open WebUI, Jan.ai, Khoj, AnythingLLM (OpenHuman, now an agent harness, belongs to the hybrid category)
Analogy: a colleague very well informed on your cases, available 24/7, but who waits for you to speak.
2. Custodian Agent (βAI that acts for youβ)
Definition: A custodian agent runs tasks autonomously on a trigger. It does not answer questions β it acts: reads files, detects issues, generates proposals, creates Git branches, waits for human validation.
Characteristics:
- Primary interaction: scheduled or event-driven trigger, then report
- Memory: task context (the vault, the repo, error logs)
- Trigger: cron, webhook, Git event, CLI command
- Autonomy: high for read/analysis, always human-in-the-loop for irreversible actions
Examples: OpenHands Agent Canvas (scheduled or event-driven automations, self-hosted), a Cursor CLI + systemd script pipeline; Aider remains usable but is frozen (see its page)1
Analogy: a junior research assistant who works overnight, leaves proposals on your desk in the morning, and signs nothing without your approval.
3. Hybrid (βAI that knows you and acts for youβ)
Definition: A combination of both. The assistant remembers your context and can trigger actions β web search, file updates, notifications β with or without validation depending on action risk level.
Characteristics:
- Can both answer and act
- Requires fine-grained permission and autonomy level management
- Higher complexity; risk of unwanted side effects if misconfigured
Examples: Open WebUI with tools and sub-agents (v0.11, July 2026)2, OpenHuman, Khoj (agent mode enabled), interactive conversations in OpenHands Agent Canvas
Warning: hybrid complexity is real. A poorly designed implementation can give the AI the ability to modify files, send email, or run commands without sufficient guardrails. Prefer an explicit architecture (assistant or custodian) to start.
Most of these tools expose their actions through the Model Context Protocol (MCP). Its July 28, 2026 revision is detailed in π§© RAG & Agents: check that the MCP servers and clients in your stack follow the same revision before stacking permissions.
π Comparison table of the three patterns
| Criterion | Personal Assistant | Custodian Agent | Hybrid |
|---|---|---|---|
| Interaction mode | Real-time dialogue | Batch / event-driven | Both |
| Trigger | Human | Cron / webhook | Human or automatic |
| Action autonomy | Low (responses) | High (tasks) | Variable |
| Memory required | Long, personal | Short, task context | Both |
| LLM model | Large (response quality) | Small OK (routing) + 14B+ coder (file editing) | Both |
| Minimum VRAM | 8β24 GB (7β14B model) | 12β16 GB (14B coder model minimum) | 24+ GB |
| Install complexity | Low to medium | Medium to high | High |
| Side-effect risk | Low | Medium (without guardrails) | High without guardrails |
| Sovereignty | Varies by tool | Controllable if open stack | Controllable if well architected |
π Relationship between the two tracks
The two tracks in this section are not competitors β they are complementary and can coexist on the same infrastructure.
flowchart TB
subgraph Machine["Your machine (or on-premise server)"]
A["**Track A β Personal Assistant**\nβ’ Knows you\nβ’ Answers your questions\nβ’ Long memory"] -->|"feeds"| B["**Track B β Custodian Agent**\nβ’ Maintains your vault\nβ’ Proposes fixes\nβ’ Creates branches/PRs\nβ’ Notifies you"]
A --> ENG["**Inference engine**\n(Ollama / vLLM)"]
B --> ENG
end
How they feed each other:
- The custodian agent keeps the vault up to date β the personal assistant has a fresh knowledge base to query.
- The personal assistant identifies unclear areas in your notes β the custodian agent can be triggered to enrich them.
- Both share the same inference engine β one Ollama or vLLM server is enough for both tracks.
- Since June 2026, Agent Canvas (OpenHands) can also orchestrate third-party agents via the Agent Client Protocol and assign a local model (vLLM, Ollama) per automation: Track B becomes an orchestration layer rather than a single agent1.
π§ Which architecture for which need?
| Your situation | Recommended architecture |
|---|---|
| You want a ChatGPT that knows your documents | Personal Assistant β Track A |
| You want to automate vault maintenance | Custodian Agent β Track B |
| You are starting out, hardware budget < β¬3,500 | Blueprint A + a simple assistant (Jan.ai or Open WebUI) |
| SME, 5β20 concurrent users | Blueprint B + Open WebUI or AnythingLLM |
| You want both (knows + acts) | Start with Track A, add Track B after validation |
| Production, multi-site, strict SLA | Blueprint D + controlled hybrid architecture |
π Hardware sizing
Both tracks share the same inference engine but do not have the same requirements.
| Track | Typical LLM model | Minimum VRAM | Comment |
|---|---|---|---|
| Personal Assistant (dialogue quality) | 14Bβ70B | 16β48 GB | Response quality matters β avoid < 7B |
| Custodian Agent (routing + editing) | 7B for routing only, 14B coder model minimum for editing files, 24β32B recommended (models: target architecture) | 12β24 GB | Routing does not need a large model; file editing does |
| Hybrid | 14Bβ70B | 24β48 GB | Compromise between the two |
For detailed sizing, see Blueprints AβD.
π» Getting started with code (external resources)
This guide covers architecture theory. To move to practice, here are the recommended entry points for each track:
Track A β Personal Assistant
| Tool | Starting point |
|---|---|
| Open WebUI | Official documentation β Docker install in 5 minutes, Ollama connection |
| AnythingLLM | AnythingLLM GitHub β full local RAG, multi-model interface |
| Khoj | Khoj self-hosting setup β personal memory + local file access (maintenance slowed since March 2026)3 |
Track B β Custodian Agent
| Tool | Starting point |
|---|---|
| Aider | Aider quickstart β local coding agent, Ollama-compatible; frozen project (see the Aider page), to be used only with full awareness |
| OpenHands Agent Canvas | Agent Canvas (README, Docker) β self-hosted control center for conversations and automations of coding agents (OpenHands, or Claude Code / Codex / Gemini CLI via ACP); the SDK and the agent live in software-agent-sdk1 |
| LiteLLM + Ollama | LiteLLM proxy quickstart β unified routing to a local model; require version β₯ 1.106.9 / 1.108.6 / 1.109.1 depending on your line (security advisory: see the LiteLLM page) |
| SmolAgents | SmolAgents cookbook β minimal agent framework, HuggingFace (reduced activity since May 2026)4 |
| LangGraph | LangGraph βlocal agentβ tutorial β agent orchestration with state graphs |
π See also
- π Sovereignty & Privacy β tool evaluation grid
- π§βπΌ On-Premise Personal Assistants
- π€ On-Premise Custodian Agents
- Autonomous agent Β· RAG
π Sources and References
Footnotes
-
Aider-AI, aider (GitHub repository: last commit on 2026-05-22, latest release v0.86.0 of 2025-08-09), accessed 2026-10-10; OpenHands, Introducing Agent Canvas (2026-06-16) and Use any coding agent in OpenHands with ACP (2026-06-18). https://github.com/Aider-AI/aider Β· https://www.openhands.dev/blog/introducing-agent-canvas Β· https://www.openhands.dev/blog/use-any-coding-agent-in-openhands-with-acp β© β©2 β©3
-
Open WebUI, Release v0.11.0 (sub-agents, LDAP group synchronization), 2026-07-27. https://github.com/open-webui/open-webui/releases/tag/v0.11.0 β©
-
Khoj, Self-Host (Docker / pip installation), read on 2026-10-10; khoj-ai, khoj β Releases (latest version 2.0.0-beta.28 of 2026-03-26). https://docs.khoj.dev/get-started/setup Β· https://github.com/khoj-ai/khoj/releases β©
-
Hugging Face, smolagents β Releases (latest version v1.26.0 of 2026-05-29, repository not archived), accessed 2026-10-10. https://github.com/huggingface/smolagents/releases β©