OpenViking · Semantic Memory for AI Agents ⬇ Download Markdown

The Practical Guide

How to store, search, and maintain a durable knowledge base with OpenViking — the semantic-vector memory layer that makes an AI agent actually remember.

2core skills, fully documented
2storage tiers: resources & memories
6mistake classes you can stop repeating
14related skills worth loading
00 · Executive Summary

Should you use OpenViking?

OpenViking is a durable, semantic-vector memory store. An agent writes facts, events, and digests into it once; every future session can retrieve them by meaning — not by exact text match. If your agent forgets things between sessions, this is the fix. Here is the honest scoreboard first, detail below.

Key benefits what you gain

  • Durable memory that survives sessionspersistence
  • Semantic search — find by meaning, not keywordrecall
  • Free tier: operational data stored without token costcost
  • Deterministic writes — no LLM hallucination in cronreliability
  • Mistake capture loop — stop repeating the same errorlearning
  • Auto-extraction of 6 categories per sessionautomation

Drawbacks know before you build

  • No time decay — newest is not highest-rankedsearch
  • Deep search mode is flaky (1 of 4 succeeds)search
  • Writes need explicit directory creation firstAPI
  • Python SDK is broken — HTTP API is the only sane pathAPI
  • Root API key silently blocks all data accessauth
  • LLM tool calls in cron hallucinate successreliability
Bottom line: OpenViking is worth it if your agent needs durable recall and you follow the deterministic-write discipline. It is not a fire-and-forget system — the taxonomy, the two stores, and the write modes must be understood up front or you will hit silent failures (writes that land but never become searchable).
01 · Core Concepts

What OpenViking is

OpenViking is a context database for AI agents: a vector-backed store with an HTTP API. Two kinds of data live in it — resources (large operational digests, cheap to keep) and memories (high-signal facts that get injected into relevant conversations). The diagram below shows the full write/read path.

AI AGENT viking_search (semantic) viking_remember (write) viking_read / browse HTTP content/write HTTP content/read Cron scripts write via HTTP directly — no LLM tool calls ✓ deterministic OPENVIKING api_key auth · port 1933 semantic index (embeddings) fs/mkdir · reindex · health Two stores inside: default store — canonical entities, resources peer store — session facts (viking_remember target) DEFAULT STORE canonical entities resources/ · memories/entities/ HTTP content/write target PEER STORE session-extracted facts hash filenames (mem_*.md) viking_remember target — viking tool write/read - - HTTP API (canonical writes) - - - reindex (make searchable)
Figure 1 — The agent writes through two paths: viking_remember → peer store (hash files), HTTP content/write → default store (canonical). Both are searchable; bulk imports need content/reindex.
Two stores — the #1 confusion

viking_remember writes to the peer store with hash filenames. The HTTP content/write API writes to the default store. They are separate. Searching with the tool finds peer-store memories; querying the API directly finds default-store resources. If one search finds nothing, check the other store — that is expected behavior, not data loss.

02 · Storage Model

Resources vs Memories

The single most important decision in OpenViking: where a write goes determines whether it costs tokens later. Memories are injected into relevant conversations — that is a feature when the fact matters, a tax when the data is routine.

Data typeScopeToken costNaming
Daily news digestresources/Nonenews-YYYY-MM-DD.md
Market reports (macro, anomalies, Fed)resources/Nonemarket-YYYY-MM-DD-<type>.md
Congress / SEC disclosuresresources/Nonemarket-YYYY-MM-DD-congress.md
Mistakes & root causesmemories/cases/Injectedauto-hash
Investment thesesmemories/theses/Injectedtheses/<ticker>/
High-signal discrete factsmemories/Injecteddescriptive name
📦

resources/ — the cheap tier

Operational output: digests, reports, market data. Stored and searchable but never injected into conversations. Zero token tax on recall.

  • Large daily files belong here
  • Use write_resource()
  • Date in filename = freshness
🧠

memories/ — the signal tier

Facts the agent should carry into future conversations: preferences, corrections, theses, mistakes. Each one burns tokens when relevant.

  • High-signal facts only
  • Use write_memory()
  • Consolidate — one digest beats 47 fragments
🗑️

Never store here

Some things belong elsewhere entirely — not in OpenViking at all.

  • Raw session transcripts → SQLite FTS5
  • Easily re-discovered facts
  • Routine daily noise
  • Procedures → skills, not OpenViking
03 · Write Discipline

Taxonomy — the rules that keep it sane

Two violations cause most long-term pain. Fragmentation: 10 tiny hash files where one digest belongs. Wrong scope: operational data in memories/, silently taxing every relevant conversation. The decision tree:

New data to store operational / dated? fact / preference / mistake? YES → resources/ digest naming, no token cost YES → memories/ facts auto-injected when relevant Consolidate: one digest > 47 fragments
Figure 2 — Scope decision tree. When in doubt, ask: would this fact help a future conversation? If yes, memory. If it is output, resource.
Superseded entities must retire

Writing a corrected entity while the stale one lives creates ontology drift: semantic search returns both, and nothing flags the contradiction. When you correct a fact, either delete the old file or mark it Status: superseded by <uri>. Never leave two entities asserting different values for the same attribute.

Canonical entity conventions

Human-readable filename. supersedes: links on rewrites. source_of_truth: pointers for live values — ports, versions, counts, models. Never copy a queryable number into an entity; store the pointer and the rule, not the data. A Last verified: date on every entity.

05 · Models in the Stack

Which models power the memory

OpenViking uses two model roles. Knowing them matters because each is a different failure surface, and the /ready probe reports on exactly one of them.

RoleModelServesFailure surface
EmbeddingsNomic (nomic-embed-text)Converts stored content into vectors; powers semantic similarity rankingIf /ready shows "ollama": "not_configured", embeddings never run — writes land but are not searchable
Search / extractionDeepSeek V4 (optionally served via vLLM)Semantic query understanding and the optional LLM extraction pass (semantic_status)Slow responses under load; model provider outages stall the extraction queue
What this means operationally

Embeddings are the load-bearing piece: no embeddings, no search. The nightly monitor checks the queue drains; the readiness probe checks Ollama is configured. The search/extraction model is swappable — any OpenAI-compatible model can serve that role (vLLM is a common local host for it) — but the embedding model choice is baked into the vector index. Change embeddings and you must reindex everything.

06 · API Patterns

Reading and writing — the proven patterns

Use the HTTP API directly. The Python SDK has binary compatibility issues across Python versions; curl or urllib is the reliable path. Three write modes, one reindex endpoint, and the directory trap:

ModeMeaningResult
"create"New file — required for first writeOK, or 409 ALREADY_EXISTS
"replace"Overwrite an existing URI in placeOK
"overwrite"Does not exist — invalidHTTP 400. Never use.
# Write a new memory file — mode:"create" is mandatory for new URIs
curl -s -X POST -H "Authorization: Bearer $OV_KEY" \
  -H "Content-Type: application/json" \
  "http://<openviking-host>:1933/api/v1/content/write" \
  -d '{"uri": "viking://user/peers/hermes/memories/<category>/<file>.md",
       "content": "<content>", "mode": "create"}'

# Daily files written twice/day: try create, on 409 retry with replace
# (the shared write_resource() does exactly this)

# Read back
curl -s -H "Authorization: Bearer $OV_KEY" \
  "http://<openviking-host>:1933/api/v1/content/read?uri=viking://user/peers/hermes/memories/<category>/<file>.md"
The directory trap

content/write with mode:"create" does not auto-create parent directories. Writing to a new category path without first calling fs/mkdir returns File not found — the file is not created. Order: fs/mkdircontent/write.

Indexing: "skipped" is not a failure — but know the difference

A write response has semantic_status and vector_status. semantic_status:"skipped" is NORMAL (the optional LLM extraction pass is off). The real proof of indexing is vector_status:"complete" with queue_status.Embedding.processed > 0. If you bulk-imported via HTTP and files are not searchable, run content/reindex on the URI tree.

Read response shapes differ by store

Peers-store URIs return result as a string; default-store URIs return an object with .content. A naive parser reports peers files as "0 bytes" — parse defensively:

txt = res.get("result")
txt = txt.get("content", "") if isinstance(txt, dict) else (txt or "")
07 · Reliability

Cron writes: deterministic, never LLM tool calls

The biggest production lesson: LLM agents in cron hallucinate tool calls. They report "28 headlines stored" without a single write landing. The fix is structural — move all writes into Python scripts that call the HTTP API directly, and let the cron agent only run the script and report.

🚫

The failure mode

DeepSeek-class cron models skip viking_remember even when the toolset includes it — then claim success.

  • Reported success ≠ stored data
  • Audit every cron job's actual writes
  • Check created_at dates match the run
⚙️

The fix: deterministic writes

A shared writer module posts to the HTTP API directly. The cron prompt becomes: run the script, report output. No tool calls.

  • write_resource() → digests
  • write_memory() → high-signal facts
  • Do not pass tags — HTTP 400
🔗

One chained command

Never make a cron agent run a 3+ step script chain — it silently skips the tail. Expose one --stage all command.

  • Print a one-line summary per stage
  • Verify writes landed with a search
  • Convert any LLM-write job on evidence
Plugin required — silent Broken pipe

memory.provider: openviking enables the backend but does NOT register viking_remember. The plugin must also be in plugins.enabled. Missing it makes cron jobs fail with RuntimeError: [Errno 32] Broken pipe. Check grep openviking config.yaml — it must appear under plugins:, not only memory:.

08 · Learning Loop

The mistake feedback loop

An agent that cannot record its own errors repeats them. The loop: catch the correction → 5 Whys to root cause → store in OpenViking → search before acting next time. The key standard: holistic patterns, not one-off logs. Trace every mistake to a repeatable reasoning bias.

Mistake symptom Why?Why?Why?Why?Why? 5 Whys dig to root Root cause behavioral pattern viking_remember category: case The key question before any claim: "Would I bet money on this being true?"
Figure 4 — Every correction becomes a stored case with its 5-Whys chain and one-sentence root cause.
Store itSkip it
User corrections ("that's wrong", "actually it's X")Trivial typos / syntax errors
Task failure from a wrong assumptionEasily re-discovered facts
Repeated errors from the mistakes logSession-specific temporary state
Single-source citations presented as fact
Implausible numbers presented without flagging
Capture template

Every mistake stored as: Mistake5 Whys (why ×5 to root) → Root cause (one-sentence behavioral pattern) → Prevention (rule for next time). Also append to a local mistakes log for backup, and check for recurring patterns before storing: search OpenViking for similar mistakes first.

09 · Coding Applications

Making your coding agent learn

Six evidence-based patterns for a coding-enabled agent stack — proven on a production box, not theory.

📜

1 · Global AGENTS.md

The coding agent natively loads ~/.pi/agent/AGENTS.md every session (verified in shipped source: scans AGENTS.override.md → AGENTS.md → CLAUDE.md, then walks up from cwd for per-project files).

  • Environment-specific traps live here
  • Keep under ~600 tokens
  • Repo facts → per-project AGENTS.md
🧪

2 · Deterministic lesson capture

A tiny append script (dedup on first 60 chars, token-budget warning, exit codes) plus a mandatory capture step per work wave.

  • A wave that corrected you must produce a lesson
  • Judgment errors → mistakes-feedback-loop
🛡️

3 · Mechanical rails

For multi-agent concurrency: checkpoint auto-commit before mutations, destructive-op gate (block reset --hard, rm -rf on repos), ownership locks with stale TTL, revert-never-reset restore.

  • Enforce in code, not in prompts
  • Rules in code outrank model memory
🐴

4 · Anti-over-engineering default

A YAGNI-ladder extension (needs to exist? → in codebase? → stdlib? → one line? → minimum) on by default, with a review command returning a delete-list.

  • Guards style, complements safety rails
📋

5 · Spec-driven waves

Every build carries a spec template with constraints: verify-before-acting, do-not-touch paths, shared-path locks, "never cut validation/security/accessibility".

  • The spec is the primary rules channel
  • AGENTS.md is the failsafe for sloppy specs
🧩

6 · Skills ≠ OpenViking

Skills are procedural memory (how to do things); OpenViking is declarative (what things are). Never duplicate skills into OpenViking — stale copies with no linked files.

  • Store system-specific facts, not procedures

How this powers pi.dev in a real workflow

pi.dev is a minimal, stateless coding agent: it executes the spec it is given and remembers nothing between sessions. That is a feature — but it means all institutional knowledge must arrive in the prompt or the rails. OpenViking is the declarative layer that makes the loop self-improving:

HERMES (orchestrator) · recalls context from OpenViking · writes the wave spec · reviews pi.dev's output · runs capture step 6 OPENVIKING (declarative) · past decisions, mistakes, facts · environment reference, patterns · semantic recall — how the orchestrator remembers pi.dev (executor) · loads AGENTS.md every session · follows the spec constraints · rails: safety + YAGNI gates · writes code + tests AGENTS.md (procedural) · environment traps, git rules, model-drift insurance · loaded natively by pi.dev every session · <600 tokens ① semantic recall capture: judgment patterns ② spec + constraints ④ review ③ loaded every session capture: coding lessons → The loop: ① recall → ② spec → ③ execute → ④ review → capture → ① recall. Each wave starts smarter than the last.
Figure 5 — OpenViking feeds the orchestrator's memory; AGENTS.md feeds the executor's session. One loop, two memory channels, zero state in pi.dev itself.
Why two channels?

pi.dev is stateless by design — it cannot query OpenViking mid-task, so it needs its lessons injected as AGENTS.md (procedural, loaded every session). The orchestrator can query OpenViking, so it stores the declarative layer there: past decisions, environment facts, mistake patterns. The mandatory capture step is what connects them — every wave that corrected you produces a lesson that lands in one or both channels, so the next wave inherits it.

10 · Auth & Keys

The API key hierarchy — the #1 "it's broken" cause

Three key levels exist. The root key can manage accounts but cannot touch data — a gateway configured with the root key passes health checks while every read and write fails silently with PERMISSION_DENIED. This is the most common false alarm in the system.

Key typeCan doCannot do
RootManage accounts, create users, list accountsRead/write/search any user data
AdminRead/write/search for a scoped accountManage other accounts
UserRead/write/search own data onlyAdmin/root operations
# Create account + admin user in one call (returns the USER key)
curl -s -X POST -H "Authorization: Bearer $ROOT_KEY" \
  -H "Content-Type: application/json" \
  "http://<openviking-host>:1933/api/v1/admin/accounts" \
  -d '{"account_id": "hermes", "admin_user_id": "default"}'

# The user_key from that response is what the gateway must use.
# Never configure the root key in the gateway.
Key format

Keys look like base64(account).base64(user).signature. If the server reconfigures or rotates keys, the gateway's stored key goes stale: every request returns UNAUTHENTICATED: Invalid API Key. The gateway reads the key from its process environment (e.g. /run/s6/container_environment/OPENVIKING_API_KEY) — not from .env. Editing .env has no effect on the gateway.

11 · Troubleshooting

Fast diagnosis of the common failures

SymptomRoot causeFix
PERMISSION_DENIED: ROOT API keys...Gateway has a root keyReplace with the user/admin key from account creation
UNAUTHENTICATED: Invalid API KeyGateway key ≠ server key (rotation)Test with curl; update the gateway env key
Broken pipe in cronPlugin not in plugins.enabledAdd - openviking, restart gateway
"Server not connected" but curl worksovcli_config_path unreadableRemove the openviking: sub-block; restart
Search returns empty memoriesMissing target_uri scopeScope to memories/; verify with fs/ls first
fs/tree 401s on peers storetree unsupported thereUse fs/ls recursion for peers paths
Writes return "File not found"Parent dir not createdfs/mkdir before content/write
vector_status: queued foreverEmbedding worker stalledCheck /ready for ollama; trigger processing; restart
Readiness probe first

curl http://<openviking-host>:1933/ready — if "ollama": "not_configured", the embedding provider is not set (needs config + restart). If "ok", look elsewhere. Config changes never hot-reload — the server must restart.

Recovery after a container wipe

Without a correct volume mount, every restart wipes ALL data. Recovery order: verify wipe → fix volume (set storage.workspace in ov.conf, mount the data dir) → recreate account → save the user key → update the gateway → re-migrate memories, Tier 1 operational essentials first.

12 · The Skill Library

Related skills worth loading

These companion skills carry the full procedures behind this guide. Load them when the trigger condition appears.

SkillWhy it mattersLoad when
karpathy-guidelinesVerify-first discipline, surgical patches, honest failure reportingAny coding task
autonomous-ai-agents/piCoding agent usage: auth, extensions, lesson inheritanceDelegating code work
autonomous-ai-agents/delegate-waveWave orchestration: spec template, rails, mandatory captureMulti-agent coding waves
test-driven-developmentRED-GREEN-REFACTOR enforced before codeFeatures that must work
spec-driven-agent-buildsWhole-app builds with verifiable success criteriaApp builds via background agents
delegate-task-resilienceWrite-to-disk proof, caps, recovery for subagent batchesParallel subagent research
systematic-debugging4-phase root cause debugging: understand before fixingBugs surviving one fix attempt
requesting-code-reviewPre-commit review: security scan, quality gatesBefore non-trivial commits
agent-self-maintenanceMemory hygiene, storage-layer policy, data-source principlePruning persistent memory
hermes-configuration-patternsOpenViking external setup, provider options, config pitfallsDiagnosing provider config
session-conductSession management discipline for agent-user interactionsLong-running sessions
open-source-tool-vettingVet OSS tools pre-install: local-only, cloud-dep checkInstalling any new tool
system-backupFull backup/restore of config and user dataBefore risky migrations
mistakes-feedback-loopJudgment-error capture with 5 Whys → OpenVikingAfter corrections or failures
13 · Handoff Checklist

Before you consider a write done

  • Content consolidated — one digest, not fragmented tiny files
  • Correct scope — operational data in resources/, facts in memories/
  • Descriptive filename — never a bare hash for canonical entities
  • No duplicate writes — checked existing content first
  • Searchable after write — verified via viking_search or reindex
Compiled 2026-08-15 from a production-tested Hermes + OpenViking deployment. Replace <openviking-host> with your server's address. All examples are generic — no project, person, or system identifiers are referenced. For agents: download the full guide as markdown, or fetch llms.txt.