syntaxbullet/kimiko-agent
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(memory): add memory extension with core functionality and tests

Browse Source 
- Implemented MemoryStore class for managing concepts with YAML frontmatter.
- Added methods for creating, reading, updating, and deleting concepts.
- Introduced locking mechanism for concurrent access.
- Developed search and hinting capabilities for concept retrieval.
- Created tests for access control and store functionality.
- Added long-term memory skill documentation and operational guidelines.
- Included package.json and package-lock.json for dependency management.
This commit is contained in:
syntaxbullet
2026-07-23 20:35:08 +02:00
parent 212af9301e
commit 91e5f44cc8
13 changed files with 1333 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@@ -10,7 +10,9 @@ models-store.json
npm/ npm/
git/ git/
# Runtime files # Runtime files and local personal memory
node_modules/
/memory/
pi-debug.log pi-debug.log
*.log *.log
.DS_Store .DS_Store

31
README.md
View File

@@ -4,7 +4,13 @@ This directory is the user/config directory for an isolated [Pi coding agent](ht
## Start Pi ## Start Pi
From this directory: Install the memory extension's pinned YAML parser after cloning:
```bash
npm ci --prefix extensions/memory
```
Then start from this directory:
```bash ```bash
./launch ./launch
@@ -46,8 +52,9 @@ Or use provider API-key environment variables before starting Pi. `auth.json`, t
- `settings.json` — instance-wide Pi settings - `settings.json` — instance-wide Pi settings
- `models.json` — custom providers/models - `models.json` — custom providers/models
- `keybindings.json` — keybinding overrides - `keybindings.json` — keybinding overrides
- `extensions/` — global extensions for this instance - `extensions/` — global extensions for this instance, including deterministic OKF memory tools
- `skills/` — global skills for this instance - `skills/` — global skills for this instance, including progressively disclosed memory guidance
- `memory/` — the human-readable OKF long-term-memory bundle
- `prompts/` — global prompt templates for this instance - `prompts/` — global prompt templates for this instance
- `themes/` — global themes for this instance - `themes/` — global themes for this instance
- `sessions/` — generated conversation history - `sessions/` — generated conversation history
@@ -62,6 +69,24 @@ Install a package into only this instance by using the launcher:
./launch list ./launch list
``` ```
## Long-term memory
Kimiko stores durable memory as an OKF v0.1 bundle in `memory/`. It is ignored by Git by default because it may contain personal data; remove that ignore rule only after choosing an explicit private backup/versioning policy. Each user turn resets detailed memory tools. Explicit memory language reveals them for that turn so the model can see the capability; otherwise `memory_search` progressively enables them. Conservative metadata hints may be added to that turn's system prompt, but are never written to session history and disappear on the next turn. Existing concepts cannot be updated or forgotten until they have been returned by search and read in the same turn. The extension owns frontmatter, timestamps, path validation, generated indexes, logs, locking, atomic writes, verification, repair, and one retry. The model supplies only semantic content.
```text
/memory-path Show the bundle location
/memory-check Validate without changing concepts
/memory-check --repair Repair structure and regenerate indexes
```
Run the deterministic store tests with:
```bash
npm test --prefix extensions/memory
```
See [`docs/memory-design.md`](docs/memory-design.md) for scope, tradeoffs, and staged production improvements.
## Isolation boundary ## Isolation boundary
`PI_CODING_AGENT_DIR` isolates Pi's user config, credentials, sessions, model config, and managed packages from `~/.pi/agent`. It is not an OS sandbox. Pi can still use inherited environment variables and shell/user configuration, and—when trusted—load project-local `.pi` resources. Pi also discovers the cross-agent global skill location `~/.agents/skills`; use `--no-skills` when you need to suppress all automatic skill discovery for a run. `PI_CODING_AGENT_DIR` isolates Pi's user config, credentials, sessions, model config, and managed packages from `~/.pi/agent`. It is not an OS sandbox. Pi can still use inherited environment variables and shell/user configuration, and—when trusted—load project-local `.pi` resources. Pi also discovers the cross-agent global skill location `~/.agents/skills`; use `--no-skills` when you need to suppress all automatic skill discovery for a run.

1
SYSTEM.md
View File

@@ -1 +1,2 @@
You are a personal assistant named Kimiko. You are a personal assistant named Kimiko.
Use memory_search before relying on prior conversations or changing durable memory.

90
docs/memory-design.md Normal file
View File

@@ -0,0 +1,90 @@
# Kimiko long-term memory design
## Decision
Use OKF Markdown as the source of truth and a small Pi extension as the deterministic producer/consumer. Do not add a database, embeddings, autonomous summarization, or automatic Git commits in the first version.
```text
User request
-> memory_search (metadata-first lexical retrieval)
-> memory_read (selected bodies)
-> model chooses semantic content
-> memory_write / memory_forget
-> deterministic storage, validation, indexing, and logging
```
This keeps memory inspectable, portable, diffable, and useful without a service.
## Responsibility boundary
### Model
- Decides whether information is durable and relevant.
- Chooses atomic concepts, stable IDs, types, summaries, tags, Markdown bodies, links, and citations.
- Resolves semantic contradiction or ambiguity with the user.
### Extension
- Enforces safe bundle-relative lowercase IDs and reserved filenames.
- Parses YAML, preserves unknown metadata on valid updates, and requires the local producer profile's `type` and `timestamp`.
- Generates timestamps from the system clock only after meaningful changes; no-op writes preserve timestamps.
- Maintains marked generated sections in indexes while retaining human text outside them.
- Maintains the root log.
- Serializes writes under a filesystem lock, writes through an atomic rename, reads back and semantically verifies output, then retries once.
- Repairs missing metadata from deterministic signals. Missing timestamps use filesystem mtime; missing types use `Recovered Concept`.
- Backs up malformed documents under `.recovery/`, preserves recoverable bodies, and marks repaired concepts for review.
The model never writes frontmatter, timestamps, indexes, or logs and is not asked to validate its own formatting.
## Retrieval and context budget
Each user turn resets access state and removes detailed memory tools. Explicit memory language reveals those tools for that turn so weaker models know the capability exists; otherwise only `memory_search` remains and dynamically enables `memory_read`, `memory_write`, and `memory_forget`. Visibility is not authorization: existing concepts must be returned by search and read in the same turn before update or deletion, and new concepts require a current-turn duplicate search. Search returns metadata rather than bodies; full reads are limited to ten concepts and 45,000 characters.
Before the model runs, deterministic lexical retrieval may add at most two conservative metadata candidates to that turn's system prompt. Hints require distinctive metadata matches, contain no bodies, do not authorize mutation, are not session messages, and disappear when the next turn rebuilds the system prompt.
Retrieval is weighted deterministic lexical matching over path, title, description, tags, type, and body, with stop-word filtering. This is intentionally simple and adequate for a personal bundle containing hundreds or a few thousand concepts. The root and directory indexes remain useful to humans even though search scans files directly.
The always-loaded instruction is one line in `SYSTEM.md`. Behavioral detail lives in the on-demand `long-term-memory` skill, and operational recovery detail is another level down in its reference document.
## Failure behavior
1. A mutation obtains a bundle-wide lock. Stale locks older than 30 seconds are removed.
2. The extension writes a same-directory temporary file, flushes it, and atomically renames it.
3. It reads the file back, parses it, checks required metadata, and compares semantic fingerprints.
4. Failed verification repeats the complete atomic write once from the structured input.
5. A second failure is reported as a tool error rather than delegated to the model.
6. Startup runs deterministic repair and index regeneration. `/memory-check` supports explicit validation; `--repair` applies the same repair path.
An interrupted operation can leave a concept updated before its index or log. Startup repair regenerates indexes. The root log is human-oriented, not a transactional journal; use an explicit external backup policy when durability beyond the local filesystem is required.
## Safety and privacy
- The skill prohibits credentials, tokens, transient chat, and unsupported inference.
- Forget requires the ID to be repeated exactly, deletes the current file and matching recovery backups, redacts its links from the root log, and records only an anonymous deletion event.
- No process auto-commits because Git history may retain data a user intended to forget.
- The bundle inherits host filesystem permissions; OKF itself is not an access-control system.
- `memory/` is ignored by Git by default to reduce accidental disclosure. Version it only after selecting a private remote, retention, and deletion policy.
- Repaired malformed source is retained in `.recovery/`; operators must remove those backups when handling a privacy deletion.
## Deliberately deferred
Add these only when observed scale or quality requires them:
1. **SQLite FTS5 cache** keyed by path and content hash for larger bundles. Markdown remains authoritative; the cache is disposable and rebuilt deterministically.
2. **Embeddings/reranking** only after lexical retrieval has measured misses. Store vectors outside OKF and version the embedding model.
3. **Contradiction workflow** with explicit supersession links and human confirmation; do not silently merge claims.
4. **Source freshness policy** using producer-specific `valid_until`, authority, or confidence metadata.
5. **Cross-process transaction journal** if concurrent writers or strict log durability become real requirements.
6. **Git automation** only with an explicit privacy, signing, remote, and retention policy.
## Production thresholds
Revisit the design when any of these occurs:
- Search latency is consistently above 100 ms.
- The bundle exceeds roughly 5,000 concepts or 50 MB.
- Multiple processes write concurrently.
- Memory has regulatory retention/deletion requirements.
- Retrieval quality is measured and lexical search misses important concepts.
Until then, files plus deterministic tooling are the smallest production-aware architecture.

50
extensions/memory/access.mjs Normal file
View File

@@ -0,0 +1,50 @@
export class MemoryTurnAccess {
constructor() {
this.reset();
}
reset() {
this.searched = false;
this.searchResultIds = new Set();
this.readIds = new Set();
}
recordSearch(ids) {
this.searched = true;
for (const id of ids) this.searchResultIds.add(id);
}
requireSearchedId(id) {
if (!this.searched) {
throw new Error("Memory workflow blocked: call memory_search in this user turn before reading or mutating memory.");
}
if (!this.searchResultIds.has(id)) {
throw new Error(`Memory workflow blocked: search for the exact concept '${id}' in this user turn first.`);
}
}
recordRead(ids) {
for (const id of ids) {
this.requireSearchedId(id);
this.readIds.add(id);
}
}
requireWrite(id, exists) {
if (!this.searched) {
throw new Error("Memory workflow blocked: call memory_search for duplicates in this user turn before writing.");
}
if (!exists) return;
this.requireSearchedId(id);
if (!this.readIds.has(id)) {
throw new Error(`Memory workflow blocked: read existing concept '${id}' in this user turn before updating it.`);
}
}
requireForget(id) {
this.requireSearchedId(id);
if (!this.readIds.has(id)) {
throw new Error(`Memory workflow blocked: read concept '${id}' in this user turn before forgetting it.`);
}
}
}

184
extensions/memory/index.ts Normal file
View File

@@ -0,0 +1,184 @@
import { fileURLToPath } from "node:url";
import { dirname, join, resolve } from "node:path";
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
import { Type } from "typebox";
import { MemoryTurnAccess } from "./access.mjs";
import { MemoryStore } from "./store.mjs";
const DETAIL_TOOLS = ["memory_read", "memory_write", "memory_forget"];
const EXPLICIT_MEMORY_INTENT = /\b(?:forget|forgot|memories|memory|recall|remember|remembered)\b|\b(?:last time|previous conversation)\b/i;
const extensionInstanceRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..");
const memoryRoot = join(process.env.PI_CODING_AGENT_DIR ?? extensionInstanceRoot, "memory");
const store = new MemoryStore(memoryRoot);
function textResult(text: string, details: unknown = {}) {
return { content: [{ type: "text" as const, text }], details };
}
export default function memoryExtension(pi: ExtensionAPI) {
const turnAccess = new MemoryTurnAccess();
let preparedInput: string | undefined;
function resetTurnAccess() {
turnAccess.reset();
const compactTools = pi.getActiveTools().filter((name) => !DETAIL_TOOLS.includes(name));
pi.setActiveTools([...new Set([...compactTools, "memory_search"])]);
}
function enableDetailTools() {
pi.setActiveTools([...new Set([...pi.getActiveTools(), ...DETAIL_TOOLS])]);
}
function prepareTurn(text: string) {
resetTurnAccess();
// Explicit memory requests reveal the gated tools early so the model knows the
// capability exists. The workflow gate still requires search/read in order.
if (EXPLICIT_MEMORY_INTENT.test(text)) enableDetailTools();
}
pi.registerTool({
name: "memory_search",
label: "Search Memory",
description: "Required first step for every recall, remember, update, or forget request. Search durable memory by keywords; executing this tool enables detailed memory tools even when no matches are found.",
promptSnippet: "First step for every recall, remember, update, or forget request; enables detailed memory tools",
promptGuidelines: [
"Always call memory_search first for recall, remember, update, or forget requests; it enables the detailed memory tools even when no match exists.",
],
parameters: Type.Object({
query: Type.String({ description: "Keywords for the person, project, preference, decision, or fact" }),
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: 20, description: "Maximum candidates (default 8)" })),
}),
async execute(_toolCallId, params) {
enableDetailTools();
const result = await store.search(params.query, params.limit ?? 8);
turnAccess.recordSearch(result.results.map((item) => item.id));
if (!result.results.length) {
return textResult(`No memory candidates found for: ${params.query}\nDetailed memory tools are now enabled.`, result);
}
const lines = result.results.map((item) => {
const tags = item.tags.length ? ` [${item.tags.join(", ")}]` : "";
const description = item.description ? ` — ${item.description}` : "";
return `- ${item.id} | ${item.type} | ${item.timestamp ?? "timestamp unavailable"}${tags}${description}`;
});
if (result.warnings.length) lines.push(`\nWarnings: ${result.warnings.length} invalid concept(s); use /memory-check --repair.`);
return textResult(lines.join("\n"), result);
},
});
pi.registerTool({
name: "memory_read",
label: "Read Memory",
description: "Load full bodies for up to 10 concept IDs returned by memory_search in the current user turn. Output is capped at 45,000 characters.",
parameters: Type.Object({
ids: Type.Array(Type.String({ description: "Bundle-relative concept ID without .md" }), { minItems: 1, maxItems: 10 }),
}),
async execute(_toolCallId, params) {
const normalizedIds = params.ids.map((id) => store.normalizeId(id));
for (const id of normalizedIds) turnAccess.requireSearchedId(id);
const result = await store.read(normalizedIds);
turnAccess.recordRead(result.documents.map((document) => document.id));
return textResult(result.text, result);
},
});
pi.registerTool({
name: "memory_write",
label: "Write Memory",
description: "Create or replace one atomic durable concept after searching in the current user turn; existing concepts must also be read first. Supply semantic fields and Markdown body only; storage mechanics are deterministic.",
parameters: Type.Object({
id: Type.String({ description: "Stable lowercase bundle-relative ID, e.g. people/alice; no .md" }),
type: Type.String({ description: "Short descriptive type, e.g. Person, Preference, Project, Decision" }),
title: Type.Optional(Type.String({ description: "Human-readable title" })),
description: Type.Optional(Type.String({ description: "One-sentence retrieval summary" })),
tags: Type.Optional(Type.Array(Type.String(), { maxItems: 20 })),
body: Type.String({ description: "Structured Markdown body without YAML frontmatter or a timestamp" }),
}),
async execute(_toolCallId, params) {
const id = store.normalizeId(params.id);
turnAccess.requireWrite(id, await store.exists(id));
const result = await store.upsert({ ...params, id });
return textResult(`${result.action}: ${result.id} (${result.timestamp})${result.attempts > 1 ? " after verified retry" : ""}`, result);
},
});
pi.registerTool({
name: "memory_forget",
label: "Forget Memory",
description: "Permanently delete one durable concept after searching and reading it in the current user turn. Confirmation must exactly match the concept ID.",
parameters: Type.Object({
id: Type.String({ description: "Concept ID to delete" }),
confirm: Type.String({ description: "Repeat the exact concept ID to confirm permanent deletion" }),
}),
async execute(_toolCallId, params) {
const id = store.normalizeId(params.id);
turnAccess.requireForget(id);
const result = await store.forget({ ...params, id });
return textResult(`deleted: ${result.id}`, result);
},
});
pi.on("input", (event) => {
// Do not revoke access while an earlier turn is still executing. Queued or
// steering input is prepared when its own before_agent_start event arrives.
if (event.streamingBehavior !== undefined) return;
prepareTurn(event.text);
preparedInput = event.text;
});
pi.on("before_agent_start", async (event) => {
if (preparedInput !== event.prompt) prepareTurn(event.prompt);
preparedInput = undefined;
try {
const hints = await store.hints(event.prompt, 2);
if (hints.length === 0) return;
const lines = hints.map((hint) => {
const summary = (hint.description || hint.title).replace(/\s+/g, " ").slice(0, 180);
return `- ${hint.id} — ${summary}`;
});
return {
// This is rebuilt for each user turn and is never appended to session history.
systemPrompt: `${event.systemPrompt}\n\nEphemeral long-term memory candidates (metadata only):\n${lines.join("\n")}\nTreat these only as retrieval hints. If relevant, call memory_search and then memory_read; otherwise ignore them.`,
};
} catch (error) {
console.warn(`Memory hint retrieval failed: ${error instanceof Error ? error.message : String(error)}`);
}
});
pi.on("session_start", async (_event, ctx) => {
try {
const report = await store.initialize();
resetTurnAccess();
if (report.repaired > 0 && ctx.hasUI) {
ctx.ui.notify(`Memory repaired ${report.repaired} concept(s); run /memory-check for details.`, "warning");
}
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
console.warn(`Memory initialization failed: ${message}`);
if (ctx.hasUI) ctx.ui.notify(`Memory initialization failed: ${message}`, "error");
}
});
pi.registerCommand("memory-check", {
description: "Validate memory; pass --repair for deterministic repair and regenerated indexes",
handler: async (args, ctx) => {
try {
const report = await store.audit({ repair: args.trim() === "--repair" });
const summary = `${report.valid ? "valid" : "invalid"}: ${report.concepts} concepts, ${report.issues.length} issues, ${report.repaired} repaired`;
if (ctx.hasUI) ctx.ui.notify(summary, report.valid ? "info" : "warning");
else console.log(summary);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (ctx.hasUI) ctx.ui.notify(message, "error");
else console.error(message);
}
},
});
pi.registerCommand("memory-path", {
description: "Show the OKF memory bundle path",
handler: async (_args, ctx) => {
if (ctx.hasUI) ctx.ui.notify(memoryRoot, "info");
else console.log(memoryRoot);
},
});
}

28
extensions/memory/package-lock.json generated Normal file
View File

@@ -0,0 +1,28 @@
{
"name": "kimiko-memory-extension",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "kimiko-memory-extension",
"dependencies": {
"yaml": "^2.8.1"
}
},
"node_modules/yaml": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.0.tgz",
"integrity": "sha512-2AvhNX3mb8zd6Zy7INTtSpl1F15HW6Wnqj0srWlkKLcpYl/gMIMJiyuGq2KeI2YFxUPjdlB+3Lc10seMLtL4cA==",
"license": "ISC",
"bin": {
"yaml": "bin.mjs"
},
"engines": {
"node": ">= 14.6"
},
"funding": {
"url": "https://github.com/sponsors/eemeli"
}
}
}
}

11
extensions/memory/package.json Normal file
View File

@@ -0,0 +1,11 @@
{
"name": "kimiko-memory-extension",
"private": true,
"type": "module",
"scripts": {
"test": "node --test test/*.test.mjs"
},
"dependencies": {
"yaml": "^2.8.1"
}
}

695
extensions/memory/store.mjs Normal file
View File

@@ -0,0 +1,695 @@
import { open, mkdir, readFile, readdir, rename, rm, stat, unlink, writeFile } from "node:fs/promises";
import { randomUUID } from "node:crypto";
import { basename, dirname, join, posix, relative, resolve, sep } from "node:path";
import YAML from "yaml";
const RESERVED = new Set(["index", "log"]);
const INDEX_START = "<!-- kimiko-memory:index:start -->";
const INDEX_END = "<!-- kimiko-memory:index:end -->";
const LOCAL_REQUIRED_KEYS = ["type", "timestamp"];
function sleep(ms) {
return new Promise((resolvePromise) => setTimeout(resolvePromise, ms));
}
function normalizeText(value) {
return value.replace(/^\uFEFF/, "").replace(/\r\n?/g, "\n");
}
function normalizeBody(value) {
const normalized = normalizeText(value).trim();
return normalized ? `${normalized}\n` : "";
}
function isPlainObject(value) {
return value !== null && typeof value === "object" && !Array.isArray(value);
}
function stableValue(value) {
if (Array.isArray(value)) return value.map(stableValue);
if (!isPlainObject(value)) return value;
return Object.fromEntries(Object.keys(value).sort().map((key) => [key, stableValue(value[key])]));
}
function fingerprint(metadata, body) {
const meaningful = { ...metadata };
delete meaningful.timestamp;
return JSON.stringify({ metadata: stableValue(meaningful), body: normalizeBody(body) });
}
function validTimestamp(value) {
return typeof value === "string" && /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d{1,3})?Z$/.test(value)
&& Number.isFinite(Date.parse(value));
}
function parseFrontmatter(raw) {
const text = normalizeText(raw);
if (!text.startsWith("---\n")) {
throw new Error("missing YAML frontmatter");
}
const lines = text.split("\n");
let closing = -1;
for (let index = 1; index < lines.length; index += 1) {
if (lines[index] === "---" || lines[index] === "...") {
closing = index;
break;
}
}
if (closing < 0) throw new Error("unterminated YAML frontmatter");
let metadata;
try {
metadata = YAML.parse(lines.slice(1, closing).join("\n")) ?? {};
} catch (error) {
throw new Error(`invalid YAML frontmatter: ${error instanceof Error ? error.message : String(error)}`);
}
if (!isPlainObject(metadata)) throw new Error("frontmatter must be a YAML mapping");
return { metadata, body: normalizeBody(lines.slice(closing + 1).join("\n")) };
}
function serializeConcept(metadata, body) {
const yaml = YAML.stringify(metadata, { lineWidth: 0 }).trimEnd();
return `---\n${yaml}\n---\n\n${normalizeBody(body)}`;
}
function extractRecoverableBody(raw) {
const text = normalizeText(raw);
if (!text.startsWith("---\n")) return normalizeBody(text);
const match = text.match(/^---\n[\s\S]*?\n(?:---|\.\.\.)\n?/);
if (!match) return normalizeBody(text);
return normalizeBody(text.slice(match[0].length));
}
function titleFromId(id) {
const leaf = id.split("/").at(-1) ?? id;
return leaf
.split(/[-_.]+/)
.filter(Boolean)
.map((part) => part.charAt(0).toUpperCase() + part.slice(1))
.join(" ");
}
function normalizeTags(tags) {
if (tags === undefined) return undefined;
const result = [];
const seen = new Set();
for (const raw of tags) {
const tag = raw.trim();
if (!tag || seen.has(tag)) continue;
if (tag.length > 64) throw new Error(`tag exceeds 64 characters: ${tag.slice(0, 20)}…`);
seen.add(tag);
result.push(tag);
}
if (result.length > 20) throw new Error("at most 20 tags are allowed");
return result;
}
function assertShortSingleLine(name, value, max, required = false) {
if (value === undefined) {
if (required) throw new Error(`${name} is required`);
return;
}
if (!value.trim()) throw new Error(`${name} must not be empty`);
if (value.length > max) throw new Error(`${name} exceeds ${max} characters`);
if (/[\r\n]/.test(value)) throw new Error(`${name} must be one line`);
}
function escapeMarkdown(value) {
return value.replace(/([\\[\]])/g, "\\$1");
}
function replaceGeneratedSection(text, section) {
const start = text.indexOf(INDEX_START);
const end = text.indexOf(INDEX_END);
if (start >= 0 && end >= start) {
return `${text.slice(0, start).trimEnd()}\n\n${section}\n${text.slice(end + INDEX_END.length).trimStart()}`.trimEnd() + "\n";
}
return `${text.trimEnd()}\n\n${section}\n`;
}
const SEARCH_STOP_WORDS = new Set([
"about", "after", "again", "also", "been", "before", "being", "could", "does", "from", "have", "into",
"just", "know", "like", "more", "most", "much", "need", "only", "other", "should", "some", "such", "than",
"that", "their", "them", "then", "there", "these", "they", "this", "those", "through", "user", "very", "want",
"what", "when", "where", "which", "while", "with", "would", "your", "youre", "remember", "memory",
]);
const HINT_GENERIC_TERMS = new Set(["decision", "fact", "person", "preference", "project", "remember", "memory"]);
function tokenize(text) {
return [...text.toLocaleLowerCase().matchAll(/[\p{L}\p{N}]+/gu)]
.map((match) => match[0])
.filter((term) => term.length > 2 && !SEARCH_STOP_WORDS.has(term));
}
export class MemoryStore {
constructor(root, options = {}) {
this.root = resolve(root);
this.clock = options.clock ?? (() => new Date());
this.afterWriteAttempt = options.afterWriteAttempt;
this.lockPath = join(this.root, ".kimiko-memory.lock");
}
normalizeId(input) {
if (typeof input !== "string") throw new Error("concept id must be a string");
let id = input.trim().replace(/^@/, "");
if (id.endsWith(".md")) id = id.slice(0, -3);
if (!id || id.length > 240) throw new Error("concept id must contain 1-240 characters");
if (id.startsWith("/") || id.includes("\\") || id.split("/").some((part) => part === "" || part === "." || part === "..")) {
throw new Error("concept id must be a safe bundle-relative path");
}
if (posix.normalize(id) !== id || !id.split("/").every((part) => /^[a-z0-9][a-z0-9._-]*$/.test(part))) {
throw new Error("concept id segments may contain lowercase letters, numbers, dots, underscores, and hyphens");
}
if (RESERVED.has(id.split("/").at(-1))) throw new Error("index and log are reserved filenames");
return id;
}
pathForId(id) {
const normalized = this.normalizeId(id);
const path = resolve(this.root, ...normalized.split("/")) + ".md";
if (relative(this.root, path).startsWith(`..${sep}`)) throw new Error("concept path escapes bundle root");
return { id: normalized, path };
}
async initialize() {
await mkdir(this.root, { recursive: true });
return this.withLock(() => this.auditUnlocked({ repair: true }));
}
async withLock(task) {
await mkdir(this.root, { recursive: true });
let handle;
for (let attempt = 0; attempt < 20; attempt += 1) {
try {
handle = await open(this.lockPath, "wx", 0o600);
await handle.writeFile(`${process.pid}\n${this.clock().toISOString()}\n`, "utf8");
break;
} catch (error) {
if (error?.code !== "EEXIST") throw error;
try {
const info = await stat(this.lockPath);
if (Date.now() - info.mtimeMs > 30_000) await unlink(this.lockPath);
} catch (statError) {
if (statError?.code !== "ENOENT") throw statError;
}
await sleep(Math.min(50 * (attempt + 1), 250));
}
}
if (!handle) throw new Error("memory bundle is locked by another process");
try {
return await task();
} finally {
await handle.close().catch(() => {});
await unlink(this.lockPath).catch(() => {});
}
}
async atomicWrite(path, content) {
await mkdir(dirname(path), { recursive: true });
const temporary = join(dirname(path), `.${basename(path)}.${process.pid}.${randomUUID()}.tmp`);
const handle = await open(temporary, "wx", 0o600);
try {
await handle.writeFile(content, "utf8");
await handle.sync();
} finally {
await handle.close();
}
try {
await rename(temporary, path);
} catch (error) {
await rm(temporary, { force: true });
throw error;
}
}
async verifiedWrite(path, content, verify, kind = "file") {
let lastError;
for (let attempt = 1; attempt <= 2; attempt += 1) {
try {
await this.atomicWrite(path, content);
await this.afterWriteAttempt?.({ path, attempt, kind });
const persisted = await readFile(path, "utf8");
await verify(persisted);
return attempt;
} catch (error) {
lastError = error;
}
}
throw new Error(`${kind} verification failed after deterministic retry: ${lastError instanceof Error ? lastError.message : String(lastError)}`);
}
async backup(path, raw) {
const rel = relative(this.root, path).split(sep).join("/");
const stamp = this.clock().toISOString().replace(/[:.]/g, "-");
const backupPath = join(this.root, ".recovery", `${rel}.${stamp}.bak`);
await mkdir(dirname(backupPath), { recursive: true });
await writeFile(backupPath, raw, { encoding: "utf8", mode: 0o600 });
return relative(this.root, backupPath).split(sep).join("/");
}
async conceptFiles(directory = this.root) {
const found = [];
let entries = [];
try {
entries = await readdir(directory, { withFileTypes: true });
} catch (error) {
if (error?.code === "ENOENT") return found;
throw error;
}
for (const entry of entries) {
if (entry.name.startsWith(".")) continue;
const path = join(directory, entry.name);
if (entry.isDirectory()) found.push(...await this.conceptFiles(path));
else if (entry.isFile() && entry.name.endsWith(".md") && !["index.md", "log.md"].includes(entry.name)) found.push(path);
}
return found.sort();
}
async bundleDirectories(directory = this.root) {
const found = [directory];
let entries = [];
try {
entries = await readdir(directory, { withFileTypes: true });
} catch (error) {
if (error?.code === "ENOENT") return found;
throw error;
}
for (const entry of entries) {
if (entry.isDirectory() && !entry.name.startsWith(".")) {
found.push(...await this.bundleDirectories(join(directory, entry.name)));
}
}
return found;
}
async inspect(path) {
const raw = await readFile(path, "utf8");
const { metadata, body } = parseFrontmatter(raw);
const errors = [];
if (typeof metadata.type !== "string" || !metadata.type.trim()) errors.push("missing non-empty type");
if (!validTimestamp(metadata.timestamp)) errors.push("missing or invalid ISO-8601 UTC timestamp (local producer profile)");
return { raw, metadata, body, errors };
}
async search(query, limit = 8) {
const boundedLimit = Math.max(1, Math.min(20, limit));
const terms = [...new Set(tokenize(query))];
const normalizedQuery = query.trim().toLocaleLowerCase();
const results = [];
const warnings = [];
for (const path of await this.conceptFiles()) {
const id = relative(this.root, path).split(sep).join("/").slice(0, -3);
try {
const { metadata, body, errors } = await this.inspect(path);
if (errors.length) warnings.push(`${id}: ${errors.join(", ")}`);
const title = typeof metadata.title === "string" ? metadata.title : titleFromId(id);
const description = typeof metadata.description === "string" ? metadata.description : "";
const tags = Array.isArray(metadata.tags) ? metadata.tags.filter((tag) => typeof tag === "string") : [];
const weighted = [
["id", id, 8], ["title", title, 7], ["description", description, 5], ["tags", tags.join(" "), 5],
["type", String(metadata.type ?? ""), 3], ["body", body, 1],
];
const matchedFields = new Set();
const metadataTerms = new Set();
let score = terms.length === 0 ? 0 : terms.reduce((total, term) => total + weighted.reduce(
(subtotal, [field, value, weight]) => {
if (!String(value).toLocaleLowerCase().includes(term)) return subtotal;
matchedFields.add(String(field));
if (field !== "body") metadataTerms.add(term);
return subtotal + Number(weight);
}, 0,
), 0);
if (normalizedQuery && weighted.some(([, value]) => String(value).toLocaleLowerCase().includes(normalizedQuery))) score += 10;
if (terms.length > 0 && score === 0) continue;
results.push({
id,
type: String(metadata.type ?? "Unknown"),
title,
description,
tags,
timestamp: validTimestamp(metadata.timestamp) ? metadata.timestamp : undefined,
score,
matches: { fields: [...matchedFields], metadataTerms: [...metadataTerms] },
});
} catch (error) {
warnings.push(`${id}: ${error instanceof Error ? error.message : String(error)}`);
}
}
results.sort((left, right) => right.score - left.score
|| String(right.timestamp ?? "").localeCompare(String(left.timestamp ?? ""))
|| left.id.localeCompare(right.id));
return { results: results.slice(0, boundedLimit), warnings };
}
async hints(prompt, limit = 2) {
const { results } = await this.search(prompt.slice(0, 2_000), 20);
return results.filter((result) => {
const distinctive = result.matches.metadataTerms.filter((term) => !HINT_GENERIC_TERMS.has(term));
if (distinctive.length === 0) return false;
const direct = result.matches.fields.some((field) => ["id", "title", "tags"].includes(field));
return direct || (distinctive.length >= 2 && result.matches.fields.includes("description"));
}).slice(0, Math.max(1, Math.min(3, limit)));
}
async exists(id) {
const { path } = this.pathForId(id);
try {
await stat(path);
return true;
} catch (error) {
if (error?.code === "ENOENT") return false;
throw error;
}
}
async read(ids, maxCharacters = 45_000) {
const unique = [...new Set(ids.map((id) => this.normalizeId(id)))];
if (unique.length > 10) throw new Error("at most 10 concepts may be read at once");
const documents = [];
let text = "";
let truncated = false;
for (const id of unique) {
const { path } = this.pathForId(id);
let inspected;
try {
inspected = await this.inspect(path);
} catch (error) {
if (error?.code === "ENOENT") throw new Error(`memory concept not found: ${id}`);
throw error;
}
if (inspected.errors.length) throw new Error(`${id} is invalid; run /memory-check --repair`);
const heading = `## ${id}\n\nType: ${inspected.metadata.type}\nTimestamp: ${inspected.metadata.timestamp}\n\n`;
const block = `${heading}${inspected.body}`;
if (text.length + block.length > maxCharacters) {
const remaining = Math.max(0, maxCharacters - text.length);
text += block.slice(0, remaining);
truncated = true;
break;
}
text += `${block}\n`;
documents.push({ id, metadata: inspected.metadata });
}
if (truncated) text += "\n[Memory output truncated; read fewer concept IDs.]";
return { text: text.trimEnd(), documents, truncated };
}
async upsert(input) {
return this.withLock(async () => {
const { id, path } = this.pathForId(input.id);
assertShortSingleLine("type", input.type, 100, true);
assertShortSingleLine("title", input.title, 200);
assertShortSingleLine("description", input.description, 500);
if (typeof input.body !== "string") throw new Error("body is required");
if (input.body.length > 100_000) throw new Error("body exceeds 100,000 characters");
const tags = normalizeTags(input.tags);
let existing;
let existingRaw;
try {
existingRaw = await readFile(path, "utf8");
existing = parseFrontmatter(existingRaw);
} catch (error) {
if (error?.code !== "ENOENT" && existingRaw !== undefined) {
await this.backup(path, existingRaw);
existing = { metadata: {}, body: extractRecoverableBody(existingRaw) };
} else if (error?.code !== "ENOENT") {
throw error;
}
}
const metadata = { ...(existing?.metadata ?? {}) };
metadata.type = input.type.trim();
if (input.title !== undefined) metadata.title = input.title.trim();
if (input.description !== undefined) metadata.description = input.description.trim();
if (tags !== undefined) metadata.tags = tags;
const body = normalizeBody(input.body);
const changed = !existing || fingerprint(metadata, body) !== fingerprint(existing.metadata, existing.body);
if (!changed && validTimestamp(existing.metadata.timestamp)) {
return { id, action: "unchanged", timestamp: existing.metadata.timestamp, attempts: 0 };
}
if (changed) metadata.timestamp = this.clock().toISOString();
else {
const info = await stat(path);
metadata.timestamp = info.mtime.toISOString();
}
const serialized = serializeConcept(metadata, body);
const expectedFingerprint = fingerprint(metadata, body);
const attempts = await this.verifiedWrite(path, serialized, (persisted) => {
const parsed = parseFrontmatter(persisted);
if (parsed.metadata.timestamp !== metadata.timestamp || fingerprint(parsed.metadata, parsed.body) !== expectedFingerprint) {
throw new Error("persisted concept differs from requested concept");
}
if (!LOCAL_REQUIRED_KEYS.every((key) => parsed.metadata[key])) throw new Error("persisted concept lacks required producer metadata");
}, "concept");
await this.updateIndexes();
await this.appendLog(changed && existing ? "Update" : existing ? "Repair" : "Creation", id, metadata.title ?? titleFromId(id));
return { id, action: existing ? (changed ? "updated" : "repaired") : "created", timestamp: metadata.timestamp, attempts };
});
}
async forget(input) {
return this.withLock(async () => {
const { id, path } = this.pathForId(input.id);
if (input.confirm !== id) throw new Error("confirm must exactly match the concept id");
let title = titleFromId(id);
try {
const inspected = await this.inspect(path);
if (typeof inspected.metadata.title === "string") title = inspected.metadata.title;
} catch (error) {
if (error?.code === "ENOENT") throw new Error(`memory concept not found: ${id}`);
}
await unlink(path);
const recoveryBase = join(this.root, ".recovery", `${id}.md`);
const recoveryDirectory = dirname(recoveryBase);
const recoveryPrefix = `${basename(recoveryBase)}.`;
let removedBackups = 0;
try {
for (const entry of await readdir(recoveryDirectory, { withFileTypes: true })) {
if (entry.isFile() && entry.name.startsWith(recoveryPrefix) && entry.name.endsWith(".bak")) {
await unlink(join(recoveryDirectory, entry.name));
removedBackups += 1;
}
}
} catch (error) {
if (error?.code !== "ENOENT") throw error;
}
await this.redactLogReferences(id);
await this.updateIndexes();
await this.appendLog("Deletion", id, title);
return { id, action: "deleted", removedBackups };
});
}
async updateIndexes() {
const files = await this.conceptFiles();
const directories = new Set(await this.bundleDirectories());
for (const file of files) {
let current = dirname(file);
while (current.startsWith(this.root)) {
directories.add(current);
if (current === this.root) break;
current = dirname(current);
}
}
for (const directory of [...directories].sort((a, b) => b.length - a.length)) {
const immediateFiles = files.filter((file) => dirname(file) === directory);
const childDirs = [...new Set(files
.filter((file) => dirname(file) !== directory && relative(directory, file) && !relative(directory, file).startsWith(".."))
.map((file) => relative(directory, file).split(sep)[0]))].sort();
const lines = [INDEX_START, "_Generated inventory; text outside this section is preserved._"];
if (childDirs.length) {
lines.push("", "## Directories");
for (const child of childDirs) {
const count = files.filter((file) => relative(join(directory, child), file) && !relative(join(directory, child), file).startsWith("..")).length;
lines.push(`- [${escapeMarkdown(titleFromId(child))}](${child}/) - ${count} concept${count === 1 ? "" : "s"}.`);
}
}
if (immediateFiles.length) {
lines.push("", "## Concepts");
for (const file of immediateFiles) {
const id = relative(this.root, file).split(sep).join("/").slice(0, -3);
let title = titleFromId(id);
let description = "";
try {
const { metadata } = await this.inspect(file);
if (typeof metadata.title === "string") title = metadata.title;
if (typeof metadata.description === "string") description = metadata.description;
} catch {}
lines.push(`- [${escapeMarkdown(title)}](${basename(file)})${description ? ` - ${description}` : ""}`);
}
}
if (!childDirs.length && !immediateFiles.length) lines.push("", "_No concepts._");
lines.push(INDEX_END);
const section = lines.join("\n");
const indexPath = join(directory, "index.md");
let current = "";
try { current = await readFile(indexPath, "utf8"); } catch (error) { if (error?.code !== "ENOENT") throw error; }
if (directory === this.root) {
let body = current;
let rootMetadata = {};
if (current.startsWith("---\n")) {
try {
const parsed = parseFrontmatter(current);
rootMetadata = parsed.metadata;
body = parsed.body;
} catch {
await this.backup(indexPath, current);
body = current;
}
}
rootMetadata.okf_version = "0.1";
if (!/^#\s+/m.test(body)) body = `# Memory\n\n${body}`;
current = `---\n${YAML.stringify(rootMetadata, { lineWidth: 0 }).trimEnd()}\n---\n\n${body}`;
} else if (!/^#\s+/m.test(current)) {
current = `# ${titleFromId(basename(directory))}\n\n${current}`;
}
const next = replaceGeneratedSection(current, section);
if (normalizeText(current) !== next) {
await this.verifiedWrite(indexPath, next, (persisted) => {
if (!persisted.includes(INDEX_START) || !persisted.includes(INDEX_END) || !/^#\s+/m.test(persisted)) {
throw new Error("generated index markers or heading missing");
}
}, "index");
}
}
}
async ensureLog() {
const path = join(this.root, "log.md");
let current;
try {
current = normalizeText(await readFile(path, "utf8"));
} catch (error) {
if (error?.code !== "ENOENT") throw error;
current = "";
}
if (current.startsWith("# Update Log")) return;
if (current) await this.backup(path, current);
const next = `# Update Log\n${current ? `\n${current.trim()}\n` : ""}`;
await this.verifiedWrite(path, next, (persisted) => {
if (!persisted.startsWith("# Update Log")) throw new Error("log heading was not persisted");
}, "log repair");
}
async redactLogReferences(id) {
const path = join(this.root, "log.md");
let current;
try {
current = normalizeText(await readFile(path, "utf8"));
} catch (error) {
if (error?.code === "ENOENT") return;
throw error;
}
const target = `](/${id}.md)`;
const next = current.split("\n").filter((line) => !line.includes(target)).join("\n").replace(/\n{3,}/g, "\n\n").trimEnd() + "\n";
if (next !== current) {
await this.verifiedWrite(path, next, (persisted) => {
if (persisted.includes(target)) throw new Error("deleted concept remains in log");
}, "log redaction");
}
}
async appendLog(action, id, title) {
const path = join(this.root, "log.md");
let current = "# Update Log\n";
try { current = normalizeText(await readFile(path, "utf8")); } catch (error) { if (error?.code !== "ENOENT") throw error; }
if (!current.startsWith("# Update Log")) {
await this.backup(path, current);
current = `# Update Log\n\n${current.trim()}\n`;
}
const date = this.clock().toISOString().slice(0, 10);
const entry = action === "Deletion"
? "- **Deletion**: Removed a concept."
: `- **${action}**: ${action === "Creation" ? "Added" : "Changed"} [${escapeMarkdown(String(title))}](/${id}.md).`;
if (current.includes(entry)) return;
const heading = `## ${date}`;
let next;
const headingAt = current.indexOf(heading);
if (headingAt >= 0) {
const insertion = headingAt + heading.length;
next = `${current.slice(0, insertion)}\n\n${entry}${current.slice(insertion)}`;
} else {
const firstLineEnd = current.indexOf("\n");
next = `${current.slice(0, firstLineEnd + 1)}\n${heading}\n\n${entry}\n${current.slice(firstLineEnd + 1).trimStart()}`;
}
next = next.trimEnd() + "\n";
await this.verifiedWrite(path, next, (persisted) => {
if (!persisted.startsWith("# Update Log") || !persisted.includes(entry)) throw new Error("log entry was not persisted");
}, "log");
}
async audit(options = {}) {
return this.withLock(() => this.auditUnlocked(options));
}
async auditUnlocked({ repair = false } = {}) {
const issues = [];
let repaired = 0;
for (const path of await this.conceptFiles()) {
const id = relative(this.root, path).split(sep).join("/").slice(0, -3);
let raw;
try {
raw = await readFile(path, "utf8");
const parsed = parseFrontmatter(raw);
const errors = [];
if (typeof parsed.metadata.type !== "string" || !parsed.metadata.type.trim()) errors.push("missing type");
if (!validTimestamp(parsed.metadata.timestamp)) errors.push("missing/invalid timestamp");
if (!errors.length) continue;
issues.push({ id, errors, repaired: repair });
if (!repair) continue;
const info = await stat(path);
const metadata = { ...parsed.metadata };
if (typeof metadata.type !== "string" || !metadata.type.trim()) metadata.type = "Recovered Concept";
if (!validTimestamp(metadata.timestamp)) metadata.timestamp = info.mtime.toISOString();
const content = serializeConcept(metadata, parsed.body);
await this.verifiedWrite(path, content, (persisted) => {
const checked = parseFrontmatter(persisted);
if (!checked.metadata.type || !validTimestamp(checked.metadata.timestamp)) throw new Error("safe repair did not restore required metadata");
}, "concept repair");
repaired += 1;
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
issues.push({ id, errors: [message], repaired: repair });
if (!repair || raw === undefined) continue;
const info = await stat(path);
const backup = await this.backup(path, raw);
const metadata = {
type: "Recovered Concept",
title: titleFromId(id),
description: "Recovered deterministically from an invalid concept document; review recommended.",
timestamp: info.mtime.toISOString(),
repair_backup: backup,
};
const content = serializeConcept(metadata, extractRecoverableBody(raw));
await this.verifiedWrite(path, content, (persisted) => {
const checked = parseFrontmatter(persisted);
if (!checked.metadata.type || !validTimestamp(checked.metadata.timestamp)) throw new Error("recovery did not restore required metadata");
}, "concept recovery");
repaired += 1;
}
}
if (repair) {
await this.updateIndexes();
await this.ensureLog();
}
const files = await this.conceptFiles();
return { concepts: files.length, issues, repaired, valid: issues.length === 0 || issues.every((issue) => issue.repaired) };
}
}
export { parseFrontmatter, serializeConcept, validTimestamp };

31
extensions/memory/test/access.test.mjs Normal file
View File

@@ -0,0 +1,31 @@
import assert from "node:assert/strict";
import test from "node:test";
import { MemoryTurnAccess } from "../access.mjs";
test("requires current-turn search and read before updating an existing concept", () => {
const access = new MemoryTurnAccess();
assert.throws(() => access.requireWrite("projects/atlas", true), /memory_search/);
access.recordSearch(["projects/atlas"]);
assert.throws(() => access.requireWrite("projects/atlas", true), /read existing concept/);
access.recordRead(["projects/atlas"]);
assert.doesNotThrow(() => access.requireWrite("projects/atlas", true));
});
test("new concepts require duplicate search but cannot be read unless returned", () => {
const access = new MemoryTurnAccess();
access.recordSearch([]);
assert.doesNotThrow(() => access.requireWrite("projects/new-project", false));
assert.throws(() => access.requireSearchedId("projects/new-project"), /exact concept/);
});
test("turn reset removes prior authorization", () => {
const access = new MemoryTurnAccess();
access.recordSearch(["people/alice"]);
access.recordRead(["people/alice"]);
assert.doesNotThrow(() => access.requireForget("people/alice"));
access.reset();
assert.throws(() => access.requireForget("people/alice"), /this user turn/);
});

151
extensions/memory/test/store.test.mjs Normal file
View File

@@ -0,0 +1,151 @@
import assert from "node:assert/strict";
import { mkdtemp, readFile, readdir, rm, utimes, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
import { MemoryStore, parseFrontmatter } from "../store.mjs";
async function fixture(options = {}) {
const root = await mkdtemp(join(tmpdir(), "kimiko-memory-test-"));
const store = new MemoryStore(root, options);
await store.initialize();
return { root, store };
}
async function cleanup(root) {
await rm(root, { recursive: true, force: true });
}
test("writes valid concepts, preserves timestamps on no-op, and updates generated files", async () => {
const dates = [new Date("2026-07-22T09:30:00.000Z"), new Date("2026-07-23T10:00:00.000Z")];
let dateIndex = 0;
const { root, store } = await fixture({ clock: () => dates[Math.min(dateIndex, dates.length - 1)] });
try {
const created = await store.upsert({
id: "people/alice",
type: "Person",
title: "Alice",
description: "A test person.",
tags: ["test", "test"],
body: "# Facts\n\n- Likes deterministic systems.",
});
assert.equal(created.action, "created");
assert.equal(created.timestamp, dates[0].toISOString());
dateIndex = 1;
const unchanged = await store.upsert({
id: "people/alice",
type: "Person",
title: "Alice",
description: "A test person.",
tags: ["test"],
body: "# Facts\n\n- Likes deterministic systems.",
});
assert.equal(unchanged.action, "unchanged");
assert.equal(unchanged.timestamp, dates[0].toISOString());
const parsed = parseFrontmatter(await readFile(join(root, "people", "alice.md"), "utf8"));
assert.equal(parsed.metadata.type, "Person");
assert.equal(parsed.metadata.timestamp, dates[0].toISOString());
assert.deepEqual(parsed.metadata.tags, ["test"]);
const rootIndex = await readFile(join(root, "index.md"), "utf8");
const peopleIndex = await readFile(join(root, "people", "index.md"), "utf8");
const log = await readFile(join(root, "log.md"), "utf8");
assert.match(rootIndex, /okf_version: "?0\.1"?/);
assert.match(rootIndex, /\[People\]\(people\/\)/);
assert.match(peopleIndex, /\[Alice\]\(alice\.md\) - A test person\./);
assert.match(log, /\*\*Creation\*\*/);
} finally {
await cleanup(root);
}
});
test("verified writes retry once after deterministic verification failure", async () => {
let injected = false;
const { root, store } = await fixture({
clock: () => new Date("2026-07-22T09:30:00.000Z"),
afterWriteAttempt: async ({ path, attempt, kind }) => {
if (!injected && kind === "concept" && attempt === 1 && path.endsWith("retry.md")) {
injected = true;
await writeFile(path, "broken", "utf8");
}
},
});
try {
const result = await store.upsert({ id: "retry", type: "Reference", body: "# Retried" });
assert.equal(result.attempts, 2);
assert.equal(parseFrontmatter(await readFile(join(root, "retry.md"), "utf8")).metadata.type, "Reference");
} finally {
await cleanup(root);
}
});
test("audit repairs missing and malformed frontmatter without discarding bodies", async () => {
const { root, store } = await fixture({ clock: () => new Date("2026-07-22T09:30:00.000Z") });
try {
const missingPath = join(root, "missing.md");
const malformedPath = join(root, "malformed.md");
await writeFile(missingPath, "# Original\n\nKeep this body.\n", "utf8");
await writeFile(malformedPath, "---\ntype: [broken\n---\n\n# Also Original\n", "utf8");
const mtime = new Date("2026-01-02T03:04:05.000Z");
await utimes(missingPath, mtime, mtime);
const report = await store.audit({ repair: true });
assert.equal(report.repaired, 2);
assert.equal(report.valid, true);
const missing = parseFrontmatter(await readFile(missingPath, "utf8"));
const malformed = parseFrontmatter(await readFile(malformedPath, "utf8"));
assert.equal(missing.metadata.type, "Recovered Concept");
assert.equal(missing.metadata.timestamp, mtime.toISOString());
assert.match(missing.body, /Keep this body/);
assert.match(malformed.body, /Also Original/);
assert.ok(malformed.metadata.repair_backup);
assert.ok((await readdir(join(root, ".recovery"))).length >= 1);
} finally {
await cleanup(root);
}
});
test("search is metadata-first and forget requires an exact confirmation", async () => {
const { root, store } = await fixture({ clock: () => new Date("2026-07-22T09:30:00.000Z") });
try {
await store.upsert({
id: "projects/atlas",
type: "Project",
title: "Atlas",
description: "Infrastructure provisioning platform.",
tags: ["platform"],
body: "# Secret Detail\n\nBody-only retrieval token: zephyrquartz.",
});
const found = await store.search("infrastructure atlas", 5);
assert.equal(found.results[0].id, "projects/atlas");
assert.equal("body" in found.results[0], false);
assert.equal((await store.hints("What did we decide about Atlas?"))[0].id, "projects/atlas");
assert.deepEqual(await store.hints("What did we discuss about this project?"), []);
assert.deepEqual(await store.hints("zephyrquartz"), []);
await assert.rejects(() => store.forget({ id: "projects/atlas", confirm: "atlas" }), /confirm must exactly match/);
assert.equal((await store.forget({ id: "projects/atlas", confirm: "projects/atlas" })).action, "deleted");
await assert.rejects(() => store.read(["projects/atlas"]), /not found/);
const log = await readFile(join(root, "log.md"), "utf8");
assert.doesNotMatch(log, /projects\/atlas|Atlas/);
assert.match(log, /\*\*Deletion\*\*: Removed a concept/);
assert.doesNotMatch(await readFile(join(root, "projects", "index.md"), "utf8"), /Atlas/);
} finally {
await cleanup(root);
}
});
test("rejects unsafe or reserved concept ids", async () => {
const { root, store } = await fixture();
try {
for (const id of ["../escape", "/absolute", "People/Alice", "index", "notes/log", "a//b"]) {
assert.throws(() => store.normalizeId(id));
}
} finally {
await cleanup(root);
}
});

35
skills/long-term-memory/SKILL.md Normal file
View File

@@ -0,0 +1,35 @@
---
name: long-term-memory
description: Use Kimiko's durable OKF memory when the user asks to remember, forget, or recall something; refers to prior conversations; asks about a known person, project, preference, or decision; states a durable correction or preference; or says something is off the record. Do not use for current-turn-only facts.
---
# Long-term memory
## Decision table
| Situation | Action |
|---|---|
| Explicit “remember this” | Search for duplicates; read a matching concept; then write. |
| Explicit recall or “what do you remember?” | Search, then read only relevant concepts. |
| Reference to an earlier conversation | Search before answering. |
| Stable preference or correction | Ask whether to remember it unless storage was explicitly requested. |
| Confirmed long-lived project decision | Offer to remember it. |
| New information conflicts with memory | Clarify; never silently overwrite. |
| Transient task, speculation, or casual remark | Do not store. |
| Credential, secret, or sensitive fact | Do not store without an explicit informed request. |
| “Don’t remember this” or “off the record” | Do not store it. |
| Explicit forget request | Search, read, then forget with exact-ID confirmation. |
Store only information likely to matter in another session, sufficiently stable, and explicitly requested or confirmed.
## Workflow
1. Call `memory_search` in the current user turn. Retrieval hints do not replace this step.
2. Call `memory_read` only for relevant IDs. Existing concepts must be read before update or deletion.
3. Write one coherent concept with `memory_write`; update an existing ID rather than duplicating it.
4. Use structured Markdown and explicit links such as `[Atlas](/projects/atlas.md)`.
5. After mutation, tell the user briefly which concept was created, updated, or forgotten.
Never author frontmatter, timestamps, indexes, or logs. The extension generates, validates, repairs, and verifies them deterministically.
For diagnostics and recovery policy, read [operations.md](references/operations.md).

26
skills/long-term-memory/references/operations.md Normal file
View File

@@ -0,0 +1,26 @@
# Memory operations
## Storage
The OKF bundle is `${PI_CODING_AGENT_DIR}/memory`. Concept IDs are lowercase bundle-relative paths without `.md`. `index.md` and `log.md` are reserved.
## Commands
- `/memory-path` — show the bundle path.
- `/memory-check` — report structural problems without modifying concepts.
- `/memory-check --repair` — repair deterministically and regenerate indexes.
## Turn-scoped access
Each user turn clears prior search/read authorization and removes detailed memory tools. Explicit memory language may reveal the tools early, and `memory_search` enables them otherwise; visibility never bypasses the workflow gate. Existing concepts must be returned by search and read in that same turn before mutation. Metadata-only retrieval hints are injected through the ephemeral per-turn system prompt, never session messages, so previous hints do not accumulate.
## Repair policy
- Missing/invalid timestamps use the file's modification time; the model never supplies them.
- Missing types become `Recovered Concept`.
- Invalid or absent frontmatter is backed up under `memory/.recovery/`, then replaced with valid metadata while retaining the recoverable body.
- Generated index sections are replaced between marker comments; human text outside the markers is retained.
- Writes use a bundle lock, atomic rename, read-after-write semantic verification, and one deterministic retry.
- No-op writes retain the prior timestamp and do not create a log entry.
Inspect any concept carrying `repair_backup` because malformed metadata may require human review. The bundle is ignored by Git by default; opt into private versioning only with an explicit retention and deletion policy. The extension never auto-commits.