# Memory Management Guide

Memories are the fundamental unit of information in TokST. Memories are tagged for filtering, organized within an atlas, and embedded into a 1536-dimensional vector space when the configured embedding provider is available.

## Memory Types

Choose the type that best describes the information you're storing.

| Type | Use Case | Example |
|---|---|---|
| `fact` | Verifiable, objective information | "The API gateway URL is https://api.tokst.com" |
| `decision` | A choice made, with rationale | "Chose Supabase over Firebase for Postgres-native tooling" |
| `preference` | Subjective preference | "Prefer REST over GraphQL for simple CRUD endpoints" |
| `task` | A task, TODO, or action item | "Migrate legacy users to new auth flow by Friday" |
| `architecture` | System design or architectural notes | "Auth flow uses JWT with 1h expiry and auto-refresh" |
| `note` | General information (default) | "Met with the team, discussed Q3 roadmap priorities" |

```bash
tokst remember "The database runs on Supabase Postgres" --type fact
tokst remember "Use Turborepo for monorepo management" --type decision --tags architecture,infra
```

## Write structured Markdown

Plain text works well for short facts. Use Markdown for decisions, architecture,
meeting notes, and tasks so each memory remains easy to scan in the dashboard.

| Type | Suggested sections |
|---|---|
| `fact` | Conclusion, Source, Scope |
| `decision` | Decision, Context, Rationale, Impact |
| `preference` | Preference, When it applies, Avoid |
| `task` | Goal, task checklist, Done when |
| `architecture` | Goal, Components, Data flow, Constraints |
| `note` | Summary, Notes, Next actions |

The dashboard editor offers these templates and Markdown formatting controls.
For long CLI input, keep the content in a Markdown file:

```bash
tokst remember --type decision --tags api,auth --stdin < decision.md
```

Agents capture confirmed durable information in the same structure. Keep
credentials, private keys, raw reasoning, and transient tool output outside
memory records.

## Kind

Every memory includes a `kind` field that describes its derivation:

| Kind | Description |
|---|---|
| `raw` | Directly recorded, original information |
| `summary` | A distilled or condensed version of information |
| `snapshot` | A point-in-time capture of context |

The kind is assigned automatically but can be overridden.

## Source Types

Track where a memory originated:

| Source | Description |
|---|---|
| `human` | Recorded by a person via CLI or dashboard |
| `agent` | Created by an AI agent via MCP |
| `import` | Imported from an external system |
| `system` | Generated by TokST internals (e.g., auto-routing) |

```bash
tokst remember "Auto-scaling group configured for 2-10 instances" --source-type agent --source codex
```

## Tags

Tags are comma-separated labels used for filtering and discovery. Unlike types (which are mutually exclusive), tags are additive — a memory can have many tags.

```bash
tokst remember "Deploy process documented in Notion" --tags deploy,documentation,notion
```

Tags power filtered searches:

```bash
tokst search "deploy" --tags production
tokst search "architecture" --type decision
```

## Search

TokST uses a **keyword-first, semantic-fallback** search pipeline:

1. **Keyword matching** — Traditional text search on memory content
2. **Semantic vector search** — Embedding similarity in 1536-dimensional space

Direct keyword matches return immediately and avoid an embedding round trip. When no keyword result exists, TokST generates a query embedding and runs a 1536-dimensional vector search constrained to the authenticated user and accessible workspaces.

```bash
tokst search "database connection issues"       # Semantic
tokst search "Supabase connection string"       # Keyword match
tokst search "auth" --type architecture         # Filtered
tokst search "api" --limit 20 --json            # With options
```

## Context Snapshots

The `context` command generates a formatted snapshot of recent memories in the active atlas. This is particularly useful for providing conversation context to AI agents.

```bash
tokst context                         # Active atlas
tokst context --atlas <id>            # Specific atlas
tokst context --limit 50              # More memories
```

The output includes memory content, type, tags, and timestamps in a readable format designed to be consumed by both humans and agents.

## Memory Lifecycle

Memories progress through three states:

```
Active  -->  Archived  -->  Deleted
```

| State | Description | Visible in search? | Recoverable? |
|---|---|---|---|
| **Active** | Normal, searchable | Yes | — |
| **Archived** | Soft-hidden, out of default results | No | Yes (`restore`) |
| **Deleted** | Permanently removed | No | No |

```bash
tokst memory archive <id>     # Soft-hide
tokst memory restore <id>     # Bring back
tokst memory delete <id>      # Permanent
```

## Trusted Memory Lifecycle

Use trusted metadata for facts, decisions, and policies that need a clear source or review trail.

| Field | Purpose |
|---|---|
| `evidence` | URL or file path supporting the memory |
| `confidence` | Confidence score from `0` to `1` |
| `validUntil` | Optional ISO date-time after which the record needs review |
| `reviewStatus` | `unreviewed`, `verified`, `needs_review`, or `superseded` |

```bash
tokst memory verify mem_xxx --evidence https://example.com/policy --confidence 0.95
tokst memory verify mem_xxx --valid-until 2027-01-01T00:00:00Z
tokst memory supersede mem_old mem_new
```

Verification preserves the memory and marks it as reviewed. Superseding links an older memory to its replacement and marks the earlier record as `superseded`, keeping historical context available.

## Embedding

When a memory is stored or its content changes, TokST requests an embedding vector. The embedding process:

- Produces a **1536-dimensional vector** when the provider is configured and available
- Runs on the server at write time
- Is transparent — you never interact with vectors directly
- Powers semantic fallback queries

Memory writes remain available when the embedding provider is not configured; keyword search continues to work and semantic fallback becomes available after embeddings are generated.

## File Attachments

Memories can have files attached via the `--file` flag. See the [File Attachments Guide](attachments) for details.

```bash
tokst remember "Sprint planning notes" --file sprint-planning.pdf
tokst memory attach <id> --file diagram.png
```

## Batch Import

Import an entire folder of files as memories. Text files (md, code, json, csv, etc.) are auto-extracted; binary files are uploaded as attachments.

```bash
tokst import ./docs --dry-run                  # preview
tokst import ./docs --type note --tags imported # import
```

See the [CLI Reference](cli#batch-import) for all flags.

## Best Practices

- **Use types consistently** — This makes filtered searches more reliable
- **Tag liberally** — Tags are the primary mechanism for cross-cutting organization
- **Archive, don't delete** — Archived memories can be restored if needed
- **Take context snapshots** — Run `tokst context` before agent sessions to provide background
- **Append to existing memories** — Use `append` instead of creating duplicates when adding related information