Private Namespaces
Store and search data that only you can read. Namespaces are isolated, access-controlled collections with client-side AES-256-GCM encryption — miners store ciphertext and never see your original text.
Overview
By default, all data on Engram is public — any miner can serve it and any client can query it. Private namespaces change this for use cases where data confidentiality matters:
| Public (default) | Private namespace | |
|---|---|---|
| Access control | None | Key required to read/write |
| Data at rest | Plaintext on miner | AES-256-GCM encrypted |
| Semantic search | ✓ | ✓ (vectors unencrypted, text encrypted) |
| Miner sees text | Yes | No — ciphertext only |
| Miner sees vectors | Yes | Yes (needed for search) |
| CID format | v1::sha256… | v1::sha256… (same) |
How it works
When you create an EngramClient with a namespace and namespace_key, the client automatically:
- Derives an AES-256 encryption key from your namespace key using PBKDF2-HMAC-SHA256 (100k iterations)
- Computes embeddings locally before sending anything to the network
- Encrypts your text and metadata with AES-256-GCM and a random 12-byte IV per message
- Sends only the float32 vector + encrypted blob to the miner
- On query, sends the query vector (computed locally) and decrypts result metadata client-side
Tip
Miners can serve semantic search because the embedding vectors are unencrypted. Your actual text and metadata are never transmitted or stored in plaintext.
Quick start
python
from engram.sdk import EngramClient# Create a private client — all data encrypted to this namespaceclient = EngramClient("http://miner:8091",namespace="acme-docs",namespace_key="your-secret-key-min-16-chars",)# Store — text is encrypted before leaving your machinecid = client.ingest("Q4 revenue was $4.2M, up 18% YoY.")cid2 = client.ingest("Our new product launches March 15th.")# Search — query vector sent to miner, results decrypted locallyresults = client.query("revenue figures")for r in results:print(r["score"], r["metadata"]) # decrypted metadata here
Note
The first ingest to a new namespace auto-registers it on the miner with a PBKDF2 hash of your key. The key itself is never stored anywhere on the network.
Separate teams, separate keys
python
# Engineering teameng_client = EngramClient(miner, namespace="eng", namespace_key=ENG_KEY)eng_client.ingest("Our CI pipeline config and architecture notes.")# Sales team — completely isolated, different keysales_client = EngramClient(miner, namespace="sales", namespace_key=SALES_KEY)sales_client.ingest("Pipeline deal notes — Q1 close expected.")# Engineering can't see sales data and vice versaresults = eng_client.query("deal notes") # returns nothing — scoped to 'eng'
Encryption details
| Property | Value |
|---|---|
| Algorithm | AES-256-GCM (authenticated encryption) |
| Key derivation | PBKDF2-HMAC-SHA256, 100 000 iterations |
| KDF salt | Namespace name (UTF-8) |
| IV | Random 12 bytes per message (os.urandom) |
| Auth tag | 16 bytes — detects tampering |
| Wire format | base64url( IV[12] ‖ ciphertext ‖ tag[16] ) |
| Stored as | metadata["_enc"] field on the miner |
| Library | cryptography >= 42.0 (AESGCM) |
Because GCM is an authenticated mode, any attempt to tamper with stored ciphertext is detected on decryption and raises a ValueError rather than silently returning garbage.
Key management
Your namespace key never leaves your machine — only a PBKDF2 hash is stored on the miner (used to reject requests with wrong keys at the access-control layer).
Rotating keys
To rotate a namespace key, use the admin API on localhost. Note that rotating the access control key does not re-encrypt existing data — you must re-ingest if you also want to change the encryption key.
bash
curl -X POST http://localhost:8091/namespace \-H "Content-Type: application/json" \-d '{"action": "rotate","namespace": "acme-docs","key": "old-secret-key","new_key": "new-secret-key-min-16-chars"}'
Deleting a namespace
bash
curl -X POST http://localhost:8091/namespace \-H "Content-Type: application/json" \-d '{"action": "delete", "namespace": "acme-docs", "key": "your-secret-key"}'
Note
Deleting a namespace removes access control but does not wipe the encrypted vectors from the miner's store. The data remains as unreadable ciphertext until the miner's collection is pruned.
Admin API
The /namespace endpoint is restricted to 127.0.0.1 (localhost only). It cannot be called from the public internet.
| Action | Required fields | Description |
|---|---|---|
| create | namespace, key | Explicitly pre-create a namespace (optional — auto-created on first ingest) |
| delete | namespace, key | Remove namespace registration (data remains encrypted on disk) |
| rotate | namespace, key, new_key | Change the access-control key (does not re-encrypt stored data) |
| list | — | List all registered namespace names (no keys or hashes returned) |
bash
# List namespacescurl http://localhost:8091/namespace -d '{"action":"list"}' -H "Content-Type: application/json"# {"namespaces": ["acme-docs", "eng", "sales"]}
Threat model
| Threat | Protected? |
|---|---|
| Miner reads your stored text | ✓ Yes — ciphertext only on disk |
| Miner reads your query text | ✓ Yes — only float vector sent |
| Network eavesdropper reads data in transit | Partial — use TLS termination for full protection |
| Attacker guesses your namespace key | ✓ Yes — PBKDF2 with 100k iterations makes brute-force expensive |
| Miner tampers with stored data | ✓ Yes — GCM auth tag detects modification |
| Miner returns results from another namespace | ✓ Yes — server-side namespace filter + client-side decryption |
| Someone with the key reads your data | No — key holders have full access (by design) |
| Miner infers content from embedding vectors | Partial — vectors encode semantic meaning; use private miner for full isolation |
Tip
For maximum privacy, run your own miner on private infrastructure. The namespace system protects text content from third-party miners, but embedding vectors inherently encode semantic information that a sophisticated attacker could analyze.
engram docs · v0.1edit on github →