CLIO Core

Docs · Context FS

Run Context FS.

Starting Context FS is two commands: start the runtime, then mount. This page covers both on each operating system, explains what the default configuration composes, shows where to change the tiers, and lists the mount settings that matter on one node and on several.

Runtime
clio_run start
Mount
clio_cte_fuse <
Config
~/.clio/clio.yaml or CLIO_SERVER_CONF
Filesystem pool
560.0

How it starts

Two processes are involved. The runtime, started with clio_run start, reads the configuration file and composes the pools in its compose list: block devices, the CTE core, the interposition chain, the stream pool and the filesystem module. The FUSE daemon, clio_cte_fuse, is a client of that runtime. It creates or binds the filesystem pool itself, so no extra compose step is needed before mounting. The clio_run command is documented in the Context Runtime API reference.

Always mount with CLIO_WITH_RUNTIME=0. Without it the daemon starts a private runtime in its own process, and Python clients, clio_cae and other mounts will not see the same files.

Mount on each platform

Linux

shell
$ clio_run start &amp;
$ mkdir -p ~/clio-mnt
$ CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/clio-mnt -f

Every argument after the mountpoint goes to libfuse3: -f stays in the foreground, -d traces each request, -s runs single-threaded, and -o allow_other lets other users open the mount (it needs user_allow_other in /etc/fuse.conf). Unmount with the FUSE tool, not Ctrl+C, and then stop the runtime:

shell
$ fusermount3 -u ~/clio-mnt
$ clio_run stop

Installing the FUSE packages and running in containers is covered in FUSE on Linux.

macOS

The pip wheel does not include the macOS adapter, so build it once against macFUSE as described in Install. Then:

shell
$ clio_run start &amp;
$ mkdir -p ~/clio-mnt
$ CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/clio-mnt -f

On macOS 15.4 and newer with macFUSE 5.1 or later, -o backend=fskit selects FSKit and avoids the kernel extension; the mountpoint must then be under /Volumes. Unmount with umount ~/clio-mnt, or diskutil unmount force if a process still holds it open. See FUSE on macOS.

Windows

Windows uses WinFsp, and the mountpoint is a drive letter or an empty folder path whose parent exists. Use two PowerShell windows. In the first:

powershell
PS&gt;$env:CLIO_BIND_ADDR = "127.0.0.1"
PS&gt;$env:CLIO_SERVER_ADDR = "127.0.0.1"
PS&gt;clio_run start

In the second:

powershell
PS&gt;$env:CLIO_WITH_RUNTIME = "0"
PS&gt;$env:CLIO_SERVER_ADDR = "127.0.0.1"
PS&gt;clio_cte_fuse Z: -f

To unmount, stop the daemon, then the runtime:

powershell
PS&gt;Stop-Process -Name clio_cte_fuse -Force
PS&gt;clio_run stop

The default configuration binds the dashboard to 127.0.0.1, and networking examples on Windows should do the same. Installing WinFsp, the Microsoft Defender note and the -o uid=-1,gid=-1 option for git are in FUSE on Windows.

What the default compose creates

The default configuration composes one chain of pools. Each entry's next_pool_id names the pool below it, and every entry must come after the entry it points to. Read from the top, a file operation travels like this:

  • clio_cte_filesystem, pool 560.0. The filesystem module the FUSE daemon drives. Paths map to CTE tags and offsets to 1 MiB page blobs. Its next_pool_id is 563.0, the top of the chain. Its metadata_log_path, under ${CLIO_STORAGE_ROOT}/cfs_namespace_log, journals the directory table blocks this node is home for.
  • clio_cte_stream, pool 565.0. Owns each file's logical size and merges deferred appends. The filesystem creates it with defaults if it is not composed; compose it before the filesystem to configure it. Its staged appends go to 561.0, the replication pool, so they skip the cache.
  • clio_cte_cache, pool 563.0. The locality layer and chain top. A put lands below before it is acknowledged and updates a raw node-local copy; reads serve that copy. min_score: 0.5 is the floor below which the organizer will not rescore a cache copy. Points at 564.0.
  • clio_cte_indexer, pool 564.0. Owns the term index behind SemanticSearch. Writes are forwarded unchanged and enqueue a dirty key; a periodic sweep tokenizes them. Its snapshot and log live at ${CLIO_STORAGE_ROOT}/cte_indexer_index. Points at 561.0.
  • clio_cte_replication, pool 561.0. The reliability layer. replication_factor: 1 keeps only the blob's own copy, made durable by fsync and the periodic data flush. replica_score: 0.2 is where durable copies are placed. Points at 512.0.
  • clio_cte_core, pool 512.0. The storage engine every client resolves to. Its storage list defines two tiers: a RAM tier (ram::cte_ram_tier1, score 1.0, capacity 0g meaning 80% of DRAM) and a file tier at ${CLIO_STORAGE_ROOT}/cte_disk_tier.dat (score 0.2, 10 GB cap, persistence_level: temporary). The metadata write-ahead log at ${CLIO_STORAGE_ROOT}/cte_metadata_log is what lets data survive a restart.
  • clio_bdev, pool 301.0. The RAM block device that provides the shared-memory allocator other modules use. The only strictly required entry.

The default also composes clio_cae_core at 400.0, the Context Assimilation Engine entry point, pointed at the chain top so imported data is indexed. It is not part of the file path; see the CAE documentation. The compressor (562.0) and the summarizer (401.0) are present in the file but commented out.

Why the chain is ordered this way

The indexer sits above the compressor so its read-backs see uncompressed bytes. The cache sits on top so that the zero-copy shared-memory read path stays available even for blobs that are compressed or replicated below. The interposition chain explains the design.

Performance

A RAM tier is faulted in lazily, so the first writes into it pay page faults. Start the runtime with CLIO_PREFAULT=0 (or a size such as 4GB) to fault it in at startup. That and the other tuning knobs are covered in Performance and tuning.

Changing the tiers

Tiers are the storage list of the clio_cte_core entry. Each entry has a path (ram::<name> for DRAM or a file path on the device), a bdev_type (ram or file; hbm and pinned on GPU builds), a capacity_limit, a score from 0.0 (slowest) to 1.0 (fastest), and an optional persistence_level of volatile (the default), temporary or long_term. Copy the default file, edit the list, and point the runtime at your copy:

yaml · ~/clio-tiers.yaml (excerpt)
  - mod_name: clio_cte_core
    pool_name: cte_main
    pool_query: local
    pool_id: "512.0"
    storage:
      - path: "ram::hot"
        bdev_type: "ram"
        capacity_limit: "16GB"
        score: 1.0
      - path: "/mnt/nvme/clio/tier"
        bdev_type: "file"
        capacity_limit: "200GB"
        score: 0.9
      - path: "/mnt/hdd/clio/tier"
        bdev_type: "file"
        capacity_limit: "4TB"
        score: 0.2
        persistence_level: "long_term"
    performance:
      metadata_log_path: "${CLIO_STORAGE_ROOT}/cte_metadata_log"
      transaction_log_capacity: "32MB"
shell
$ clio_run stop
$ export CLIO_SERVER_CONF=~/clio-tiers.yaml
$ clio_run start &amp;
$ CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/clio-mnt -f &amp;

Three rules when editing:

  • Keep pool_id: "512.0" on the core entry. The FUSE daemon and every client look for the storage engine there.
  • Keep metadata_log_path. Without it the persistent tier's bytes survive a restart but nothing records which file they belong to.
  • Durable copies only go to a tier whose persistence_level is temporary or long_term. If you change the durable tier's score, set the replication entry's replica_score to match.

A tier can also be registered while the runtime runs, from the dashboard's pool page or its REST route; such tiers last until the next restart. Compose local tiers walks through all of this with a four-tier example.

Keeping the namespace across restarts

Three logs decide what survives clio_run stop and the next clio_run start:

  • metadata_log_path on the clio_cte_core entry: which blobs exist and where their bytes are.
  • metadata_log_path on the clio_cte_filesystem entry: the directory table blocks this node is home for. Unset, directory blocks are stored as CTE blobs instead, with one blob write per change.
  • log_path on the clio_cte_stream entry: file sizes and pending appends. Unset, file sizes do not survive a stop.

The filesystem journal is node-local. A node that never returns takes its directory blocks with it. clio_run start --fresh discards all three logs on that node.

One node or many

The daemon reads the runtime's networking.hostfile. With more than one line it switches to multi-node defaults; CLIO_FUSE_MULTINODE=0 or 1 overrides the decision.

  • Attribute cache. One node: the kernel caches attributes for 1 s. Several nodes: 0 s, so a reader sees a size another node just extended. CLIO_FUSE_ATTR_CACHE_S sets it; use 0 on one node when several processes must see each other's metadata at once.
  • Appends. One node: the kernel's cached size supplies the O_APPEND offset. Several nodes: appends are staged and merged at the file's stream home. CLIO_FUSE_APPEND=kernel|reserve|deferred chooses explicitly.
  • Writeback caching is off everywhere. CLIO_FUSE_WRITEBACK=1 turns it on; it is only safe with a single writer that fsyncs.
  • Batched creates and asynchronous close are on for a single node and off on several.

Every variable, with its default, is in the API reference. For a filesystem deployment that should fail fast while a node is down, the runtime's CLIO_NET_DEAD_FAIL_FAST=1 makes operations on names the dead node owns return EIO at once instead of stalling.

Advanced compose configurations