Docs · Context FS

Context FS: a mounted directory over the stack.

Context FS is the filesystem module of CLIO Core plus the clio_cte_fuse daemon that mounts it. Any program that opens, reads and writes files uses it unchanged. Underneath, every file is tags and blobs in the Context Transfer Engine, so the tiers, replication, durability and search you compose apply to the mount as well.

Mount
clio_cte_fuse <
Filesystem pool
560.0
Page size
1 MiB
Drivers
libfuse3, macFUSE, WinFsp

What it is

Context FS has two parts.

  • The filesystem module, clio_cte_filesystem, runs inside the Context Runtime at pool 560.0. It owns the namespace: directories, inode records, file sizes, links and extended attributes. It stores all of that in the Context Transfer Engine (CTE) beneath it.
  • The FUSE daemon, clio_cte_fuse, is a client of that module. It mounts a directory through the operating system's FUSE driver and turns every system call into tasks on the filesystem pool and the CTE chain.

There is no metadata server. Each directory's rows are split into table blocks, and each block has a home container that holds it in memory and journals it to a node-local log. Every node caches what it reads.

What problem it solves

Applications that only know POSIX files cannot use a key-value store, a tiered buffer or a search index directly. Context FS gives them an ordinary directory while the stack decides where each page lives: RAM for the working set, NVMe or disk for colder data, a persistent copy for restarts, and an index entry for keyword search. Nothing has to be copied out of the mount to be searched or imported by the other engines, because the files already are CTE data.

Data model

  • A file is a tag. The tag is named by the file's path. Its attributes (mode, owner, times) live in the tag's extended-attribute section.
  • File bytes are 1 MiB page blobs. Page n of a file is the blob named "n" in the file's tag. A write touches only the pages it covers.
  • The logical size is owned by the stream pool (clio_cte_stream, 565.0), which also merges deferred appends from several nodes. The filesystem creates it with defaults if it is not composed.
  • Directories are inode-table blocks. Each block maps names to inode ids and is homed on one container. Blocks split when they pass dir_split_entries rows (default 1024).
  • Durability follows the chain. Pages travel through the cache, indexer and replication modules to the CTE core. fsync returns once the bytes are on a tier at the configured persistence level, and the namespace journal is synced with it.

Filesystem and FUSE design explains the inode table, the shared-memory mirror that answers stat, and the write path in detail.

How it relates to the other technologies

  • Context Runtime hosts the filesystem module as a pool and runs clio_run, the daemon the mount attaches to. The runtime's documentation covers clio_run and the compose file; this section covers only the filesystem entries of that file.
  • Context Transfer Engine stores every page and inode record. The CTE's C++ and Python clients can read files written through the mount by tag name.
  • Context Assimilation Engine imports external data into the same CTE pools, so imported objects and mounted files share tiers and the index.
  • Context Exploration Engine queries files on the mount by name, keyword or time, served by the indexer pool the mount writes through.
  • Memory expansion and machine learning use the same tiers from inside a program instead of through a mount.

Where to go next

  • Install: which installs include the daemon, and the FUSE driver each platform needs.
  • Quickstart: start the runtime, mount, copy files, search them from Python, stop.
  • Usage: mounting on Linux, macOS and Windows, what the default compose creates, changing tiers, multi-node mounts.
  • API reference: supported operations and limits, the clio_cte_fuse command, environment variables, and the compose keys of the filesystem and stream modules.