Extend the storage system

A map, not a walkthrough

This page names the interfaces and files to change. Unlike Write a chain operator, it has no step-by-step example that was built and run for this site.

How tiers work

Each entry in the storage engine's storage list becomes a tier backed by a block device pool. A block device answers a small set of methods: allocate blocks, free blocks, write, read, and report statistics. The storage engine needs nothing else from a tier, so anything that answers those methods can be a tier.

There are two ways to attach one:

Give the tier a bdev_type and a path, and the storage engine creates a block device of that type.
Give it existing_pool_id, and it uses a pool you composed yourself. path, bdev_type and capacity_limit become optional. The erasure-coded array in Compose data operators is attached this way.

A block device module

A module of your own can be a tier if it answers the block device methods with the block device's own ids and task types. This works outside the tree: build it like the chain operator, compose it, and attach it with existing_pool_id. The erasure-coded array, clio_safe_bdev, is the in-tree example. Its clio_mod.yaml reuses the block device ids:

yaml · context-runtime/modules/safe-bdev/clio_mod.yaml

# Methods reused from bdev (same IDs as bdev so reused task types match)
kAllocateBlocks: 10     # Allocate data blocks
kFreeBlocks: 11         # Free data blocks
kWrite: 12              # Write data
kRead: 13               # Read data
kGetStats: 14           # Get performance statistics

Its implementation, in context-runtime/modules/safe-bdev/, shows how to answer each method by calling other block devices. A module that stores data in a remote service would answer them by calling that service.

A new bdev type

To make a new type selectable with bdev_type, add a transport to the block device module. This changes CLIO Core itself.

Step	Where
Implement `BdevTransport`: `Init`, `Destroy`, `AllocateBlocks`, `FreeBlocks`, `WriteBlocks`, `ReadBlocks`, `GetCapacity`, `GetRemainingSize`, and optionally `FlushAllocLog` and `Sync`	`context-runtime/modules/bdev/include/clio_runtime/bdev/transports/bdev_transport.h`. The GCS transport, `gcs_bdev_transport.h` in the same directory, is a short example.
Add a value to `BdevType` and map the YAML string to it	`CreateParams::LoadConfig` in `context-runtime/modules/bdev/include/clio_runtime/bdev/bdev_tasks.h`
Create the transport for the new type	`BdevTransportFactory::Create` in `context-runtime/modules/bdev/src/transports/bdev_transport.cc`
Offer it in the dashboard's block device form	The type list in `context-runtime/modules/bdev/src/bdev_runtime.cc`
Let the storage engine accept it as a tier type	The `bdev_type` check in `context-transfer-engine/core/src/core_config.cc` and the string-to-type mapping in `context-transfer-engine/core/src/core_runtime.cc`

Per the contributor guide, document new configuration keys in context-runtime/config/clio_default.yaml.

Placement and tiering policies

Placement and tiering run inside the storage engine and use its internal state, so new policies are added to CLIO Core itself.

Policy	Interface	Register it in	Selected by
Placement: which tier a new blob goes to	`DataPlacementEngine::SelectTargets` in `context-transfer-engine/core/include/clio_cte/core/dpe/dpe.h`	`DpeFactory::CreateDpe` and `StringToDpeType` in `core/src/dpe/dpe.cc`, plus the allowed values in `core/src/core_config.cc`	`dpe: {dpe_type: ...}`
Tiering: when blobs move between tiers	`DataOrganizer::Reorganize` in `context-transfer-engine/core/include/clio_cte/core/data_organizer/data_organizer.h`	`DataOrganizerFactory::Get` in `core/src/data_organizer/data_organizer.cc`, plus the allowed values in `core/src/core_config.cc`	`organizer: ...`

To move data from outside without a new policy, change blob scores from a client, as Compose data operators shows. The storage engine moves each blob to the tier that matches its new score.