Install the Context Transfer Engine.

Which install to pick

Every method on the general install page builds the CTE, because the CMake option that controls it, CLIO_CORE_ENABLE_CTE, defaults to ON and the packaging files pin it ON. What differs is which client you get.

• pip. pip install iowarp-core gives you the clio_run runtime command, the clio_cte_bench benchmark and the Python module clio_cte_core_ext. The wheel is built with CLIO_CORE_ENABLE_PYTHON=ON and the cache and replication modules pinned ON, so the default interposition chain above the core is present. The wheel does not install C++ headers or CMake packages. Use it for Python programs.
• conda. The iowarp-core package on the iowarp channel is built from the release preset. It installs the runtime and the C++ libraries into the environment prefix.
• Docker. iowarp/deploy-cpu runs clio_run start with the default configuration, which composes the CTE core and its chain. Clients in other containers or on the host reach it over the runtime's port.
• Spack. spack install iowarp installs the runtime and C++ libraries. Add +python for the Python module.
• Source. A source build is needed for GPU tiers (hbm, pinned), for the S3 and GCS block devices (CLIO_ENABLE_AMAZON_DRIVE, CLIO_ENABLE_GOOGLE_CLOUD), for the compressor module (CLIO_CTE_ENABLE_COMPRESS) and for the I/O adapters. The install page lists the presets and options.

Python

The module is built with nanobind from context-transfer-engine/wrapper/python/core_bindings.cc. It needs a running runtime. Start one in another terminal, then import the module with CLIO_WITH_RUNTIME=0 so the client attaches instead of starting a second runtime.

$ pip install iowarp-core
$ clio_run start

import os
os.environ["CLIO_WITH_RUNTIME"] = "0"

import clio_cte_core_ext as cte
assert cte.clio_init(cte.RuntimeMode.kClient, False)
assert cte.initialize_cte("", cte.PoolQuery.Dynamic())

tag = cte.Tag("install_check")
tag.PutBlob("hello", b"hello", 0)
assert tag.GetBlob("hello", 5, 0) == b"hello"
print("ok")

Set environment variables before the import. The client reads them once when it connects. On Windows, bind the runtime and the client to the loopback address: CLIO_BIND_ADDR=127.0.0.1 and CLIO_SERVER_ADDR=127.0.0.1.

C++

An installed tree exports the client as a CMake target. Find the umbrella package and link the target; the header directory comes with it.

find_package(clio-core CONFIG REQUIRED)   # iowarp-core is the legacy name of the same package
add_executable(cte_check cte_check.cc)
target_link_libraries(cte_check clio::cte::core_client)

The umbrella package includes the clio_cte_core module package, which defines clio::cte::core_client and clio::cte::core_runtime. The historical names clio_cte::core_client and wrp_cte::core_client are emitted as aliases. The client header is clio_cte/core/core_client.h; the runtime entry points are in clio_runtime/clio_runtime.h.

#include <clio_runtime/clio_runtime.h>
#include <clio_cte/core/core_client.h>
#include <string>

int main() {
  if (!clio::run::CLIO_INIT(clio::run::RuntimeMode::kClient)) return 1;
  if (!clio::cte::core::CLIO_CTE_CLIENT_INIT()) return 1;
  clio::cte::core::Tag tag("install_check");
  std::string in = "hello", out(5, '\0');
  tag.PutBlob("hello", in.data(), in.size());
  tag.GetBlob("hello", out.data(), out.size());
  return out == in ? 0 : 2;
}

Run it with a runtime already started by clio_run start. CLIO_INIT(kClient) attaches to it; CLIO_INIT(kClient, true) starts a runtime inside the process when none is running, which is what the adapter tests do.

Configuration the engine needs

The runtime's configuration file must compose a clio_cte_core pool with at least one storage tier. The default file installed to ~/.clio/clio.yaml by the wheel, and <prefix>/etc/clio/clio_default.yaml in other installs, already does this with a RAM tier and a file tier under CLIO_STORAGE_ROOT. The API reference lists the keys, and the compose tutorial walks through adding tiers.

Where to go next

• Usage: tags, blobs, queries and asynchronous calls with examples.
• API reference: every signature, the Python module and the YAML keys.