CLIO Core

Docs · Machine learning

fm: factorization machine regression.

Factorization machine regression, the equivalent of Spark MLlib's FMRegressor: linear terms plus pairwise feature interactions through rank-factor_size factors, squared loss, full-batch Adam. One paged pass per step.

Module
ml_fm
Header
clio_ml/fm.h
Rows
features, then target

What it does

Per-parameter gradients are summed in fixed row order and Adam runs on the host, so the model is bit-reproducible across paging, block counts and cluster nodes.

Rows

n_features feature columns followed by the regression target. clio_ml.COLUMNS["fm"] is 1.

Python

python
import numpy as np, clio_ml as cm
rng = np.random.default_rng(1)
X = rng.normal(size=(4096, 7)).astype(np.float32)
y = X[:, 0] * X[:, 1] + 0.5 * X[:, 2]
data = np.column_stack([X, y]).astype(np.float32)

r = cm.fit_fm(data, factor_size=4, iters=20, learning_rate=0.05)
print(r.loss_history[-1])
print(cm.predict("fm", r, data[0, :7]))      # predicted value
  • fit_fm(X, **options).
  • Options (FmOptions): factor_size (4), iters (300), learning_rate (0.05, Adam), init_std (0.1), seed (1), threads (256).
  • Result FmResult: model (FmModel with nf, k and params laid out as [w0 | w (nf) | V (nf*k)]), loss_history (mean 0.5 * r^2 before each step), stats.
  • predict("fm", r, x) returns the predicted value.

C++

cpp
struct FmOptions {
  u32 factor_size = 4;
  u32 iters = 300;
  float learning_rate = 0.05f;  // Adam
  float init_std = 0.1f;
  u32 seed = 1;
  u32 threads = 256;
  ClusterExchange cluster;
};
struct FmModel {
  u32 nf = 0, k = 0;
  std::vector<float> params;  // [w0 | w (nf) | V (nf*k)]
  std::vector<float> Flatten() const;
};
struct FmResult {
  FmModel model;
  std::vector<double> loss_history;
  VecStats stats;
};
FmResult FitFm(Dataset &ds, const FmOptions &opt);
FmResult FitFmPlain(const float *data, u64 n_rows, u32 nf, u64 elems_per_page,
                    u32 nblocks, const FmOptions &opt);
FmResult FitFmReference(/* same arguments */);
float FmPredict(const FmModel &m, const float *x);

The ChiMod is module ml_fm.

Related: ALS, linear regression.