linreg · CLIO Core

What it does

Each iteration is one paged pass that accumulates the mean-squared-error gradient over all rows, followed by one update with a fixed learning rate. There is no regularisation and no early stop. The loss recorded before each update is the mean squared error.

Rows

n_features feature columns followed by the target. The Dataset therefore has n_features + 1 columns. clio_ml.COLUMNS["linreg"] is 1.

Python

python

import numpy as np, clio_ml as cm
rng = np.random.default_rng(1)
X = rng.normal(size=(4096, 7)).astype(np.float32)
y = X[:, 0] - 2 * X[:, 1] + 0.5
data = np.column_stack([X, y]).astype(np.float32)

r = cm.fit_linreg(data, iters=20, learning_rate=0.1)
w, b = r.weights[:7], r.weights[7]
print(r.loss_history[0], "->", r.loss_history[-1])
print(cm.predict("linreg", r, data[0, :7]))    # linear value for one row

fit_linreg(X, **options) calls the GLM engine with GlmKind.linear.
Options (GlmOptions): iters (100), learning_rate (0.1), threads (256).
Result GlmResult: weights (n_features coefficients, then the intercept), loss_history (mean loss before each update), stats.
predict("linreg", r, x) returns the linear value w . x + b.

C++

cpp

enum class GlmKind { kLinear = 0, kLogistic = 1, kSvm = 2 };
struct GlmOptions {
  u32 iters = 100;
  float learning_rate = 0.1f;
  u32 threads = 256;
  ClusterExchange cluster;
};
struct GlmResult {
  std::vector<float> weights;       // F coefficients, then intercept
  std::vector<float> loss_history;  // mean loss before each update
  VecStats stats;
};
GlmResult FitGlm(GlmKind kind, Dataset &ds, const GlmOptions &opt);
GlmResult FitLinearRegression(Dataset &ds, const GlmOptions &opt);   // FitGlm(kLinear, ...)
GlmResult FitGlmPlain(GlmKind kind, const float *data, u64 n_rows, u32 n_features,
                      u64 elems_per_page, u32 nblocks, const GlmOptions &opt);
GlmResult FitGlmReference(GlmKind kind, const float *data, u64 n_rows, u32 n_features,
                          u64 elems_per_page, u32 nblocks, u32 iters, float learning_rate);
float GlmPredict(GlmKind kind, const std::vector<float> &w, const float *x);

clio_ml/linear_regression.h only includes glm.h. The ChiMod is module ml_linreg.