CLIO Core

Docs · Machine learning

gmm: Gaussian mixture model.

A Gaussian mixture model fitted by expectation-maximisation with diagonal covariance, the equivalent of Spark MLlib's GaussianMixture restricted to diagonal covariances. Each EM iteration is one paged pass: the E-step runs per row in the kernel and the host does the M-step.

Module
ml_gmm
Header
clio_ml/gmm.h
Rows
features, then one ignored column

What it does

The kernel computes responsibilities by log-sum-exp and accumulates [sum r, sum r x, sum r x^2] per component together with the log-likelihood. Weights start uniform and variances at 1; you supply the initial means. Bit-reproducible across paging and block counts.

Rows

n_features feature columns followed by one ignored column (a label, for instance), so the Dataset has n_features + 1 columns. clio_ml.COLUMNS["gmm"] is 1.

Python

python
import numpy as np, clio_ml as cm
rng = np.random.default_rng(1)
X = rng.normal(size=(4096, 7)).astype(np.float32)
data = np.column_stack([X, np.zeros(4096)]).astype(np.float32)

r = cm.fit_gmm(data, k=2, iters=5)          # init_means defaults to X[:k]
m = r.model
print(m.weight, np.asarray(m.mean).reshape(m.K, m.nf))
print(r.loglik[0], "->", r.loglik[-1])      # mean log-likelihood per E-step
print(cm.predict("gmm", r, data[0, :7]))    # most likely component
  • fit_gmm(X, init_means=None, **options). For an in-memory array the initial means default to the first k rows' features; for a Dataset you must pass init_means (shape k x features).
  • Options (GmmOptions): k (2), iters (20), var_floor (1e-6), threads (256).
  • Result GmmResult: model, loglik (mean log-likelihood at each E-step), stats. model (GmmModel): nf, K, weight (K), mean (K*nf), var (K*nf).
  • predict("gmm", r, x) returns the most likely component.

C++

cpp
struct GmmOptions {
  u32 k = 2;
  u32 iters = 20;
  float var_floor = 1e-6f;
  u32 threads = 256;
  ClusterExchange cluster;
};
struct GmmModel {
  u32 nf = 0, K = 0;
  std::vector<float> weight, mean, var;  // K, K*nf, K*nf
  std::vector<float> Flatten() const;
};
struct GmmResult {
  GmmModel model;
  std::vector<double> loglik;
  VecStats stats;
};
GmmResult FitGmm(Dataset &ds, const std::vector<float> &init_means, const GmmOptions &opt);
GmmResult FitGmmPlain(const float *data, u64 n_rows, u32 nf, u64 elems_per_page,
                      u32 nblocks, const std::vector<float> &init_means,
                      const GmmOptions &opt);
GmmResult FitGmmReference(/* same arguments */);
u32 GmmPredict(const GmmModel &m, const float *x);

The ChiMod is module ml_gmm.

Related: k-means, fuzzy c-means.