CLIO Core

Docs · Machine learning

mlp: multilayer perceptron classifier.

A one-hidden-layer perceptron classifier, the equivalent of Spark MLlib's MultilayerPerceptronClassifier: features to hidden sigmoid units to num_classes softmax outputs, cross-entropy loss, full-batch Adam. Each step is one paged pass with forward and backward per row in the kernel.

Module
ml_mlp
Header
clio_ml/mlp.h
Rows
features, then class label
Limits
hidden at most 32, num_classes at most 8

What it does

Every parameter's gradient is summed in fixed row order and Adam runs on the host, so the trained network is bit-identical across paging, block counts and cluster nodes. For deeper networks or custom architectures see the torch module.

Rows

n_features feature columns followed by the class label. clio_ml.COLUMNS["mlp"] is 1.

Python

python
import numpy as np, clio_ml as cm
rng = np.random.default_rng(1)
X = rng.normal(size=(4096, 7)).astype(np.float32)
y = (X[:, 0] + X[:, 1] > 0).astype(np.float32)
data = np.column_stack([X, y]).astype(np.float32)

r = cm.fit_mlp(data, hidden=16, num_classes=2, iters=20, learning_rate=0.05)
print(r.loss_history[0], "->", r.loss_history[-1])
print(cm.predict("mlp", r, data[0, :7]))     # predicted class
  • fit_mlp(X, **options).
  • Options (MlpOptions): hidden (16), num_classes (2), iters (200), learning_rate (0.05, Adam), seed (1), threads (256).
  • Result MlpResult: model (MlpModel with nf, H, C and params laid out as [W1 | b1 | W2 | b2]), loss_history (mean cross-entropy before each step), stats.
  • predict("mlp", r, x) returns the class index.

C++

cpp
struct MlpOptions {
  u32 hidden = 16;       // <= 32
  u32 num_classes = 2;   // <= 8
  u32 iters = 200;
  float learning_rate = 0.05f;  // Adam
  u32 seed = 1;
  u32 threads = 256;
  ClusterExchange cluster;
};
struct MlpModel {
  u32 nf = 0, H = 0, C = 0;
  std::vector<float> params;  // [W1 | b1 | W2 | b2]
  std::vector<float> Flatten() const;
};
struct MlpResult {
  MlpModel model;
  std::vector<double> loss_history;
  VecStats stats;
};
MlpResult FitMlp(Dataset &ds, const MlpOptions &opt);
MlpResult FitMlpPlain(const float *data, u64 n_rows, u32 nf, u64 elems_per_page,
                      u32 nblocks, const MlpOptions &opt);
MlpResult FitMlpReference(/* same arguments */);
u32 MlpPredict(const MlpModel &m, const float *x);

The ChiMod is module ml_mlp.

Related: 1-D CNN, LSTM, GNN, torch.