CLIO Core

Docs · Machine learning

lda: latent Dirichlet allocation.

Latent Dirichlet allocation fitted by batch variational EM, the equivalent of Spark MLlib's LDA with the EM optimizer. Rows are bag-of-words counts over a fixed vocabulary; each EM iteration is one paged pass with the per-document E-step in the kernel and the M-step on the host.

Module
ml_lda
Header
clio_ml/lda.h
Rows
one count per vocabulary word
Limits
k at most 16

What it does

Expected topic-word counts are summed in fixed row order, so the topics are bit-reproducible. The model holds the topic-word distributions only; per-document topic mixtures are not returned.

Rows

One document per row with vocab_size columns of word counts. clio_ml.COLUMNS["lda"] is 0.

Python

python
import numpy as np, clio_ml as cm
docs = np.random.default_rng(1).poisson(2.0, (4096, 8)).astype(np.float32)

r = cm.fit_lda(docs, k=2, iters=3)
topics = np.asarray(r.model.topics).reshape(r.model.K, r.model.V)   # each row sums to 1
print(topics.argsort(axis=1)[:, ::-1][:, :3])                        # top-3 words per topic
  • fit_lda(X, **options).
  • Options (LdaOptions): k (3), iters (30), inner (20, per-document E-step iterations), alpha (0.1, document-topic Dirichlet), eta (0.01, topic-word smoothing), seed (1), threads (256).
  • Result LdaResult: model (LdaModel with V, K and topics, K*V) and stats.
  • No predictor.

C++

cpp
struct LdaOptions {
  u32 k = 3;            // <= 16
  u32 iters = 30;
  u32 inner = 20;
  float alpha = 0.1f;
  float eta = 0.01f;
  u32 seed = 1;
  u32 threads = 256;
  ClusterExchange cluster;
};
struct LdaModel {
  u32 V = 0, K = 0;
  std::vector<float> topics;  // K*V
  std::vector<float> Flatten() const;
};
struct LdaResult { LdaModel model; VecStats stats; };
LdaResult FitLda(Dataset &ds, const LdaOptions &opt);
LdaResult FitLdaPlain(const float *data, u64 n_docs, u32 vocab, u64 elems_per_page,
                      u32 nblocks, const LdaOptions &opt);
LdaResult FitLdaReference(/* same arguments */);

The ChiMod is module ml_lda.

Related: word2vec, frequent itemsets.