DISTRIBUTIONSCORE

Entropy & Uncertainty

Entropy scores surprise: equilibrium, cyclic, random and complex outcomes sit at different levels of uncertainty.

λ mean ≈ variance
counts per window · recreated schematic

How it works

  • Shannon entropy sums −p·log p over outcomes: zero when certain, highest when all outcomes are equally likely.
  • It measures surprise in bits — a fair coin is 1 bit, a loaded coin less.
  • Equilibria and cycles score low, pure noise scores high, complex systems sit in between.
Use → Classify how predictable a system is, compare category mixes, measure diversity.
Watch out → High entropy is not the same as important — pure noise scores highest of all.
Try it → Compute the entropy of your product-category mix this year vs last year.
REDCAPE jobs → Explain · Communicate · Predict
Free sources → SFI Information Theory tutorial • Wikipedia: Entropy (information theory) (CC BY-SA)
◍ Mentor soonSee REDCAPE playbook →

Attach to a lattice — combines with

Different families, shared jobs. Disagreement between them is where insight lives.

DIFFUSION
Broadcast Model
One source reaches a fixed pool at a steady rate: fast early, then fading returns — the r-shaped curve.
SHARED JOBS: PREDICT · COMMUNICATE
LINEAR & NONLINEAR
Categorical & Threshold Models
Buckets and cutoffs turn messy attributes into testable if-then rules.
SHARED JOBS: PREDICT · COMMUNICATE
LEARNING
Diversity Prediction Theorem
Crowd error equals average error minus diversity. Diverse models beat the lone genius.
SHARED JOBS: PREDICT · COMMUNICATE

More in Distributions

Normal Distribution
Bell-curve averaging: many small independent effects sum to a predictable center with thin tails.
Central Limit & Six Sigma
Sample averages settle to normal; variation reduction pays until returns diminish.
Power Law / Long Tail
Rich-get-richer: a few huge winners, many tiny players. Straight line on log-log.
← Poisson ArrivalsLinear Regression →