Don't trust us. Check.
A statistics package is only as good as its numbers. Every check below runs in the open test suite on every change, and you can run it yourself with cargo test.
NIST Statistical Reference Datasets
The U.S. National Institute of Standards and Technology publishes datasets with certified results computed in high-precision arithmetic, designed to expose numerical weaknesses.Modellum runs all of them through its command language:
| Suite | Datasets | Result |
|---|---|---|
| Linear regression | 11 | LRE ≥ 7 on every statistic, except where double precision itself caps it (Wampler5 at 5.7, as with LAPACK) |
| Analysis of variance | 11 | LRE ≥ 7, except SmLs07–09 at ≈ 4, the ceiling for exact arithmetic on those inputs |
| Univariate summary statistics | 9 | LRE ≥ 7 |
| Nonlinear regression | 26 | 51 of 52 problems solved from both NIST starting points at LRE ≥ 6.8; Lanczos1's residuals are rounding error; from BoxBOD's first start nl stops where MINPACK does, and warns |
LRE (log relative error) is roughly the number of correct significant digits.
Stata's published output
Survey design effects from the examples in Stata's [SVY] estat manual reproduce to every printed digit on the Stata Press datasets.
Same answer on any number of cores
Parallel code is a common source of results that change from run to run. Modellum splits work into fixed-size chunks and combines partial results in a fixed order, so the bits never depend on how many threads ran. The test suite enforces it:
- Hundreds of random expressions, with missing and extended-missing values, −0, lags, by-groups and random draws, must give identical bits and identical error codes on 1 thread and on all of them.
- Grouped statistics, summaries, regressions, logit, probit, Poisson, GLM, negative binomial and survey replicate refits must match exactly on 1, 2, 3 and all threads.
- Random numbers come from the counter-based Philox4x32-10 generator, so each draw depends only on the seed, the row and the call.
The GPU agrees with the CPU
With set gpu always, every NIST check is rerun on the graphics card. Regressions must match the CPU to 10⁻¹⁰, drop the same collinear columns, and repeat bit-for-bit on the same card. Designs too ill-conditioned for the fast mixed-precision method fall back to the CPU automatically.
File formats
The .dta and .sav readers are written from the published specifications and tested against files produced by pandas and pyreadstat.
Coming next
Golden-file comparisons against R and Stata output for every estimator, published as a validation report with each release. Found a number you disagree with? Pleaseemail support@modellum.app with the data and the command.