New built-in Newton optimizer
(backend = "builtin") for PLN, ZIPLN and PLNnetwork:
envelope-theorem Newton steps with strong Wolfe line search, no
dependency on NLOPT. Substantially faster and more accurate than nlopt
on large datasets with full covariance.
PLNPCA’s "builtin" backend rewritten as a
profiled trust-region Newton: the variational block
(M, S) is profiled out with a per-observation Newton
VE-step, and the loadings (B, C) are optimised with a
saddle-aware trust-region Newton on the resulting objective (analytic
Schur-complement Hessian-vector products, Jacobi-preconditioned
Steihaug-CG) — replacing the previous spectral projected-gradient
"builtin". It reliably reaches a higher variational bound
than "nlopt" at comparable-to-better speed; tuning keys
cg_maxit, maxit_out, ftol_out,
gtol, delta0 in config_optim (see
?PLNPCA_param). "nlopt" remains the default
for PLNPCA.
Backend defaults revisited package-wide, based
on extensive benchmarking: PLN and PLNPCA keep "nlopt" (PLN
now consistently faster thanks to profiled = TRUE, see
below); PLNnetwork and ZIPLNnetwork now default to
"builtin", which finds a better optimum at a modest speed
cost; ZIPLN keeps its "builtin" default. All backends
remain configurable via the backend argument; see the
corresponding *_param() documentation for the trade-offs.
The torch backend is now clearly marked
experimental everywhere.
Quality and speed improvements:
config_optim$profiled = TRUE is now the default for
full-covariance nlopt fits (faster, slightly better loglik); ZIPLN’s
variational step now optimises (M, ψ, R) jointly via Newton
instead of sequentially; PLNPCA shares a single SVD initialisation
across ranks and can warm-start from a pre-fitted PLNfit
for large ranks (inception/init_method, see
?PLNPCA_param); ZIPLN’s starting point no longer relies on
pscl::zeroinfl (now an internal LM + binomial GLM routine),
which is both much faster and a better starting point —
pscl is no longer a dependency.
Fixed a critical nlopt convergence bug affecting PLN/PLNPCA: ill-conditioned covariate scaling could trigger the XTOL stopping criterion after very few iterations, well before convergence. The built-in backend was never affected; nlopt is now also fixed via better parameter scaling.
CovTraitsBase in covariance_pln.h) instead of
being duplicated per structure. PLNPCA and ZIPLN’s variational step now
reuse the same machinery instead of separate hand-rolled
implementations, removing a substantial amount of duplicated code and
fixing minor inefficiencies along the way (e.g. ZIPLN’s VE-step used to
treat the precision matrix as dense even for diagonal/spherical
covariance, at unnecessary O(np^2) cost).{backend}_optimize_{structure} convention.Uniform covariate normalization: a
normalize_covariates() helper (zero mean, unit variance per
column) is now applied consistently in all optimize()
methods (PLN, PLNPCA, PLNnetwork, ZIPLN). This makes the nlopt XTOL
criterion scale-invariant and stabilises the torch backend.
Parallelism backend:
future.apply::future_lapply is replaced by
parallel::mclapply throughout (stability selection for
PLNnetwork / ZIPLNnetwork). Use options(mc.cores = N) to
set the number of cores.
Bug fixes: PLNnetwork/ZIPLNnetwork’s inception
(warm-start) model didn’t inherit
ftol_em/maxit_em from the user’s
config_optim, silently falling back to defaults and
producing a wrong penalty grid; the PLNPCA rank-model objective used
A − Y where it should use A − Y ⊙ Z; various
ZIPLN prediction/initialization fixes (#146, #149, #150, #152).
microcosm data now included (#153, #154); AIC added for PLN and ZIPLN classes (#151); other fixes (#155).
CRAN check fixes (no user-visible effect):
removed an unused -fopenmp compilation flag in
src/Makevars that was inadvertently turning on Armadillo’s
internal OpenMP parallelisation and inflating the CPU/elapsed time ratio
of several examples; fixed a GCC -Wmismatched-new-delete
false positive in src/packing.cpp’s internal test helper by
restructuring the code (no diagnostic-suppressing pragma
involved).
variance_jackknife() and
variance_bootstrap() to prevent estimation recycling,
results from those functions are now comparable to doing jackknife /
bootstrap “by hand”.predict() function for PLNfit model
to (i) return fitted values if newdata is missing or (ii) perform one VE
step to improve fit if responses are provided (fix issue #114)scale argument compute_offset() to force the
offsets (RLE, CSS, GMPR, Wrench) to be on the same scale as the counts,
like TSS.clusters) is not of the form 1:K_maxPLNLDA() and
changing extract_model() to conform with
model.frame()$VEStep() for PLN-PCA, dealing
with low rank matrices$project() for PLN-PCA, used to
project newdata into PCA space$latent_pos() which is equivalent to
active binding $latentNEWS.md file to track changes to the
package.