FlexCode: Nonparametric CDE & Generative Recommendation

• FlexCode is a framework for conditional density estimation that reformulates the problem into a series of regression tasks using an orthogonal-series expansion.
• It allows flexible integration of off-the-shelf regression methods to handle high-dimensional, multimodal, or heteroscedastic data distributions.
• A separate application of FlexCode in generative recommendation employs dual-codebook architectures to balance collaborative and semantic signals in item prediction.

Searching arXiv for papers on FlexCode and closely related references. Using the arXiv search tool to retrieve relevant papers. FlexCode most commonly denotes a fully nonparametric approach to conditional density estimation (CDE) that reformulates CDE as a non-parametric orthogonal series problem where the expansion coefficients are estimated by regression (Izbicki et al., 2017). In this formulation, a univariate conditional density is expanded in an orthonormal basis on the response domain, and truncation converts density estimation into a finite collection of regression problems (Chin et al., 30 Jan 2026). The name also appears in later work on generative recommendation, where FlexCode denotes a popularity-aware dual-codebook framework rather than a conditional-density estimator (Hui et al., 15 Nov 2025).

1. Terminology and scope

In the CDE literature, FlexCode is presented as a basis-expansion approach that estimates the full conditional density $p(y\mid x)$ rather than only a point forecast (Dalmasso et al., 2019). The original method was introduced as “Flexible nonparametric conditional density estimation via regression,” with the central claim that one can efficiently estimate conditional densities in high dimensions by drawing upon the success in high-dimensional regression (Izbicki et al., 2017).

A closely related spelling, FlexCoDE, is used for software implementations in Python and R (Dalmasso et al., 2019). The capitalization difference reflects packaging and naming conventions rather than a different statistical method. The 2026 review places FlexCode alongside Hall and Yao’s single-index method, DeepCDE, the generative conditional distribution sampler, and conditional denoising diffusion probabilistic models as one representative family in the modern CDE landscape (Chin et al., 30 Jan 2026).

The method targets settings in which the conditional law is multimodal, asymmetric, or heteroscedastic, and standard regression is therefore insufficient (Izbicki et al., 2017). This suggests that FlexCode is best understood not as a single estimator with a fixed inductive bias, but as a modular reduction from density estimation to regression.

2. Orthogonal-series formulation

The core identity underlying FlexCode is an orthogonal-series expansion of the conditional density. For an orthonormal basis $\{\phi_j\}$ of $L^2(\mathbb{R})$, the method uses

$$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$

with

$$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$

After truncation to the first $J$ terms, the estimator becomes

$$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$

The key observation is that each coefficient function $\beta_j(x)$ is a regression function of $x$ (Chin et al., 30 Jan 2026).

The original paper states the same idea with notation $\psi_k$: $\{\phi_j\}$0 where

$\{\phi_j\}$1

This equivalence makes the method an “orthogonal series” estimator whose difficulty is displaced from density estimation onto coefficient regression (Izbicki et al., 2017).

Common basis choices include cosine, Fourier, and wavelet systems (Chin et al., 30 Jan 2026). The 2017 formulation additionally lists orthogonal polynomials, indicator or spline bases for discrete or mixed responses, and tensor-product bases for multivariate outputs (Izbicki et al., 2017). In a quasar photometric-redshift application, the basis was a Fourier basis on the interval $\{\phi_j\}$2 (Nakazono et al., 2024).

3. Estimation procedure

The practical workflow begins by rescaling or standardizing $\{\phi_j\}$3 so that the chosen orthonormal basis applies, selecting a basis family, and choosing a truncation level $\{\phi_j\}$4 (Chin et al., 30 Jan 2026). For each basis index $\{\phi_j\}$5, the training response is transformed to

$\{\phi_j\}$6

and a nonparametric regression model is fit to the pairs $\{\phi_j\}$7 (Chin et al., 30 Jan 2026). Prediction for a new $\{\phi_j\}$8 requires evaluating all fitted coefficient regressors and summing the truncated series.

The 2017 paper describes this as forming pseudo-responses

$\{\phi_j\}$9

then fitting any off-the-shelf regression learner to $L^2(\mathbb{R})$0 on $L^2(\mathbb{R})$1 for each $L^2(\mathbb{R})$2 (Izbicki et al., 2017). Examples given across the literature include random forests, $L^2(\mathbb{R})$3-nearest neighbors, sparse additive methods, support distribution machines, kernel-ridge regression, local polynomial regression, spectral-series regression, functional kernel regression, and XGBoost-style regressors in the software examples (Chin et al., 30 Jan 2026). Because the $L^2(\mathbb{R})$4 regressions are independent, they can be run in parallel (Chin et al., 30 Jan 2026).

A common misconception is that FlexCode specifies a single predictive model. In fact, the regression engine is deliberately interchangeable. Another misconception is that the raw series estimate is automatically a proper density. The software-oriented treatment notes that small negative “ringing” or non-normalization can appear after summation; one may threshold densities below $L^2(\mathbb{R})$5 and renormalize, with $L^2(\mathbb{R})$6 itself selected by validation (Dalmasso et al., 2019).

4. Model selection, diagnostics, and software

FlexCode does not maximize a log-likelihood directly in the formulations summarized here. Instead, truncation level $L^2(\mathbb{R})$7 (or $L^2(\mathbb{R})$8) and regressor hyper-parameters are selected by minimizing an empirical CDE loss on held-out data (Dalmasso et al., 2019). The empirical loss used for tuning is

$L^2(\mathbb{R})$9

up to an additive constant independent of $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$0 (Dalmasso et al., 2019). The original FlexCode algorithm similarly chooses $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$1 by minimizing a validation density loss over candidate truncation levels (Izbicki et al., 2017).

The software ecosystem described in 2019 includes four CDE packages in Python and R—\texttt{NNKCDE}, \texttt{RFCDE}, \texttt{FlexCode}, and \texttt{DeepCDE}—together with the \texttt{cdetools} package for computing CDE loss and calibration diagnostics (Dalmasso et al., 2019). The FlexCode API is described as scikit-learn style: a constructor specifies basis type, maximum basis size, regression class, and hyper-parameter grid; fit(X_train,y_train) fits the coefficient regressions; tune(X_val,y_val) selects $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$2 and threshold $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$3; and predict(X_test,B=ngrid) returns an array of density values on a grid (Dalmasso et al., 2019).

Calibration diagnostics include the probability integral transform (PIT) for scalar responses and HPD coverage for multivariate responses (Dalmasso et al., 2019). Uniform PIT or HPD histograms indicate well-calibrated densities, whereas U-shaped PIT indicates under-dispersion and hump-shaped PIT indicates over-dispersion (Dalmasso et al., 2019). These diagnostics are implemented in \texttt{cdetools} as cde_loss, pit_coverage or cdfcoverage, and hpd_coverage (Dalmasso et al., 2019).

5. Theory, strengths, and limitations

The original theoretical analysis decomposes integrated $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$4 risk into a truncation bias term and a regression estimation term (Izbicki et al., 2017). Under smoothness in $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$5 and an estimation rate for the coefficient regressions in the effective dimension of $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$6, the variance scales as

$$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$7

the bias scales as

$$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$8

and optimizing $$f(y\mid x)=\sum_{j=1}^{\infty}\beta_j(x)\,\phi_j(y),$$9 yields an overall rate that can be near the minimax rate when the regression procedure adapts to low effective dimension rather than ambient dimension (Izbicki et al., 2017).

The 2026 review does not re-derive these rate theorems, but summarizes the balance as

$$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$0

where $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$1 denotes smoothness in $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$2 and $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$3 denotes smoothness of the coefficient regressions in $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$4 (Chin et al., 30 Jan 2026). This formulation makes explicit that FlexCode inherits its statistical behavior from both the basis truncation and the chosen regressor.

Its principal strengths are listed consistently across sources: it is conceptually straightforward, extremely flexible because any off-the-shelf regression learner may be plugged in, naturally parallelizable across basis indices, and computationally efficient in typical low- to moderate-dimensional $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$5 (Chin et al., 30 Jan 2026). Its main limitations are equally explicit: it requires truncation to finite $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$6; sharp local features in $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$7 may need very large $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$8; series estimates can go negative or fail to integrate to one; performance can be sensitive to heteroscedasticity; and choosing $$\beta_j(x)=\int f(y\mid x)\phi_j(y)\,dy = E[\phi_j(Y)\mid X=x].$$9 and regressor hyperparameters adds tuning overhead (Chin et al., 30 Jan 2026). The broader software paper also states that there is no one-size-fits-all CDE method (Dalmasso et al., 2019), which places FlexCode among adaptable but not universally dominant estimators.

6. Empirical evaluations and scientific applications

The original empirical studies span simulated and real-world data, including photometric galaxy data, Twitter data, and line-of-sight velocities in a galaxy cluster (Izbicki et al., 2017). In the simulation suite, different regression back ends were paired with the same FlexCode reduction: sparse additive methods for irrelevant covariates, spectral regression for manifold structure, random forests for mixed data, functional kernel regression for functional covariates, and support-distribution-style kernels for sample-set inputs (Izbicki et al., 2017). The unifying point is that the conditional-density estimator adapts to structure in $J$0 by changing the regression layer rather than the density representation.

A later astrophysical application used FlexCoDE for quasar photometric-redshift estimation with feature vectors built from S-PLUS colours and optional GALEX and WISE bands (Nakazono et al., 2024). In that study, missing values for tree methods were replaced by an out-of-range flag before computing colours, the basis size $J$1 was chosen by 5-fold cross-validation to minimize empirical CDE loss, and each coefficient regression used a Random Forest with 400 trees, min_samples_leaf = 2, no maximum depth, and bootstrap sampling (Nakazono et al., 2024). On the held-out test set, FlexCoDE improved from $J$2 with broad-band data only to $J$3 with broad plus narrow bands, while BMDN achieved $J$4 and $J$5 respectively (Nakazono et al., 2024). The same paper reports that the inclusion of narrow-band information provided better estimates of the probability density functions obtained with FlexCoDE and BMDN (Nakazono et al., 2024).

The 2026 comparative review evaluates FlexCode against DeepCDE, GCDS, and DDPM under a unified evaluation framework using MSE(mean), MSE(sd), and $J$6 (Chin et al., 30 Jan 2026). On simple models M1–M5, FlexCode achieves low $J$7 (approximately $J$8) and MSE(mean) approximately $J$9, comparable to DeepCDE and GCDS (Chin et al., 30 Jan 2026). Under heteroscedastic but still roughly Gaussian noise in M6–M7, FlexCode’s Wasserstein error is approximately $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$0 for M6 and $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$1 for M7, worse than DeepCDE, GCDS, and DDPM (Chin et al., 30 Jan 2026). On more complex non-Gaussian mixtures in M8–M9, it regains competitive performance, with $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$2 approximately $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$3 on M8 and $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$4 on M9, outperforming neural-based methods in certain cases (Chin et al., 30 Jan 2026). Per-epoch training time places FlexCode, via random forests, among the fastest methods and similar to DeepCDE, whereas GCDS is the slowest (Chin et al., 30 Jan 2026).

7. Later reuse of the name in generative recommendation

In 2025, the name FlexCode was reused for a different model class in generative recommendation (Hui et al., 15 Nov 2025). That framework addresses next-item prediction as autoregressive sequence generation over discrete item tokens and argues that a single shared codebook forces semantic signals and collaborative signals to collapse into the same embedding space (Hui et al., 15 Nov 2025). Its proposed solution is a popularity-aware framework that adaptively allocates a fixed token budget between a collaborative filtering codebook and a semantic codebook (Hui et al., 15 Nov 2025).

The architecture factorizes item representation into two separate vector-quantized codebooks: a collaborative codebook learned from user–item interaction embeddings via an RQ-VAE over a SASRec-style encoder, and a semantic codebook learned from textual or categorical metadata embeddings via an RQ-VAE over a pretrained text encoder (Hui et al., 15 Nov 2025). A lightweight Mixture-of-Experts router takes item-level features—log popularity, age, sparsity, and uncertainty—and outputs $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$5, which controls the split

$$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$6

at inference time (Hui et al., 15 Nov 2025). The dual-codebook tokens are concatenated and fed into a standard autoregressive Transformer decoder (Hui et al., 15 Nov 2025).

The recommendation paper reports that FlexCode achieves the best Recall@5/10 and NDCG@5/10 across three public datasets, that on Amazon-Sports NDCG@10 is $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$7 versus URL’s $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$8, and that on KuaiRand NDCG@10 is $$\widehat f(y\mid x)=\sum_{j=1}^{J}\widehat\beta_j(x)\,\phi_j(y).$$9 versus URL’s $\beta_j(x)$0 (Hui et al., 15 Nov 2025). Relative to SASRec on the industrial dataset, it improves NDCG@10 by $\beta_j(x)$1 and HR@10 by $\beta_j(x)$2 (Hui et al., 15 Nov 2025). The paper also identifies limitations: added architectural complexity, reliance on metadata quality for the semantic channel, and a router based only on simple item-level statistics (Hui et al., 15 Nov 2025).

The coexistence of these two uses of “FlexCode” is terminological rather than methodological. In statistics and uncertainty quantification, FlexCode denotes an orthogonal-series CDE framework built from coefficient regressions (Izbicki et al., 2017). In generative recommendation, it denotes a dual-codebook token-allocation architecture for balancing collaborative precision and semantic generalization (Hui et al., 15 Nov 2025).