Population Stability Index (PSI)

• Population Stability Index (PSI) is a metric that measures distributional discrepancies between baseline and current data using defined bins.
• It is computed through various formulations, including discrete bins and continuous analogs, serving as a practical indicator in risk and model monitoring.
• PSI finds applications in credit scoring, post-deployment machine learning, and federated learning, supporting decisions on model adjustments and client selection.

Population Stability Index (PSI) is a discrepancy measure for comparing two populations through their distributional proportions, most commonly a baseline or development population against a current, review, target, or deployment population. In the literature represented here, PSI is used in three principal ways: as an industry-standard scorecard monitoring statistic in banking and credit risk, as a label-free detector of post-deployment distribution shift in machine learning systems, and as a client-level non-IIDness measure for personalized federated learning under label skew. Across these domains, PSI functions as a scalar summary of distributional mismatch, but its interpretation is strongly conditioned by binning, application context, and the decision rule used around it (Pisanie et al., 2023, Khademi et al., 2023, Jimenez-Gutierrez et al., 31 May 2025).

1. Definition and mathematical forms

In credit-risk and scorecard settings, PSI is typically defined on discrete categories or bins. If $\mathbf{q}=(q_1,\dots,q_k)$ denotes the development distribution and $\mathbf{P}=(P_1,\dots,P_k)$ the empirical review proportions, then the paper on scorecard stability defines

$$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$

A banking formulation writes

$$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$

where $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$ is the model-development population, $\hat p_j=n_j/n$ are current category proportions, and the quantity is treated as a sample-based estimator of the symmetric Kullback–Leibler divergence

$$J(\mathbf p,\mathbf p_0) = \sum_{j=1}^{B}(p_j-p_{0j})(\log p_j-\log p_{0j}).$$

A continuous analog for score distributions is also given: $$\mathrm{PSI} = \int (f(s)-f_N(s)) \ln \frac{f(s)}{f_N(s)} \, ds.$$ These formulations are described as a symmetric version of Kullback–Leibler divergence, a form of J-divergence, and, in the federated-learning literature, as coinciding with the Jeffreys divergence (Pisanie et al., 2023, Potgieter et al., 2023, Pomazanov, 7 Jul 2025, Jimenez-Gutierrez et al., 31 May 2025, Jimenez-Gutierrez et al., 23 Dec 2025).

A monitoring-oriented formulation emphasizes predefined bins $\{b_i\}_{i=1}^B$ and source and target proportions $q_{s,b_i}$ and $\mathbf{P}=(P_1,\dots,P_k)$0, so that PSI compares bin-wise proportions between a source distribution and a target distribution. In that treatment, PSI is always in $\mathbf{P}=(P_1,\dots,P_k)$1, is dependent on the binning scheme, and is therefore not directly comparable across applications unless the binning and context are standardized (Khademi et al., 2023).

2. Role in credit risk, scorecards, and binary choice models

In banking and credit risk, PSI is used to compare the current population of scores, applicants, account outcomes, attributes, or PD groups against the model-development population. The operational question is whether the current population still resembles the original one closely enough that the model remains representative. Continuous attributes are first grouped into bins, nominal variables are treated as categories, and PSI is then computed from the corresponding proportions. The same logic is applied to PD groups or score bands in order to assess whether the scorecard outputs have shifted (Potgieter et al., 2023, Pisanie et al., 2023).

The best-known interpretation rule is the set of Lewis constants:

• $\mathbf{P}=(P_1,\dots,P_k)$2: the current population resembles the original; no action needed.
• $\mathbf{P}=(P_1,\dots,P_k)$3: investigate.
• $\mathbf{P}=(P_1,\dots,P_k)$4: substantial change; model reconstruction may be needed.

A related paper on binary choice models also cites a regulatory threshold from the Bank of Russia, stating that $\mathbf{P}=(P_1,\dots,P_k)$5 is an unacceptable zone. In that paper, PSI is treated together with the Kolmogorov–Smirnov statistic as a key indicator of model stability, and population instability is linked to a reduction in real discriminatory power relative to the historical Gini. The same paper reports a rough empirical relationship of the form $\mathbf{P}=(P_1,\dots,P_k)$6, while stressing that the relationship is not exact (Potgieter et al., 2023, Pomazanov, 7 Jul 2025).

The credit-risk literature is also explicit about PSI’s weaknesses. Fixed thresholds are described as arbitrary, sample-size insensitive, and insensitive to the number of categories $\mathbf{P}=(P_1,\dots,P_k)$7. A fixed PSI value can correspond to a wide and sample-dependent range of actual distribution changes. In simulations and empirical examples, PSI with standard thresholds can recommend reconstruction too often in small samples and fail to detect meaningful shifts often enough in large samples. PSI can also understate practically important drift, and it can become infinite when a category is present in one population but absent in the other. For these reasons, the cited papers treat PSI as a screening tool rather than a complete decision procedure (Potgieter et al., 2023, Pisanie et al., 2023).

Several alternatives are proposed or reviewed in the same literature. The Population Resemblance Statistic (PRS) is defined as

$\mathbf{P}=(P_1,\dots,P_k)$8

with calibration through the non-central chi-square distribution and decision rules tied to sample size, category count, tolerance, and desired power. Other reviewed alternatives include the Du Pisanie–Visagie statistic, effect-size-based monitoring, the overlapping statistic, and classical goodness-of-fit procedures such as KS. A consistent theme is that monitoring should encode business materiality and not rely mechanically on folklore thresholds (Potgieter et al., 2023, Pisanie et al., 2023).

3. Deployment monitoring and robustness after model release

In post-deployment machine learning, PSI is framed as a distribution-shift detector comparing development-stage data with deployment-stage data. This setting is explicitly motivated by the fact that models can fail silently after deployment when the input environment changes through noise, lighting, corruption, sensor issues, or other covariate shifts. PSI is attractive here because it does not require labels and can be computed from input data alone, which makes it suitable for continuous monitoring when ground truth is unavailable (Khademi et al., 2023).

The monitoring paper applies PSI to computer vision data from the LISA Traffic Light Dataset. The source distribution is the original clean development data, and the target distribution is simulated deployment-stage data produced by adding Gaussian noise, speckle noise, or salt-and-pepper noise. Because PSI operates on numeric values, it is applied to image pixel values; for color images, PSI is computed separately on the three RGB channels. In one experiment, noise intensity is varied on day-time and night-time images, and PSI generally increases with Gaussian variance, with speckle variance, and with salt-and-pepper intensity. In several cases, PSI exceeds $\mathbf{P}=(P_1,\dots,P_k)$9 even at relatively small Gaussian variance, and a similar sensitivity is reported for speckle noise during the day. In a second experiment over 10,954 day-time images and 11,527 night-time images, PSI again captures distribution shifts across many images, with high to moderate sensitivity for Gaussian and salt-and-pepper noise in both day and night, moderate sensitivity for speckle noise during the day, and lower sensitivity for speckle noise at night (Khademi et al., 2023).

The same work is careful not to equate PSI with direct performance diagnosis. PSI is presented as one component of a broader monitoring stack that should include performance metrics such as accuracy, AUC, and F-measure, along with confidence measures, explainability and root-cause analysis, and context-specific thresholds and mitigation strategies. The paper also lists several caveats: PSI depends on bin choice, lacks spatial locality information in computer vision, is best suited to numerical features, and should be interpreted as a shift indicator rather than as a precise measure of failure probability (Khademi et al., 2023).

4. PSI as a non-IIDness measure in personalized federated learning

Recent federated-learning work reinterprets PSI as a heterogeneity metric for label-skewed clients. In this setting, the server compares each client’s local label probability mass function with the global label probability mass function of the federation. For client $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$0, the label-skew PSI is defined as

$$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$1

where $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$2 is the number of classes, $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$3 is the global probability mass function for class $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$4, and $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$5 is client $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$6’s local probability mass function. The weighted federation-level summary is

$$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$7

where $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$8 is the number of clients, $$\Psi(\mathbf{q},\mathbf{P})=\sum_{j=1}^{k}(P_j-q_j)\log\!\left(\frac{P_j}{q_j}\right).$$9 is the number of samples on client $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$0, and $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$1 is the total number of samples. Lower $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$2 or $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$3 indicates a more homogeneous federation, while higher values indicate stronger heterogeneity (Jimenez-Gutierrez et al., 31 May 2025).

The PSI-PFL framework uses this quantity as a client-selection signal for personalized federated learning. Its workflow is: each client sends only label frequencies to the server; the server aggregates them into the global label distribution; the server computes each client’s $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$4; clients with PSI below a threshold $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$5 are selected for training; the global model is trained using those selected clients; and, once convergence is reached, the resulting global model is distributed to all clients, including those not selected during training. The stated intuition is that clients whose local label histograms are closer to the global label histogram are more homogeneous, so selecting them reduces update divergence and leads to more stable aggregation (Jimenez-Gutierrez et al., 31 May 2025).

This line of work is explicit that it targets label skew, treated as one of the most damaging forms of non-IIDness in federated learning because class-proportion mismatch causes local gradients to point in different directions. In experiments on ACSIncome, Dutch, CelebA, and Sent140, with non-IID structure generated through Dirichlet partitioning controlled by parameter $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$6, PSI-PFL is reported to improve global accuracy over FedAvg, CL, FedProx, FedYogi, FedAdam, FedAdagrad, HACCS, and FedCLS, with up to a 10% relative improvement under non-IID conditions. The same study reports fairer local behavior, including a much lower Client Parity value under highly heterogeneous settings, lower average distance to centralized learning, and lower variability in client-level performance differences. One highlighted result states that under $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$7, PSI-PFL reaches a CP of $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$8, whereas some baselines are around $$\mathrm{PSI} = \sum_{j=1}^{B} (\hat p_j-p_{0j})(\log \hat p_j-\log p_{0j})\mathbb{I}(\hat p_j>0),$$9 or $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$0; under the most heterogeneous cases, PSI-PFL achieves an average distance of about $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$1 from the centralized-learning target, versus about $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$2 for some baselines (Jimenez-Gutierrez et al., 31 May 2025).

5. PSI feature representations and clustering in federated learning

A later clustered extension, Clust-PSI-PFL, treats PSI not only as a scalar but also as a feature representation for grouping clients with similar label-skew patterns. In addition to the client-level divergence $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$3, the paper defines class-wise contributions

$\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$4

and constructs the client feature vector

$\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$5

These features are standardized to zero mean and unit variance and clustered with K-means++; the number of clusters is chosen through a silhouette-based search over $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$6. The paper reports that this procedure usually selects a small number of clusters, often around 3–4, so the overhead stays low (Jimenez-Gutierrez et al., 23 Dec 2025).

Within that framework, PSI is presented as lightweight, interpretable, client-specific, and directly useful for clustering because it requires only label counts and scales with the number of classes rather than the number of samples. The paper contrasts PSI with Hellinger distance, Jensen–Shannon distance, and Earth Mover’s distance. Its main empirical claim is that $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$7 is more informative than those alternatives for the label-skew setting: under both Dirichlet and Similarity partition protocols, $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$8 follows an exponential-decay trend as data becomes more IID, while EMD can be non-monotonic. In feature-importance analyses using LightGBM, SVM, MLP, and regression tree models to predict partition parameters, $\mathbf{p}_0=(p_{01},\ldots,p_{0B})^\top$9 is reported as the strongest predictive metric (Jimenez-Gutierrez et al., 23 Dec 2025).

The empirical results are correspondingly stronger than in the earlier selection-only framework. Across six datasets, two partition protocols, and multiple client sizes, Clust-PSI-PFL is reported to deliver up to 18% higher global test accuracy than baselines such as CFL and FedSoft, especially under severe label skew, while near-IID settings do not suffer a penalty. Fairness is measured through

$\hat p_j=n_j/n$0

and its dispersion $\hat p_j=n_j/n$1, with smaller values better. The paper reports a 37% relative improvement in fairness under severe non-IID conditions and states that under a pathological Similarity partition $\hat p_j=n_j/n$2, Clust-PSI-PFL achieved near-zero $\hat p_j=n_j/n$3. At the same time, the method is explicitly limited to label skew, assumes that clients can report label-frequency summaries, and does not include a privacy mechanism such as DP or MPC in the implementation (Jimenez-Gutierrez et al., 23 Dec 2025).

6. Interpretation, calibration problems, and recurring misconceptions

A central controversy in PSI practice concerns thresholding. In deployment monitoring, there is stated to be no established reliable mechanism for choosing PSI thresholds, and thresholding is described as ad hoc in practice. In credit-risk monitoring, the Lewis constants are widely used but criticized for not being derived from a formal statistical decision framework, not adjusting for sample size, and not adjusting for the number of categories. A plausible implication is that PSI values are most defensible when interpreted within a standardized binning scheme and a domain-specific monitoring policy rather than as universally transferable cutoffs (Khademi et al., 2023, Potgieter et al., 2023).

The federated-learning literature responds to the same issue differently by calibrating an application-specific threshold $\hat p_j=n_j/n$4 on the empirical distribution of client PSI values. The proposed workflow is to compute PSI for every client, examine the empirical distribution, choose candidate thresholds from the 10th, 25th, 50th, 75th, and 90th percentiles, train PSI-PFL for each candidate, and then pick the threshold that gives the best validation or test performance, typically global accuracy. The reported behavior is that low and medium-low thresholds often give the best global accuracy in many non-IID settings, whereas highly heterogeneous settings with many clients may benefit from a higher $\hat p_j=n_j/n$5 because of greater representativeness (Jimenez-Gutierrez et al., 31 May 2025).

Another recurring misconception concerns the scope of the term “population stability.” The paper “Population Stability and Momentum” develops a demographic framework based on newly added population, removed population, population momentum, and a replacement metric $\hat p_j=n_j/n$6. It does not introduce the standard Population Stability Index used in applied statistics, credit scoring, model monitoring, or federated learning. The demographic replacement metric and the statistical PSI both function as distance-from-equilibrium ideas, but the paper states that they are mathematically and substantively distinct (Rao, 2015).

Across the cited literature, the most stable conclusion is therefore narrow rather than universal. PSI is a simple and widely used summary of population shift, but its numeric value is shaped by representation choices such as bins or classes, and its operational meaning depends on the monitoring problem in which it is embedded. In credit risk it remains an industry-standard baseline, in post-deployment ML it serves as a label-free warning signal for covariate shift, and in personalized federated learning it becomes a lightweight heterogeneity score that can drive selection or clustering. The same breadth of use explains both its persistence and the repeated cautions against treating a PSI threshold as a context-free law (Pisanie et al., 2023, Khademi et al., 2023, Jimenez-Gutierrez et al., 31 May 2025).