---
title: Extra Trees Regressor (ETR)
url: https://www.emergentmind.com/topics/extra-trees-regressor-etr
type: topic
---

# Extra Trees Regressor (ETR)

The Extra Trees Regressor (ETR), also known as “Extremely Randomized Trees,” is an ensemble machine learning model consisting of decision trees with maximally randomized split selection at both the feature and threshold levels. ETR has demonstrated efficacy in regression tasks that require both high predictive accuracy and robust generalization, particularly in physical sciences for properties prediction directly from high-dimensional descriptors. Its defining attributes—extensive tree decorrelation through randomization and the absence of bootstrapped samples—render ETR a fast, low-variance, and high-performance estimator for applications in computational materials science and related high-throughput screening tasks [2511.13202].

## 1. Algorithmic Structure and Randomization Principles

ETR constructs an ensemble of individual decision trees, each built from the full training dataset without resampling. At each tree node, a subset of candidate features ($m$ features, where $m$ is determined by the model’s `max_features` hyperparameter) is randomly selected. For each candidate feature within this subset, instead of greedily testing all possible split thresholds as in standard Random Forests (RF), ETR samples a split threshold uniformly at random from the interval defined by the feature’s observed minimum and maximum values in the current node’s sample.

Among all $m$ random splits, the split associated with the maximum impurity decrease, typically quantified as mean squared error (MSE) for regression, is chosen. Consequently, predictive diversity among individual trees is amplified, reducing ensemble variance. The principal distinctions from RF are: (1) ETR eschews data bootstrapping (each tree sees all data), (2) split thresholds are sampled randomly instead of optimally selected, and (3) extensive ensemble averaging is employed.

## 2. Model Hyperparameters and Default Setting Rationale

In the cited application, ETR was instantiated via the scikit-learn (v0.24) `ExtraTreesRegressor` class, with unmodified defaults reflecting standard settings. The key hyperparameters were:

| Hyperparameter        | Value      | Effect                          |
|----------------------|------------|----------------------------------|
| n_estimators         | 100        | Number of trees in ensemble      |
| criterion            | "mse"      | Split impurity metric            |
| max_depth            | None       | Unlimited depth (pure leaves)    |
| min_samples_split    | 2          | Node split minimum sample count  |
| min_samples_leaf     | 1          | Leaf node minimum sample count   |
| max_features         | "auto"     | $\sqrt{p}$, $p$ = feature count  |
| bootstrap            | False      | Full data for each tree          |
| random_state         | None       | No fixed seed                    |

No additional hyperparameter tuning was performed due to both strong intrinsic model randomization and the presence of significant regression signal in the selected data [2511.13202]. This suggests robustness of ETR to hyperparameter specification within the tested context.

## 3. Feature Engineering and Data Preparation

The regression dataset comprised 4,127 sample points across 150 crystalline compounds, each accompanied by temperature-dependent ab initio (DFT-computed) lattice thermal conductivity, $\kappa_L(T)$, spanning $100\text{–}1000$ K. The regression target was the base-10 logarithm of $\kappa_L$ to address multi-order-of-magnitude variability and skewness, i.e.,
$$
y_i = \log_{10}\bigl(\kappa_{L,i}\bigr)
$$
Features were generated by the MAGPIE framework to encapsulate both compositional and crystal-structural information, such as atomic weights, electronegativities, coordination numbers, and other elemental descriptors. Initial feature dimensionality was $p_0=271$, with temperature as the 272nd input. Dimensionality reduction proceeded via: (1) a variance threshold ($\sigma^2<$ 0.16) eliminating 64 features, and (2) pairwise Pearson correlation filtering ($|\rho|>0.80$), yielding a final feature matrix of $p=53$ descriptors. Standard supervised learning splits were applied (80% training, 20% test), and 12 compounds were held entirely out to assess model transferability to unseen chemistries.

## 4. Model Training, Cross-Validation, and Computational Scaling

Ten-fold repeated cross-validation was conducted with each run comprising random shuffling, 80/20 train-test splits, model fitting, and prediction. Each fold thus trained on approximately 3,288 samples and tested on the remaining 822. For each partition, the following train-test steps were executed:
1. Model instantiation: `ExtraTreesRegressor(...)` with defaults
2. Fit: `model.fit(X_\text{train}, y_\text{train})`
3. Predict: $\hat{y}_\text{test} = \text{model.predict}(X_\text{test})$
4. Record performance metrics (RMSE, $R^2$, MAE)

Each 10-repetition CV run required approximately 3.33 minutes wall-time on a multi-core NVIDIA DGX-3 platform leveraging scikit-learn’s parallel computation capabilities.

## 5. Predictive Performance Evaluation

All performance metrics were calculated on the logarithmic regression target, $y_i = \log_{10} \kappa_{L,i}$. Definitions are:

- **Root Mean Square Error (RMSE):**
  $$
  \mathrm{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^n (y_i-\hat{y}_i)^2}
  $$
- **Coefficient of Determination ($R^2$):**
  $$
  R^2 = 1 - \frac{\sum_{i=1}^n (y_i-\hat{y}_i)^2}{\sum_{i=1}^n (y_i-\bar{y})^2}
  $$
- **Mean Absolute Error (MAE):**
  $$
  \mathrm{MAE} = \frac{1}{n}\sum_{i=1}^n |y_i-\hat{y}_i|
  $$

Average test set scores (across ten repetitions):
- $R^2 = 0.9994$
- $\mathrm{RMSE} = 0.0466$ (in $\log_{10}[W\,m^{-1}\,K^{-1}]$)
- $\mathrm{MAE} = 0.0249$

For held-out generalization (12 compounds), $R^2 = 0.961$ over predictions spanning $100\text{–}900$ K. These results demonstrate DFT-level regression fidelity and generalizability to high- and low-symmetry materials.

## 6. Model Characteristics Underlying ETR’s Superior Performance

ETR outperformed alternative regressors in this application due to multiple structural and statistical properties:
- **Enhanced Tree Decorrelation:** Random thresholds at each split decorrelate trees within the ensemble more than the greedy-split paradigm of RF, decreasing overall variance.
- **Negligible Bias Increase:** The use of 100 trees ensures that, on average, splits closely approximate optimal positioning despite randomness.
- **Minimal Tuning Required:** Aggressive randomization and ensemble averaging result in strong out-of-the-box performance even without hyperparameter optimization, in contrast with gradient boosting or AdaBoost variants.
- **Feature Attribution Analysis:** SHAP value investigations identified temperature, mean number of unfilled $p$ electrons, minimum unfilled orbitals, and minimum atomic volume as primary drivers of $\kappa_L$ variability. Low temperature and small, electrostatically simple atoms increase $\kappa_L$ (positive SHAP influence), whereas high mass contrast, greater coordination, and additional open $d$/$p$ orbitals suppress $\kappa_L$ (negative SHAP contribution).

## 7. High-Throughput Screening Workflow Implementation

The high-throughput screening protocol for $\kappa_L$ operates via a reproducible pipeline:

```python
# Load dataset
data = load_literature_dataset()

# Magpie feature calculation + add temperature
X_full = magpie_features(data.structures)
X_full = concatenate([X_full, data.temperature[:, None]], axis=1)

# Feature selection: variance + correlation filtering
sel = VarianceThreshold(threshold=0.16)
X_var = sel.fit_transform(X_full)
X = remove_highly_correlated(X_var, threshold=0.8)

# Target log-transform
y = log10(data.kappa_L)

# Repeated 10-fold CV
metrics = []
for seed in range(10):
    X_train, X_test, y_train, y_test = random_split(X, y, train_size=0.8, random_state=seed)
    model = ExtraTreesRegressor(n_estimators=100,
                                criterion='mse',
                                max_features='auto',
                                bootstrap=False,
                                random_state=None)
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)
    metrics.append(compute_R2_RMSE_MAE(y_test, y_pred))

# Averaged metrics reporting
report_average(metrics)

# Retrain full model and screen new compounds
final_model = ExtraTreesRegressor(...).fit(X, y)

for new_structure, new_T in screening_dataset:
    x_new = magpie_features([new_structure])
    x_new = sel.transform(x_new)
    x_new = correlate_filter.transform(x_new)
    x_new = append(x_new, new_T)
    logkappa_pred = final_model.predict(x_new)
    kappa_pred = 10 ** (logkappa_pred)
    record_prediction(new_structure, new_T, kappa_pred)
```

This procedure, including variance and correlation-based feature pruning, default ETR fitting, and millisecond-scale inference, enabled rapid screening of 960 half-Heusler candidates and 60,000 ICSD structures for promising thermoelectric compounds. The performance and transferability achieved with this workflow validate ETR as a reliable estimator for high-dimensional, physics-constrained regression in materials informatics [2511.13202].

Source: https://www.emergentmind.com/topics/extra-trees-regressor-etr