Feature Importance Command#
The feature-importance command extracts and visualizes feature importance or coefficients from trained models across all outer cross-validation iterations.
Usage#
respredai feature-importance --output <output_folder> --model <model_name> --target <target_name> [options]
Options#
Required#
--output, -o- Path to the output folder containing trained modelsMust be the same folder used in the
runcommandMust contain a
models/subdirectory with saved model filesExample:
./output/or./out_run_example/
--model, -m- Model name to extract importance fromMust match one of the models trained in the pipeline
Examples:
LR,RF,XGB,CatBoost,Linear_SVCCase-sensitive
--target, -t- Target name to extract importance forMust match one of the targets from the training pipeline
Example:
Target1,Ciprofloxacin_RCase-sensitive
Optional#
--top-n, -n- Number of top features to display (default: 20)Features are ranked by absolute importance
Range: 1 to total number of features
Example:
--top-n 30for top 30 features
--no-plot- Skip generating the barplotOnly CSV file will be created
Useful for batch processing or server environments
--no-csv- Skip generating the CSV fileOnly plot will be created
Useful if you only need visualizations
--seed, -s- Random seed for SHAP reproducibilityEnsures reproducible SHAP values across runs
Only affects models using SHAP fallback
--direction- Compute signed feature direction (Risk/Protective)Determines whether each feature is a risk factor or protective factor
Linear models: uses coefficient sign directly (no extra computation)
Tree-based models (RF, XGB, CatBoost): uses TreeExplainer for signed SHAP values
Other models: falls back to KernelExplainer for signed SHAP values
Adds a
Directioncolumn to the CSV output (Risk (+),Protective (-), orNeutral (~0))Colors plot bars by direction: firebrick (risk) / seagreen (protective) / gray (neutral)
Supported Models#
The command uses native importance when available, with SHAP as fallback:
Native Importance (Primary)#
Linear Models (Coefficients)
LR (Logistic Regression) - Uses signed coefficient values
Linear_SVC (Linear SVM) - Uses signed coefficient values
Coefficients refer to the preprocessed feature space and precede probability calibration; see Interpreting Linear Coefficients below.
Tree-Based Models (Feature Importances)
RF (Random Forest) - Uses Gini importance
XGB (XGBoost) - Uses gain-based importance
CatBoost - Uses feature importance scores
For tree-based models importance values are always positive.
SHAP Fallback#
For models without native importance/coefficients, SHAP (SHapley Additive exPlanations) values are computed as a fallback:
MLP (Multi-Layer Perceptron) - Uses KernelExplainer
RBF_SVC (RBF SVM) - Uses KernelExplainer
TabPFN - Uses KernelExplainer
SHAP values are computed on the test fold of each outer CV iteration and aggregated across folds. The mean absolute SHAP value represents feature importance.
Note: SHAP computation with KernelExplainer can be slow for large datasets.
Calibrated Fold Models#
With calibrate_probabilities = true each outer-fold model is a
CalibratedClassifierCV. It clones the best estimator found by the grid
search and refits it once per calibration split (probability_calibration_cv,
default 5), each time on a different subset of the outer training fold, and
fits a Platt sigmoid (or isotonic map) on the held-out part. The fold’s
probability is the average of those sub-models.
Native importances (coefficients, impurity, gain) and TreeExplainer SHAP values
are therefore computed for every inner estimator of a fold model and
averaged, so the reported value describes the whole fold model rather than one
arbitrary sub-model. KernelExplainer SHAP values already use the fold model’s
predict_proba and need no extra handling.
Interpreting Linear Coefficients#
For LR and Linear_SVC the reported importance is the raw model
coefficient. Three things determine what that number means:
Feature space. Coefficients refer to the features as the model sees them: continuous features are standardized (z-scored) inside each fold and categorical features are one-hot encoded. A coefficient is the change in log-odds per standard deviation of a continuous feature, or per unit of a one-hot column. The fitted scalers and encoders are stored per fold (
fold_transformersandfold_ohe_transformersin the saved.joblib), so a coefficient can be mapped back to the original units by dividing by the scaler’sscale_.Before probability calibration. The CSV reports the mean of the sub-models’ raw coefficients (and, in the per-fold CSV, intercepts). With sigmoid calibration scikit-learn maps a decision value
ftoexpit(-(a * f + b)), so the slope of the calibrated logit of a sub-model is-a * coefand its offset is-(a * intercept + b). The Platt offsetbalso absorbs the prior shift introduced byclass_weight="balanced". Coefficients or intercepts recovered by regressing the calibrated probabilities on the covariates are therefore not expected to match the raw values in the CSV, and the average of several sigmoids is only approximately a single logistic function.The grid-search refit is not the predictive model. When calibration is enabled,
GridSearchCV.best_estimator_(refit on the whole outer training fold) only acts as a template: its hyperparameters are kept, while its fitted coefficients and intercept are discarded and never used for prediction. Readingintercept_from it does not describe the probabilities the pipeline outputs.
Everything needed to inspect a fold model is in the saved .joblib:
import joblib
data = joblib.load("output/models/LR_Target1_models.joblib")
for position, model in enumerate(data["fold_models"]):
if model is None: # failed fold
continue
_, feature_names = data["fold_test_data"][position]
# checkpoints written by earlier releases may wrap the calibrator in a
# TunedThresholdClassifierCV; current ones store the calibrator itself
calibrator = model.estimator_ if hasattr(model, "estimator_") else model
for sub in calibrator.calibrated_classifiers_:
lr = sub.estimator # LogisticRegression of this calibration split
platt = sub.calibrators[0] # sigmoid calibrator with a_ and b_
raw_coef, raw_intercept = lr.coef_[0], lr.intercept_[0]
effective_coef = -platt.a_ * raw_coef
effective_intercept = -(platt.a_ * raw_intercept + platt.b_)
transformer = data["fold_transformers"][position]
scaler = transformer.named_transformers_["scaler"] # StandardScaler
# with imputation enabled the scaler is the last step of the
# "continuous" pipeline: named_transformers_["continuous"]["scaler"]
# scaler.mean_ and scaler.scale_ map standardized coefficients to raw units
The same walk over inner estimators is available as
respredai.visualization.feature_importance.iter_inner_estimators(model).
Without probability calibration the fold model is the grid-search estimator
itself and calibrated_classifiers_ does not exist.
Output Files#
The command generates files in the following structure:
output_folder/
└── feature_importance/
└── {target}/
├── {model}_feature_importance.csv # Native importance (if available)
├── {model}_feature_importance_per_fold.csv # One row per outer fold
├── {model}_feature_importance.png
├── {model}_feature_importance_shap.csv # SHAP importance (fallback)
├── {model}_feature_importance_shap_per_fold.csv
└── {model}_feature_importance_shap.png
Files have _shap suffix when SHAP is used instead of native importance.
All importance CSVs also include an Importance_Type column (the importance
scale, e.g. gain, impurity (MDI), coefficient, or mean_abs_shap -
not comparable across model families) and an N_folds_present column (the
number of CV folds in which the feature appeared). Cross-fold means skip folds
where a feature was absent rather than treating it as zero importance.
CSV File Format (Native)#
For models with native importance:
Column |
Description |
|---|---|
|
Feature name |
|
Mean importance across folds (signed for linear models) |
|
Standard deviation across folds |
|
Absolute mean importance (used for ranking) |
|
Formatted string with mean ± std |
|
Feature direction: |
CSV File Format (SHAP)#
For models using SHAP fallback:
Column |
Description |
|---|---|
|
Feature name |
|
Mean absolute SHAP value across folds |
|
Standard deviation across folds |
|
Formatted string with mean ± std |
CSV File Format (Per Fold)#
The _per_fold file exposes the values behind the summary mean and std,
one row per successful outer fold:
Column |
Description |
|---|---|
|
1-based outer fold number (with repeated CV, folds are numbered
consecutively across repeats, matching the order of |
|
Intercept of the fold model, averaged over its inner estimators (linear models only; same feature space and pre-calibration scale as the coefficients) |
one column per feature |
Importance/coefficient (or mean absolute SHAP value) of that fold, in the same feature order as the summary CSV |
Features are sorted by importance (absolute mean value).
Across all folds:
Calculate mean importance for each feature
Calculate standard deviation (uncertainty measure)
Rank features by importance
Plot Color Coding#
The barplot uses different colors to indicate importance type:
Method |
Color |
Meaning |
|---|---|---|
SHAP |
Orange |
Mean absolute SHAP value |
Native (tree-based) |
Blue |
Feature importance (always positive) |
Native (linear, positive) |
Red |
Positive coefficient |
Native (linear, negative) |
Green |
Negative coefficient |
Direction (risk) |
Red (firebrick) |
Risk factor ( |
Direction (protective) |
Green (seagreen) |
Protective factor ( |
Error bars show standard deviation across CV folds.
Examples#
Basic Usage#
Extract top 20 features for Logistic Regression on Target1:
respredai feature-importance --output ./output --model LR --target Target1
Custom Number of Features#
Show top 5 features:
respredai feature-importance -o ./output -m RF -t Target2 --top-n 5
Multiple Models#
Extract importance for multiple models (run separately):
respredai feature-importance -o ./output -m LR -t Target1
respredai feature-importance -o ./output -m RF -t Target1
respredai feature-importance -o ./output -m XGB -t Target1
CSV Only (No Plot)#
Generate only the CSV file for automated analysis:
respredai feature-importance -o ./output -m LR -t Target1 --no-plot
Plot Only (No CSV)#
Generate only the visualization:
respredai feature-importance -o ./output -m RF -t Target1 --no-csv
See Also#
Run Command - Train models with nested CV and save model files
Train Command - Train models on entire dataset for cross-dataset validation
Create Config Command - How to create the configuration file