Classification (logistic regression / random forest)

All metrics come from k-fold cross-validation: each sample is predicted by a model that did not see it, so they estimate performance on new samples better than training accuracy. Folds are stratified so each keeps the overall class proportions.

The ROC curve uses the cross-validated probabilities; an AUC near 1 means strong separation and 0.5 is no better than chance. With more than two classes each class gets a one-vs-rest curve. The diagonal of the confusion matrix counts correct predictions.

With standardised features, logistic regression coefficients can be compared directly: a positive value means larger values favour the second class (in sorted order). Random forest reports the mean decrease in Gini (impurity importance), which favours features with many distinct values, so compare it with permutation importance.

Permutation importance is computed on the test part of each fold: the more accuracy falls when a feature is shuffled, the more the model relies on it. Correlated features share importance.

With few samples and many features results vary; try a few seeds or fold numbers to check stability. Model performance is not evidence of causation.

Method

Logistic regression uses the same objective as scikit-learn LogisticRegression (L2, unpenalised intercept, multinomial for more than two classes), solved by Newton’s method; random forest follows Breiman (2001) Machine Learning 45:5–32 (CART, bootstrap, random feature subsets per split); stratified k-fold splits match scikit-learn StratifiedKFold; AUC confidence intervals by DeLong et al. (1988); permutation importance after Breiman (2001).

Data size

Runs in your browser: up to about 5,000 samples and 200 features is recommended; random forests are slower, so reduce the number of trees if needed.