Automate Your ML Pipeline.
Benchmark in One Line.
Train, evaluate, and visualize multiple machine learning classifiers and regressors automatically. Enjoy fault-tolerant execution, DataFrame preservation, and publication-ready diagnostic charts.
py -m pip install multimodel-analysis
Recommended for Windows PowerShell and Command
Prompt using the Python Launcher (py).
Platform Installation Guide
Full setup instructions with virtual environment configuration for every major platform.
Use the official Python 3 Launcher (py) to avoid PATH conflicts across multiple Python
versions on Windows.
PS C:\> py -m pip install multimodel-analysis
PS C:\> py -m venv venv PS C:\> .\venv\Scripts\activate (venv) pip install multimodel-analysis
Works natively on Intel and Apple Silicon (M1/M2/M3). For
Homebrew-managed Python, always call python3
explicitly.
$ python3 -m pip install multimodel-analysis
$ brew install python@3.11 $ python3.11 -m venv ml_env $ source ml_env/bin/activate (ml_env) $ pip install multimodel-analysis
For modern Debian/Ubuntu with PEP 668 protection, always install inside a virtual environment — the system Python is managed by the OS.
$ pip3 install multimodel-analysis
$ sudo apt install python3-venv $ python3 -m venv ~/.venv/ml $ source ~/.venv/ml/bin/activate (.venv) $ pip install multimodel-analysis
Conda automatically resolves C-level dependencies like OpenMP, BLAS, and MKL — critical for XGBoost and Scikit-learn performance on all platforms.
(base) $ conda create -n ml-env python=3.11 -y (base) $ conda activate ml-env (ml-env) $ pip install multimodel-analysis
(base) $ mamba create -n ml-env python=3.11 -y (base) $ mamba activate ml-env (ml-env) $ pip install multimodel-analysis
Core Execution Architecture
Engineered for zero data-leakage, fault tolerance, and developer productivity.
Automated Preprocessing
Applies StandardScaler while preserving pandas DataFrame
structure, column names, and index identifiers. No more lost feature names!
Smart Label Encoding
Automatically detects and encodes string, numerical, or categorical
target labels using LabelEncoder, supporting seamless multiclass analysis.
Stratified Splitting Guard
Implements intelligent stratified train-test splits with safeguards
(min_count >= 2) to ensure rare target classes never crash validation.
Fault-Tolerant Parallelism
Wraps individual model fittings in isolated exception blocks. If one model fails to converge, the benchmark continues cleanly without aborting.
Exact Statistical Metrics
Computes weighted Precision, Recall, F1, and One-vs-Rest macro ROC-AUC for classification, plus R², MAE, MSE, and RMSE for regression.
Headless Exporting
Designed for CI/CD pipelines and Docker containers. Easily export tables to CSV/Excel/HTML/JSON and render charts directly to PNG files.
Interactive Benchmarking Simulator
Experience how multimodel-analysis evaluates and ranks
algorithms in real time. Adjust parameters below and trigger a simulated execution.
| Rank | Estimator Name | Accuracy | Precision | Recall | F1 Score | ROC-AUC |
|---|
Evaluation Metric Formula Sandbox
Interact with the statistical formulas powering
multimodel-analysis. Adjust the confusion matrix cell values below to see real-time
metric recalculations.
Confusion Matrix Parameters
(TP + TN) / Total
TP / (TP + FP)
TP / (TP + FN)
2·(Prec·Rec)/(Prec+Rec)
Quickstart Code Showcase
Copy and paste these production-ready templates directly into your Jupyter Notebook or Python IDE.
import pandas as pd
from sklearn.datasets import load_breast_cancer
from multimodel_analysis import MultiModelClassifier, save_report
# 1. Load benchmark dataset & prepare feature matrix
data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = data.target
# 2. Instantiate pipeline with stratification and feature scaling
clf = MultiModelClassifier(
X=X, y=y, test_size=0.2, scaled_data=True, random_state=42, stratify=True
)
# 3. Train & evaluate all 8 built-in classifiers in parallel
results = clf.run_all_models(random_state=42)
# 4. Display tabular leaderboard & export report to CSV
df_report = clf.show_tabular_report(return_df=True)
clf.save_report("breast_cancer_metrics.csv")
# 5. One-line diagnostic summary & figure exports
clf.get_summary(save_prefix="breast_cancer", show_plot=False)
import pandas as pd
from sklearn.datasets import fetch_california_housing
from multimodel_analysis import MultiModelRegressor
# 1. Load continuous target dataset & prepare feature matrix
data = fetch_california_housing()
X = pd.DataFrame(data.data, columns=data.feature_names).iloc[:500] # Subset for fast execution
y = data.target[:500]
# 2. Instantiate regression pipeline with scaling
reg = MultiModelRegressor(
X=X, y=y, test_size=0.2, scaled_data=True, random_state=42
)
# 3. Train & evaluate all 7 built-in regression models
results = reg.run_all_models(random_state=42)
# 4. Display tabular evaluation leaderboard
reg.show_tabular_report()
# 5. Export summary reports & diagnostic scatter plots
reg.get_summary(save_prefix="california_housing", show_plot=False)
import pandas as pd
from xgboost import XGBClassifier
from lightgbm import LGBMClassifier
from multimodel_analysis import MultiModelClassifier, save_report
# 1. Prepare dataset for multi-estimator benchmarking
df = pd.read_csv("credit_risk.csv")
X, y = df.drop(columns=["Default"]), df["Default"]
# 2. Instantiate core MultiModelClassifier instance
clf = MultiModelClassifier(X=X, y=y, test_size=0.20, scaled_data=True)
# 3. Define custom third-party gradient boosting models
custom_models = {
"XGBoost Classifier": XGBClassifier(n_estimators=200, learning_rate=0.05, random_state=42),
"LightGBM Classifier": LGBMClassifier(n_estimators=200, learning_rate=0.05, random_state=42)
}
# 4. Benchmark built-in estimators ALONGSIDE XGBoost & LightGBM!
results = clf.run_all_models(custom_models=custom_models)
# 5. Export unified leaderboard comparing standard & boosting models
df_report = clf.show_tabular_report(results, return_df=True)
save_report(df_report, "custom_boosting_benchmark.csv")
clf.plot_comparison(results, save_path="custom_model_comparison.png", show_plot=False)
import pandas as pd
from multimodel_analysis import MultiModelClassifier, save_report
# 1. Execute benchmarking suite to produce evaluation results
df = pd.read_csv("medical_diagnosis.csv")
clf = MultiModelClassifier(X=df.drop(columns=["Target"]), y=df["Target"])
results = clf.run_all_models()
# 2. Capture tabular evaluation metrics DataFrame
df_report = clf.show_tabular_report(results, return_df=True)
# 3. Export to multiple publication & pipeline formats in 1 line each!
save_report(df_report, "exports/benchmark_results.csv") # CSV Spreadsheet
save_report(df_report, "exports/benchmark_results.xlsx") # Excel Workbook
save_report(df_report, "exports/benchmark_results.html") # Styled Web Table
save_report(df_report, "exports/benchmark_results.json") # REST API JSON Payload
# 4. Generate all diagnostic charts directly into PNG image files
clf.plot_confusion_matrices(results, save_path="exports/confusion_matrix.png", show_plot=False)
clf.plot_roc_curves(results, save_path="exports/roc_curves.png", show_plot=False)
clf.plot_comparison(results, save_path="exports/metrics_summary.png", show_plot=False)
Complete API Reference
Exhaustive specification of methods, arguments, and return types across all modules.
| Class / Module | Method Name (Click to view code example) | Parameters | Return Type | Description |
|---|---|---|---|---|
multimodel_analysis |
df: DataFrame = None, filepath: str = "report.csv" |
None |
Exports report DataFrame to disk (.csv, .xlsx, .xls, .html, .json) based on file extension. | |
MultiModelClassifier |
random_state: int = None, max_iter: int = 1000, **kwargs |
tuple |
Trains a Logistic Regression classifier and computes evaluation metrics. | |
MultiModelClassifier |
random_state: int = None, kernel: str = 'linear', probability: bool = True, **kwargs
|
tuple |
Trains a Support Vector Classifier (SVC) with probability estimation for ROC-AUC. | |
MultiModelClassifier |
random_state: int = None, **kwargs |
tuple |
Trains a Decision Tree Classifier and evaluates split accuracy metrics. | |
MultiModelClassifier |
n_neighbors: int = None, **kwargs |
tuple |
Trains K-Nearest Neighbors Classifier. Auto-calculates optimal n_neighbors if omitted. | |
MultiModelClassifier |
**kwargs |
tuple |
Trains a Gaussian Naive Bayes Classifier with automatic random state handling. | |
MultiModelClassifier |
n_estimators: int = 100, random_state: int = None, **kwargs |
tuple |
Trains an ensemble Random Forest Classifier across parallel CPU cores. | |
MultiModelClassifier |
n_estimators: int = 100, random_state: int = None, **kwargs |
tuple |
Trains a Gradient Boosting Classifier and evaluates stage-wise residual loss. | |
MultiModelClassifier |
n_estimators: int = 50, random_state: int = None, **kwargs |
tuple |
Trains an AdaBoost Classifier using decision stump base estimators. | |
MultiModelClassifier |
custom_models: dict = None, random_state: int = None |
list[tuple] |
Fits all 8 built-in classifiers plus optional custom estimators. Returns evaluation tuples. | |
MultiModelClassifier |
model: estimator, X_test: array = None, y_true: array = None |
tuple |
Evaluates a single fitted classifier model on test set data (Report, Matrix, Acc, Prec, Rec, F1, ROC-AUC). | |
MultiModelClassifier |
models: list = None, return_df: bool = False |
pd.DataFrame | None |
Displays formatted console leaderboard sorted by Accuracy and recommends the optimal classifier. | |
MultiModelClassifier |
models: list = None, save_path: str = None, show_plot: bool = True
|
None |
Generates colorful confusion matrix heatmaps with original decoded class labels. | |
MultiModelClassifier |
models: list = None, save_path: str = None, show_plot: bool = True
|
None |
Plots Receiver Operating Characteristic (ROC) curves with AUC metrics for all models. | |
MultiModelClassifier |
models: list = None, save_path: str = None, show_plot: bool = True
|
None |
Renders grouped bar charts comparing Accuracy, Precision, Recall, and F1 Score across models. | |
MultiModelClassifier |
models: list = None, save_prefix: str = None, show_plot: bool = True
|
None |
Runs full reporting and plotting suite in one call with automated file exporting. | |
MultiModelClassifier |
df_or_filepath: DataFrame|str = None, filepath: str = None |
None |
Saves the classifier's latest tabular performance report to disk. | |
MultiModelRegressor |
**kwargs |
tuple |
Trains an Ordinary Least Squares Linear Regression model. | |
MultiModelRegressor |
alpha: float = 0.1, random_state: int = None, **kwargs |
tuple |
Trains a Lasso (L1 Regularized) Regression model. | |
MultiModelRegressor |
alpha: float = 1.0, random_state: int = None, **kwargs |
tuple |
Trains a Ridge (L2 Regularized) Regression model. | |
MultiModelRegressor |
kernel: str = 'rbf', **kwargs |
tuple |
Trains a Support Vector Regressor (SVR). | |
MultiModelRegressor |
random_state: int = None, **kwargs |
tuple |
Trains a Decision Tree Regressor. | |
MultiModelRegressor |
n_estimators: int = 100, random_state: int = None, **kwargs |
tuple |
Trains an ensemble Random Forest Regressor across parallel cores. | |
MultiModelRegressor |
n_estimators: int = 100, random_state: int = None, **kwargs |
tuple |
Trains a Gradient Boosting Regressor. | |
MultiModelRegressor |
n_estimators: int = 50, random_state: int = None, **kwargs |
tuple |
Trains an AdaBoost Regressor. | |
MultiModelRegressor |
custom_models: dict = None, random_state: int = None |
list[tuple] |
Fits all 7 built-in regressors plus optional custom estimators. Caches results in regressor.models_. | |
MultiModelRegressor |
model: estimator, X_test: array = None, y_true: array = None |
tuple |
Evaluates a single fitted regressor model on test set data (MAE, MSE, RMSE, R2, y_pred). | |
MultiModelRegressor |
models: list = None, return_df: bool = False |
pd.DataFrame | None |
Displays formatted leaderboard sorted descending by R² Score (includes MAE, MSE, RMSE). | |
MultiModelRegressor |
models: list = None, save_path: str = None, show_plot: bool = True
|
None |
Generates a grid scatter plot of True vs. Predicted target values with identity (y = x) reference lines. | |
MultiModelRegressor |
models: list = None, save_path: str = None, show_plot: bool = True
|
None |
Renders a comparative bar chart of R² Scores across all evaluated regressor models. | |
MultiModelRegressor |
models: list = None, save_prefix: str = None, show_plot: bool = True
|
None |
Runs full evaluation and visualization pipeline for regressors (prints report, plots scatter & bar charts). | |
MultiModelRegressor |
df_or_filepath: DataFrame|str = None, filepath: str = None |
None |
Saves the regressor's latest tabular performance report to disk. | |
MultiModelRegressior |
Same as MultiModelRegressor |
Class |
Backward-compatibility class alias mapping directly to
MultiModelRegressor. |
Diagnostic Visualizations Gallery
Generate publication-quality seaborn and matplotlib charts with zero boilerplate plotting code.
Automatically decodes label-encoded integers back to original string class names for intuitive error diagnosis.
Plots combined ROC curves with exact AUC scores. Supports binary classification and One-vs-Rest multiclass macro AUC.
Visualizes regression residuals against the ideal identity line
(y = x) to spot non-linear bias and variance.
Best Practices & Execution Guide
Pro tips for getting maximum performance and reliability out of your model benchmarks.
1. Handling Class Imbalance
When rare classes represent less than 10% of your dataset, ensure
stratify=True remains enabled (the default). For severe imbalance, inspect
weighted F1 and Recall rather than Accuracy alone.
2. DataFrame Target Slice Formatting
When separating target columns from pandas DataFrames, pass a 1D Series
(df['target']) rather than a 2D DataFrame slice (df[['target']])
to avoid scikit-learn data conversion warnings.
3. Headless CI/CD & Server Exporting
In Docker containers, automated GitHub Actions, or SSH servers without
an X11 graphical display, pass show_plot=False and specify a
save_path to generate PNG charts directly to disk without GUI errors.
4. CPU Multi-Threading Allocation
For tabular datasets exceeding 50,000 rows, set n_jobs=-1
in constructor initialization to parallelize tree construction across Random Forest and
Extra Trees algorithms across all available CPU cores.