datacleanx is a fast, CLI-first data cleaning engine for tabular datasets. It's designed for machine learning practitioners and data engineers who want to automate cleaning workflows efficiently using a single command-line interface.

v0.2 — Rust edition The engine has been rewritten in Rust (powered by Polars) for dramatically faster CSV processing.
The original Python implementation is still available for library use; the Rust binary is the recommended way to run the CLI.

---

Why datacleanx?

• Automates repetitive cleaning steps
• Works out-of-the-box with CSV files
• Outputs timestamped cleaned files and reports
• Docker-ready for CI/CD and containerized workflows
• Includes tests and reports for reproducibility

---

Features

• Imputation: mean, median, mode
• Encoding: label, onehot
• Outlier removal using IQR
• Feature scaling: standard, minmax, robust
• Auto-saves cleaned data to outputs/
• Saves reports as structured JSON
• CLI-first design, easily scriptable
• Docker and Poetry integration

---

Installation

Option 1: Install via pip in a Virtual Environment (Recommended)

python3 -m venv venv
source venv/bin/activate
pip install datacleanx

Option 2: Install via pipx (Best for CLI tools)

sudo apt install pipx
pipx ensurepath
pipx install datacleanx

Option 3: From Source (Developer Mode)

git clone https://github.com/essiebx/datacleanx.git
cd datacleanx
poetry install
poetry run datacleanx sample_input.csv --impute median

Note for Ubuntu/Debian Users:

If you see an error like externally-managed-environment, avoid using --break-system-packages. Use a venv or pipx as shown above instead.

---

Rust Edition (Recommended for speed)

Prerequisites

• Rust toolchain — install with winget install Rustlang.Rustup (Windows) or curl https://sh.rustup.rs -sSf | sh (Linux/macOS)

Build the CLI binary

cargo build --release
# Binary: ./target/release/datacleanx  (or .exe on Windows)

Run the CLI (Rust binary)

./target/release/datacleanx sample_input.csv --impute median --encode onehot --remove-outliers --scale minmax

Use from Python (PyO3 extension)

The Rust engine can also be used directly from Python via maturin:

pip install maturin
maturin develop          # builds and installs into the current venv

Then from Python:

from datacleanx import RustCleaner

cleaner = RustCleaner(
    impute_strategy="mean",
    encode_categoricals="label",
    remove_outliers=False,
    scale_numerics="minmax",
)
cleaner.clean_csv("input.csv", "outputs/cleaned.csv")
print(cleaner.report())   # returns a dict
print(cleaner.report_json())  # returns a JSON string

Run Rust tests

cargo test

---

This will:

Save cleaned data to outputs/impute_encode_outliers_scale_.csv

Save a cleaning report to outputs/impute_encode_outliers_scale_report_.json

Custom Output Name

datacleanx sample_input.csv --output-name marketing_cleaned --impute mean --scale robust

This will generate:

outputs/marketing_cleaned_.csv

outputs/marketing_cleaned_report_.json

Project Structure

datacleanx/
├── src/                      #  Rust source (Rust edition)
│   ├── main.rs               #   CLI entry point (clap)
│   ├── lib.rs                #   PyO3 bindings
│   ├── cleaner.rs            #   Core pipeline orchestrator
│   ├── imputer.rs            #   Missing value imputation
│   ├── outliers.rs           #   IQR outlier removal
│   ├── encoder.rs            #   Label & one-hot encoding
│   ├── scaler.rs             #   Standard / minmax / robust scaling
│   └── report.rs             #   JSON report struct
├── tests/
│   ├── integration_test.rs   #   Rust integration tests
│   ├── test_cleaner.py       #   Python unit tests
│   └── test_cli.py
├── datacleanx/               # Python source (kept for compatibility)
│   ├── cleaner.py
│   ├── cli.py
│   ├── imputer.py
│   ├── report.py
│   └── __init__.py
├── outputs/                  # Auto-saved cleaned CSVs and reports
├── sample_input.csv
├── Cargo.toml                # Rust project manifest
├── Dockerfile
├── README.md
└── pyproject.toml

Example Report Output:

{ "shape_before": [4, 3], "age_imputed_with": "median", "income_imputed_with": "median", "gender_imputed_with": "median", "age_outliers_removed": 1, "income_outliers_removed": 0, "categorical_encoding": "onehot", "scaling": "minmax", "shape_after": [3, 3] }

Running Tests

poetry run pytest

Test output artifacts are saved in the tests/tests_output/ folder.

Docker Usage

Build Docker Image

docker build -t datacleanx .

Run Cleaning Task Inside Docker

docker run -v $(pwd):/app datacleanx sample_input.csv --impute median --scale standard

Project Links

PyPI: https://pypi.org/project/datacleanx/

DockerHub: https://hub.docker.com/r/essiebx/datacleanx

GitHub: https://github.com/essiebx/datacleanx