mystdscl is like a Swiss Army knife, packing the “standard deviation, scale transformation, and outlier removal” toolkit into your pocket.
When performing data cleaning, it can easily handle columns with conflicting dimensions.
In just 6 minutes, you can transform feature engineering from a “dirty job” to a “quick task”.
1. Ready to Use After Installation
With a single command using pip, mystdscl is ready to go.
After importing, let’s create some “dirty data”: height and weight with conflicting scales, plus a few outrageous values of 999.
from mystdscl import Scaler
import pandas as pd, numpy as np
df = pd.DataFrame({
'height': np.random.normal(170, 10, 100),
'weight': np.random.normal(65, 15, 100) * 100,
'age': np.random.randint(18, 45, 100)
})
df.iloc[::10] = 999 # Manual interference
print(df.head())After running, the outrageous values are clearly visible; now let’s let the scaler take action.
2. Scale Normalization, Instant Transformation
<span>fit_transform</span> first calculates the mean and standard deviation, then standardizes;
<span>clip_outliers</span> pulls back points outside of 3σ to prevent the model from being skewed.
scaler = Scaler(method='std', clip_sigma=3)
clean = scaler.fit_transform(df)
print(clean.describe())With a single line of code, height and weight are brought to the same level, and the 999 values are trimmed off.
3. Pipeline Integration, A Blessing for Perfectionists
mystdscl integrates seamlessly with sklearn pipelines.
By treating Scaler as a step in the pipeline, cross-validation no longer requires manual back-and-forth.
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
pipe = Pipeline([
('scale', Scaler(method='minmax')),
('clf', LogisticRegression(max_iter=1000))
])
# Assuming binary classification labels y
y = (df.weight > df.weight.median()).astype(int)
pipe.fit(df, y)
print("Validation Accuracy:", pipe.score(df, y))The data flow is streamlined, and the code is as clean as after a hot shower.
4. Inverse Transformation, Bringing Results Back to Reality
After standardization, model predictions are great, but the boss wants to see the results in original units.
<span>inverse_transform</span> restores the scale without getting lost.
scaled = scaler.transform(df)
recovered = scaler.inverse_transform(scaled)
print("Restoration Error:", np.abs(df - recovered).max().max())The error is visually zero, and you can report in original units, impressing your boss with your “down-to-earth” approach.
5. Batch Report Generation
mystdscl comes with a <span>report</span> method that automatically outputs mean, standard deviation, and outlier proportions as a DataFrame.
Run the batch at 10 PM, and send the report to colleagues the next day, saving time in morning meetings.
report = scaler.report(df)
print(report.head())
# Example result: column names, means, standard deviations, number of outliers, outlier proportionsWith the numbers lined up, it’s clear who should take the blame.
6. Advantages and Disadvantages
Compared to sklearn’s StandardScaler, mystdscl includes outlier trimming, eliminating the need for additional IQR coding.
It’s lightweight, with no heavy dependencies, making it runnable on Raspberry Pi.
Disadvantages: Currently only supports pandas, not friendly for pure numpy users; the method only has std, minmax, and robust options, and custom functions require source code modification.
Recommendation: For small to medium projects, dive right in; for large datasets, sample before fitting to save memory.
7. Conclusion
In just 6 minutes, feature scales no longer clash.
Try it with your data and share the number of trimmed outliers in the comments to see who has the least “hair”!
Recommended Reading:
- • spjson, an easy-to-use Python library!
- • lpsim, a clear and visible Python library!
- • kevinsr, a Python whiz!
- • sampo, a super practical Python library!