Data Preparation, Preprocessing Leaks & Clinical Imbalance

The capstone uses a stroke-risk dataset from a Ugandan Regional Referral Hospital: age, gender, hypertension, heart disease, average glucose level, BMI, smoking status, work type, residence — and whether the patient had a stroke. Three properties make it a good teacher:

• Missing values — BMI is blank for a few percent of patients.
• Categorical strings — work_type, smoking_status need encoding before most models can use them.
• Extreme class imbalance — only about 4.9% of patients had a stroke.

Each of these has a correct, standard treatment. The danger is applying it in a way that quietly leaks information or optimises the wrong metric.

With 4.9% positives, a "model" that always answers no stroke is right for 95.1% of patients. Its accuracy is 95.1% — and its recall is 0%: every stroke patient is sent home.

For clinical triage the costs are lopsided: a false negative (missed stroke) can be fatal; a false positive (an extra clinic visit) costs time. So we judge models by:

• Recall / sensitivity = TP / (TP + FN) — what share of stroke patients we catch.
• Precision = TP / (TP + FP) — what share of flagged patients actually have the condition.
• F2 score — a harmonic mean that weights recall more than precision.
• PR-AUC — area under the precision–recall curve, more informative than ROC-AUC when positives are rare.

Always look at the confusion matrix, not one number.

Leakage is any way information from the test set (or the future) reaches training. The classic version:

X_scaled = StandardScaler().fit_transform(X)       # ✗ statistics from ALL rows
X_train, X_test, ... = train_test_split(X_scaled, y)

The scaler's mean and standard deviation now include test patients, so the test score is optimistic. The same applies to imputing with a mean computed on all rows, selecting features using all rows, or oversampling before splitting.

The fix is to split first, then fit every learned transformation on the training split only. The cleanest way is an sklearn Pipeline: pipeline.fit(X_train, y_train) fits the imputer, scaler, encoder and model on training data, and pipeline.predict(X_test) only applies them.

prep = ColumnTransformer([
    ("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]),
     ["age", "avg_glucose_level", "bmi"]),
    ("cat", OneHotEncoder(handle_unknown="ignore"), ["work_type", "smoking_status", "gender"]),
])
model = ImbPipeline([("prep", prep), ("smote", SMOTE()), ("clf", XGBClassifier())])

• ColumnTransformer applies the right preprocessing to each column group.
• handle_unknown="ignore" stops the API crashing on a category it never saw.
• SMOTE creates synthetic minority examples by interpolating between real stroke patients. Inside an imblearn pipeline it runs only during fit, so test data is never oversampled. Alternatives: class_weight="balanced" or XGBoost's scale_pos_weight (≈ negatives / positives ≈ 18).

The whole pipeline is what you save and deploy — that's what makes the preprocessing identical in production.