Model Persistence: Pickle vs Joblib & Pipeline Packaging
CSC3119 · Lab

Model Serialization Benchmark: Pickle vs Joblib & Pipeline Packaging

Save the same Random Forest with pickle and joblib, watch size and load time move, and see what happens in the API when you saved only the classifier instead of the whole pipeline.

  1. 1 · Predict0/1
  2. 2 · Experiment0/2
  3. 3 · Explain0/1
  4. 4 · Code it0/1
  5. 5 · Challenge
  6. 0/4 tasks done

Predict first

Commit to a guess before you touch anything. Being wrong here is how it sticks.

300 trees × 0.4 MB each = 120 MB of arrays. Saved with a 3× compression ratio, how big is the file?

Experiment

Move the controls and watch the outputs. Goals tick off automatically when you reach them.

pickle vs joblib

Artifact on disk
8.3 MB
18.8 MB of tree arrays ÷ 2.3× compression load ≈ 81 ms · arrays can be memory-mapped

pickle and joblib both run code on load (__reduce__). Never load a model file you didn't produce — use checksums or a signed model registry.

What did you save?

POST /predict {"age": 67, "avg_glucose_level": 228, "bmi": null, "work_type": "Private"}
→ 500 ValueError: could not convert string to float: 'Private'
   (and even numeric-only rows would be unscaled → garbage probabilities)

Explain it

Put what you saw into words. This is what the exam will ask you to do.

Why save the whole pipeline rather than the classifier, and what's the security risk of unpickling?

Code it

Write real code against real data. Python and SQL run right here in your browser and hidden tests check your work; the AI reviews it like an examiner.

Pythoncore

Save and load with a checksum

Write two functions:

  • save_model(obj, path) — save with joblib.dump(obj, path, compress=3) and return the file's SHA-256 hex digest.
  • load_model(path, expected_sha256) — compute the file's SHA-256 before loading; if it doesn't match, raise ValueError without unpickling. Otherwise return joblib.load(path).

This is the minimum safety net, because unpickling a file can run arbitrary code.

Ctrl/⌘ + Enter to run

Challenge

Fixed variations, a new problem every day, and unlimited AI challenges built from your current settings.

NumPy Array Serialization: Pickle vs Joblib

A feature matrix of 1,000,000 float64 elements occupies 8.0 MB in raw pickle. Serialized with joblib.dump(..., compress=3), it achieves a 62% size reduction. What is the compressed file size in MB? (2 decimals)