scikit tutorial

1. What is scikit-learn?

scikit-learn is a popular Python library for Machine Learning. It gives you ready-made tools for:

  • Classification (spam vs not-spam)

  • Regression (predicting house prices)

  • Clustering (grouping customers)

  • Dimensionality reduction (PCA)

  • Model selection & evaluation

  • Preprocessing (scaling, encoding, etc.)

It’s built on top of NumPy, SciPy, and Matplotlib.

2. Install & Basic Setup

Installation

pip install scikit-learn

In Google Colab, you usually already have it. If not:

!pip install scikit-learn

Basic Imports

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

3. Core Concepts in scikit-learn

If you understand these 4 things, you understand 80% of sklearn:

  1. X and y

    • X → Features (inputs), usually 2D array: shape (n_samples, n_features)

    • y → Target (output labels), 1D array: shape (n_samples,)

  2. Estimator API (fit / predict / transform)

    • Models (like LinearRegression, RandomForestClassifier) use:

      • fit(X, y) → Learn from data

      • predict(X) → Make predictions

    • Transformers (like StandardScaler) use:

      • fit(X) → Learn parameters (mean, std)

      • transform(X) → Apply transformation

      • or fit_transform(X) → both together

  3. Pipelines

    • Chain preprocessing + model together

    • Example: scaling → logistic regression

    • Important for clean code and avoiding data leakage

  4. train_test_split

    • Split data into train and test sets to evaluate properly

4. First Example: Classification on Iris Dataset

Let’s build a simple classifier using iris data (built into sklearn).

Step 1: Load the dataset

from sklearn.datasets import load_iris

iris = load_iris()
X = iris.data          # features
y = iris.target        # labels

print("X shape:", X.shape)  # (150, 4)
print("y shape:", y.shape)  # (150,)

Step 2: Train-test split

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

Step 3: Choose a model (e.g., Logistic Regression)

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

Step 4: Make predictions and evaluate

y_pred = model.predict(X_test)

from sklearn.metrics import accuracy_score, classification_report

print("Accuracy:", accuracy_score(y_test, y_pred))
print("\nClassification Report:\n", classification_report(y_test, y_pred))

That’s your first end-to-end model in sklearn ✅

5. Typical ML Workflow with scikit-learn

Let’s generalize the flow:

  1. Load Data

    • From CSV / database / built-in datasets

  2. Explore & Clean

    • Handle missing values, outliers, etc.

  3. Split into Train & Test

  4. Preprocess

    • Scaling, encoding categorical features

  5. Train Model

  6. Evaluate

    • Metrics like accuracy, precision, recall, RMSE, etc.

  7. Tune Hyperparameters

    • Using GridSearchCV / RandomizedSearchCV

  8. Save Model

    • Using joblib or pickle

6. Example Project: Predict House Prices (Regression)

Assume we have a dataset housing.csv with columns like:

  • area, bedrooms, bathrooms, price

Step 1: Load data

import pandas as pd

df = pd.read_csv("housing.csv")
df.head()

Step 2: Split features and target

X = df[["area", "bedrooms", "bathrooms"]]  # features
y = df["price"]                            # target

Step 3: Train-test split

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

Step 4: Create and train model (Linear Regression)

from sklearn.linear_model import LinearRegression

reg = LinearRegression()
reg.fit(X_train, y_train)

Step 5: Evaluate with RMSE

from sklearn.metrics import mean_squared_error
import numpy as np

y_pred = reg.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)

print("RMSE:", rmse)

Step 6: Predict for a new house

new_house = [[2000, 3, 2]]  # 2000 sq ft, 3 bedrooms, 2 bathrooms
predicted_price = reg.predict(new_house)
print("Predicted price:", predicted_price[0])

7. Preprocessing with Transformers

Real-world data needs preprocessing. sklearn has tools for:

  • Scaling numeric features → StandardScaler, MinMaxScaler

  • Encoding categorical variables → OneHotEncoder, OrdinalEncoder

  • Imputing missing values → SimpleImputer

Example: Scaling Features

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)  # fit + transform

print("Before scaling:\n", X[:5])
print("After scaling:\n", X_scaled[:5])

8. Pipelines (Very Important in sklearn)

Pipelines bundle preprocessing + model into one object.

Example: Scale + LogisticRegression

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris

iris = load_iris()
X, y = iris.data, iris.target

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("clf", LogisticRegression(max_iter=200))
])

pipe.fit(X_train, y_train)
y_pred = pipe.predict(X_test)

from sklearn.metrics import accuracy_score
print("Accuracy:", accuracy_score(y_test, y_pred))

Advantages of Pipeline:

  • Cleaner code

  • No leakage: scaling is learned only on train data

  • Works nicely with cross-validation & grid search

9. Model Evaluation & Cross-Validation

Simple train-test accuracy

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)

Cross-validation

from sklearn.model_selection import cross_val_score

scores = cross_val_score(pipe, X, y, cv=5)  # 5-fold CV
print("CV scores:", scores)
print("Mean CV score:", scores.mean())

10. Hyperparameter Tuning with GridSearchCV

Example: Tune C and penalty for Logistic Regression.

from sklearn.model_selection import GridSearchCV

param_grid = {
    "clf__C": [0.1, 1, 10],
    "clf__penalty": ["l2"],
    # for newer versions, also solver, etc.
}

grid = GridSearchCV(pipe, param_grid, cv=5, scoring="accuracy")
grid.fit(X, y)

print("Best params:", grid.best_params_)
print("Best score:", grid.best_score_)

Note: clf__C means “parameter C of the step named clf in the pipeline”.

11. Common Algorithms in scikit-learn

Here are some must-know classes:

Classification

  • LogisticRegression

  • KNeighborsClassifier

  • SVC (Support Vector Classifier)

  • RandomForestClassifier

  • GradientBoostingClassifier

Regression

  • LinearRegression

  • Ridge, Lasso

  • RandomForestRegressor

  • SVR

  • GradientBoostingRegressor

Clustering

  • KMeans

  • DBSCAN

  • AgglomerativeClustering

Dimensionality Reduction

  • PCA

  • TruncatedSVD

Example: KMeans Clustering

from sklearn.cluster import KMeans
from sklearn.datasets import load_iris

iris = load_iris()
X = iris.data

kmeans = KMeans(n_clusters=3, random_state=42)
kmeans.fit(X)

print("Cluster labels:", kmeans.labels_[:10])
print("Cluster centers:\n", kmeans.cluster_centers_)

12. Saving and Loading Models

Use joblib to persist models.

import joblib

# Save
joblib.dump(pipe, "iris_model.pkl")

# Load
loaded_model = joblib.load("iris_model.pkl")
print("Loaded model accuracy:", loaded_model.score(X_test, y_test))

13. How to Practice (Good Mini-Projects)

You can try these projects with scikit-learn:

  1. Iris classifier → We already did.

  2. Digits classifier → Use load_digits() dataset.

  3. Titanic Survival Prediction → Kaggle dataset.

  4. House Price Prediction → Regression with housing CSV.

  5. Customer Segmentation → Use KMeans clustering on shopping data.


Comments

Post a Comment

Popular posts from this blog

Python Tutorial with examples

🐍 Python Syllabus (Beginner → Advanced) 1. Beginner Level Introduction What is Python, installation, IDEs Running Python scripts Basics Syntax, keywords, indentation Variables & data types Input/Output Operators (arithmetic, relational, logical) Control Flow Conditional statements ( if , elif , else ) Loops ( for , while ) Loop control ( break , continue , pass ) Collections Lists, tuples, sets, dictionaries Indexing, slicing, methods 2. Intermediate Level Functions Defining functions, arguments, return values Default & keyword arguments Lambda functions Modules & Packages Importing modules Standard libraries overview Error Handling Exceptions ( try , except , finally ) Custom exceptions File Handling Reading/writing files Working with CSV, JSON Object-Oriented Programming Classes & objects Constructors, methods Inheritance, polymorphism, encapsulation 3. Advanced Level Iterators & Generators iter() , n...
Read more

Machine learning tutorial

  📘 Definition of Machine Learning Machine learning (ML) is a subfield of artificial intelligence (AI) that focuses on building algorithms and models that can learn patterns from data and make predictions or decisions without being explicitly programmed for each task. Instead of hard-coded rules, ML systems improve their performance as they are exposed to more data. 🔑 Key Points Pattern Recognition : ML algorithms identify hidden structures in datasets and generalize them to new, unseen data. Adaptability : Models evolve dynamically as new data arrives, staying relevant in changing environments. Applications : Used in image and speech recognition, natural language processing, recommendation systems, fraud detection, autonomous vehicles, and generative AI. Types of ML : Supervised Learning : Learns from labeled data (e.g., spam detection). Unsupervised Learning : Finds patterns in unlabeled data (e.g., customer segmentation). Semi-supervised Learning : Mix of labeled a...
Read more

RAG (Retrieval-Augmented Generation) tutorial

1. What is RAG (Retrieval-Augmented Generation)? Idea: Instead of asking an LLM to answer only from its own training, you: Retrieve relevant documents from your data (PDFs, DB, website, etc.) Augment the prompt with those docs Let the Generator (LLM) answer using that context. So pipeline is: User query ➜ Retrieve top-k docs ➜ Build prompt with docs ➜ LLM answers This solves: Outdated model knowledge Hallucinations (partially) Domain customization (company docs, reports, codebase) 2. High-Level RAG Architecture Think of 2 phases: 2.1. Offline / One-Time: Build Knowledge Store Collect data PDFs, HTML pages, markdown, CSVs, Confluence, Notion, etc. Chunking Split into small text chunks (e.g., 200–500 tokens) with overlap. Why: too big ➜ irrelevant noise; too small ➜ not enough context. Embed chunks Use an embedding model (e.g., OpenAI text-embedding, or local like bge, all-MiniLM, etc.). For each chunk: embedding = embedding_model(chunk_text) Store in Vector DB Store: (id, chunk_text...
Read more