TensorFlow tutorial

 

🟢 Level 1: Beginner (The Foundation)

Goal: Understand how TensorFlow handles data and build your first "Hello World" models.

• Introduction to Tensors:

  • What are Tensors? (Scalars, Vectors, Matrices).

  • Tensor types and shapes (tf.constant, tf.Variable).

  • Basic operations: Addition, matrix multiplication, and reshaping.

• The Keras Sequential API:

  • Building simple Dense (Linear) layers.

  • Compiling models: Selection of Optimizers (Adam, SGD) and Loss Functions (MSE, Binary Cross-Entropy).

• Data Preprocessing Basics:

  • Using tf.data.Dataset for efficient data pipelines.

  • Normalizing and scaling numerical data.

• Hands-on Project: Predicting house prices using a Simple Linear Regression model.

---

🟡 Level 2: Intermediate (Computer Vision & NLP)

Goal: Build models that can see and read.

• Computer Vision with CNNs:

  • Convolutional layers, Pooling, and Dropout.

  • Image Augmentation: Flipping, rotating, and zooming to prevent overfitting.

  • Transfer Learning: Using pre-trained models like MobileNetV3 or EfficientNet from TensorFlow Hub.

• Natural Language Processing (NLP):

  • Tokenization and Text Vectorization.

  • Word Embeddings: Representing words as numbers.

  • Recurrent Neural Networks (RNNs), LSTMs, and GRUs for sequence data.

• Model Monitoring:

  • Using TensorBoard to visualize training curves and weights.

  • Callbacks: Implementing EarlyStopping and ModelCheckpoint.

• Hands-on Project: Multi-class image classifier (e.g., Food-101) or Sentiment Analysis on movie reviews.

---

🔴 Level 3: Advanced (Customization & Production)

Goal: Master low-level control and deploy models at scale.

• The Functional API & Custom Models:

  • Building non-sequential models (Multi-input/Multi-output).

  • Custom Layers and Loss Functions using subclassing.

  • Automatic Differentiation: Mastering tf.GradientTape for custom training loops.

• Generative AI & Modern Architectures:

  • Autoencoders and Variational Autoencoders (VAEs).

  • Generative Adversarial Networks (GANs).

  • Introduction to Transformers and Attention mechanisms in TensorFlow.

• Performance & Deployment:

• Mixed Precision Training: Speeding up training on modern GPUs.

• Distributed Training: Using tf.distribute.Strategy for multi-GPU setups.

• Deployment: Exporting to TensorFlow Lite (mobile), TensorFlow.js (browser), and TensorFlow Serving (cloud).

Let’s kick off your TensorFlow journey from the real beginning: tensors.
We’ll go in exactly the order you wrote:

1. What are tensors? (scalars, vectors, matrices)

2. Tensor types & shapes (tf.constant, tf.Variable)

3. Basic operations: add, matrix multiply, reshape

4. Tiny “Hello World” model at the end 🌱

All code is in Python with TensorFlow 2.x.

---

1. What are Tensors?

A tensor is just a generalization of numbers, vectors, and matrices.

Think of it as:

• Scalar → single number → 0D tensor

• Vector → list of numbers → 1D tensor

• Matrix → table of numbers → 2D tensor

• Higher-rank tensor → more dimensions (3D, 4D, …), like images, batches, etc.

Examples (conceptually)

• Scalar (0D):
  5, 3.14, -10

• Vector (1D):
  [1, 2, 3]

• Matrix (2D):
  [[1, 2, 3], [4, 5, 6]]

• 3D tensor:
  A stack of matrices, e.g. shape (batch, height, width).

In TensorFlow, all of these are represented using the tf.Tensor object.

---

2. Tensor Types & Shapes (tf.constant, tf.Variable)

First, basic import:

import tensorflow as tf

2.1 tf.constant – read-only tensor

tf.constant creates an immutable tensor (you can’t change its values after creation).

# Scalar
scalar = tf.constant(5)
print(scalar)
print("Shape:", scalar.shape)   # ()
print("Rank:", tf.rank(scalar)) # 0

# Vector
vector = tf.constant([1, 2, 3])
print(vector)
print("Shape:", vector.shape)   # (3,)
print("Rank:", tf.rank(vector)) # 1

# Matrix
matrix = tf.constant([[1, 2, 3],
                      [4, 5, 6]])
print(matrix)
print("Shape:", matrix.shape)   # (2, 3)
print("Rank:", tf.rank(matrix)) # 2

2.2 tf.Variable – trainable / mutable tensor

tf.Variable is used for model parameters that change during training (weights, biases).

w = tf.Variable([1.0, 2.0, 3.0])  # 1D variable
print(w)
print("Shape:", w.shape)          # (3,)

# You can assign new values:
w.assign([4.0, 5.0, 6.0])
print("Updated:", w)

Key idea:

• Use tf.constant for fixed data (inputs, labels, configuration).

• Use tf.Variable for values that will be updated/learned during training.

2.3 Data types and shapes

TensorFlow supports many dtypes: tf.float32, tf.int32, tf.bool, etc.

t1 = tf.constant([1, 2, 3], dtype=tf.int32)
t2 = tf.constant([1.0, 2.0, 3.0], dtype=tf.float32)

print(t1.dtype)  # <dtype: 'int32'>
print(t2.dtype)  # <dtype: 'float32'>

• Shape: describes the dimensions, e.g. (batch_size, features)

• You can check shape with .shape or tf.shape(tensor).

x = tf.constant([[1, 2], [3, 4], [5, 6]])  # 3 rows, 2 columns
print(x.shape)        # (3, 2)
print(tf.shape(x))    # tf.Tensor([3 2], shape=(2,), dtype=int32)

---

3. Basic Tensor Operations

We’ll see:

• Addition

• Matrix multiplication

• Reshaping

3.1 Addition

You can use Python operators (+) or TensorFlow ops (tf.add).

a = tf.constant([1, 2, 3], dtype=tf.float32)
b = tf.constant([4, 5, 6], dtype=tf.float32)

c1 = a + b           # element-wise add
c2 = tf.add(a, b)    # same

print(c1)  # [5. 7. 9.]
print(c2)  # [5. 7. 9.]

Shapes must be compatible (either equal or broadcastable).

3.2 Matrix multiplication

Matrix multiplication is not element-wise. You can use:

• tf.matmul(A, B)

• or A @ B (Python operator)

A = tf.constant([[1., 2.],
                 [3., 4.]])    # shape (2, 2)

B = tf.constant([[5., 6.],
                 [7., 8.]])    # shape (2, 2)

C1 = tf.matmul(A, B)
C2 = A @ B

print(C1)
# [[19. 22.]
#  [43. 50.]]

For tf.matmul(A, B), the inner dimensions must match:
If A has shape (m, n) and B has shape (n, p), result is (m, p).

3.3 Reshaping

tf.reshape changes the shape without changing the data.

x = tf.constant([[1, 2, 3],
                 [4, 5, 6]])     # shape (2, 3)

# Reshape to (3, 2)
x_reshaped = tf.reshape(x, (3, 2))
print(x_reshaped)
# [[1 2]
#  [3 4]
#  [5 6]]

# Flatten to 1D (6,)
x_flat = tf.reshape(x, (-1,))
print(x_flat)  # [1 2 3 4 5 6]

Note:

• -1 tells TensorFlow: “infer this dimension automatically”.

---

4. A Tiny “Hello World” Model in TensorFlow

Let’s build a super simple model:

We want to learn the relationship:

[
y = 2x + 1
]

We’ll use:

• Tensors for data

• tf.Variable for weights

• Basic operations to compute predictions and loss.

4.1 Create data tensors

import tensorflow as tf

# Training data: x and y = 2x + 1
x = tf.constant([[0.], [1.], [2.], [3.]], dtype=tf.float32)  # shape (4, 1)
y_true = tf.constant([[1.], [3.], [5.], [7.]], dtype=tf.float32)

4.2 Define parameters as Variables

# Initialize weight and bias
W = tf.Variable(tf.random.normal(shape=(1, 1)))
b = tf.Variable(tf.random.normal(shape=(1,)))

4.3 Forward pass: prediction

def model(x):
    return x @ W + b   # matrix multiply + bias

4.4 Loss and one training step

We’ll use Mean Squared Error (MSE) and a simple gradient descent step.

optimizer = tf.keras.optimizers.SGD(learning_rate=0.1)

for step in range(200):
    with tf.GradientTape() as tape:
        y_pred = model(x)
        loss = tf.reduce_mean(tf.square(y_true - y_pred))

    # Compute gradients wrt W and b
    grads = tape.gradient(loss, [W, b])

    # Update variables
    optimizer.apply_gradients(zip(grads, [W, b]))

    if step % 50 == 0:
        print(f"Step {step}, Loss: {loss.numpy():.4f}")

After training:

print("Learned W:", W.numpy())
print("Learned b:", b.numpy())

You should see W close to 2 and b close to 1. 🎉
This is your “Hello World” using tensors + variables + basic ops.

---

1️⃣ What is the Keras Sequential API?

tf.keras.Sequential is the simplest way to build neural networks.

It stacks layers one after another:

Input → Layer1 → Layer2 → Output

Best for:

• Simple feed-forward networks

• Linear regression

• Basic classification

• MLPs (Multi-Layer Perceptrons)

Not ideal for:

• Complex architectures (multiple inputs/outputs)

• Residual connections

• Custom graphs (use Functional API instead)

---

2️⃣ Building Dense (Linear) Layers

What is a Dense Layer?

A Dense layer performs:

[
output = activation(XW + b)
]

Where:

• W → weights

• b → bias

• activation → optional function (relu, sigmoid, etc.)

In Keras:

tf.keras.layers.Dense(units, activation=None)

• units → number of neurons

• activation → optional activation function

---

Example 1: Simple Linear Model (Regression)

We want to learn:

[
y = 2x + 1
]

Step 1: Import

import tensorflow as tf

Step 2: Create Model

model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1, input_shape=(1,))
])

Explanation:

• units=1 → 1 output neuron

• input_shape=(1,) → input has 1 feature

This is exactly:

[
y = Wx + b
]

---

3️⃣ Compiling Models

Before training, you must compile the model.

model.compile(
    optimizer=...,
    loss=...,
    metrics=[...]
)

---

🔹 Optimizers

Optimizers update weights to minimize loss.

1️⃣ SGD (Stochastic Gradient Descent)

tf.keras.optimizers.SGD(learning_rate=0.01)

• Simple

• Slower convergence

• Good for understanding fundamentals

---

2️⃣ Adam (Adaptive Moment Estimation)

tf.keras.optimizers.Adam(learning_rate=0.001)

• Adaptive learning rate

• Faster convergence

• Default choice in most projects

👉 In practice: Use Adam unless you have a reason not to.

---

🔹 Loss Functions

For Regression

Mean Squared Error (MSE)

[
MSE = \frac{1}{n}\sum(y - \hat{y})^2
]

loss='mse'
# or
loss=tf.keras.losses.MeanSquaredError()

Used when:

• Predicting numbers

• House price prediction

• Stock prediction

• Continuous outputs

---

For Binary Classification

Binary Cross-Entropy (BCE)

[
Loss = -[y \log(\hat{y}) + (1-y)\log(1-\hat{y})]
]

loss='binary_crossentropy'

Used when:

• Spam detection

• Fraud detection

• Yes/No prediction

• Output layer uses sigmoid

---

4️⃣ Full Regression Example (MSE + Adam)

import tensorflow as tf
import numpy as np

# Data
x = np.array([0, 1, 2, 3], dtype=float)
y = np.array([1, 3, 5, 7], dtype=float)

# Model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))
])

# Compile
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
    loss='mse'
)

# Train
model.fit(x, y, epochs=200, verbose=0)

# Test
print(model.predict([10.0]))  # Should be close to 21

You just built:
✔ Linear regression
✔ Using Keras
✔ Using Adam optimizer
✔ Using MSE loss

---

5️⃣ Binary Classification Example (BCE + Sigmoid)

Let’s classify:

0 → negative class
1 → positive class

Example: OR Gate

x1	x2	y
0	0	0
0	1	1
1	0	1
1	1	1

---

Model

import numpy as np
import tensorflow as tf

# Data
X = np.array([[0,0],
              [0,1],
              [1,0],
              [1,1]], dtype=float)

y = np.array([[0],[1],[1],[1]], dtype=float)

# Model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, activation='sigmoid', input_shape=(2,))
])

# Compile
model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)

# Train
model.fit(X, y, epochs=200, verbose=0)

# Test
print(model.predict([[1,1]]))

Notice:

• Output layer uses sigmoid

• Loss is binary_crossentropy

• Metric is accuracy

---

🔥 When to Use What?

Regression

• Output: 1 neuron

• Activation: None

• Loss: MSE

• Optimizer: Adam

---

Binary Classification

• Output: 1 neuron

• Activation: Sigmoid

• Loss: Binary Cross-Entropy

• Metric: Accuracy

---

Multi-class Classification (just bonus)

• Output: N neurons

• Activation: Softmax

• Loss: Categorical Crossentropy

---

🧠 Conceptual Flow

1. Build model (architecture)
2. Compile (optimizer + loss)
3. Fit (train)
4. Predict (inference)

Nice, now we’re getting into “real project” territory 😎
Let’s cover exactly what you wrote:
tf.data.Dataset for efficient data pipelines
Normalizing & scaling numerical data
Hands-on: Predict house prices with Simple Linear Regression (Keras + tf.data)
1️⃣ tf.data.Dataset – Efficient Data Pipelines
tf.data.Dataset is TensorFlow’s way of handling data efficiently:
Streams data in batches
Supports shuffling, mapping, prefetching
Can handle large datasets that don’t fit in memory
1.1 Creating a Dataset from tensors / NumPy
import tensorflow as tf
import numpy as np

# Example NumPy data
features = np.array([[750], [800], [900], [1000], [1200]], dtype=np.float32)  # e.g. size in sq ft
labels   = np.array([[75], [80], [82], [90], [105]], dtype=np.float32)       # e.g. price in lakhs

dataset = tf.data.Dataset.from_tensor_slices((features, labels))
Now dataset yields (x, y) pairs.
1.2 Batching
batched_ds = dataset.batch(2)

for batch_x, batch_y in batched_ds:
    print("X:", batch_x.numpy(), "Y:", batch_y.numpy())
batch(2) → gives data in groups of 2
This is what model.fit() expects internally.
1.3 Shuffling
Shuffling helps break any order bias.
shuffled_batched_ds = dataset.shuffle(buffer_size=100).batch(2)
buffer_size = how many samples to hold in the shuffle buffer
Larger buffer → better randomness
1.4 Prefetching (performance boost)
final_ds = dataset.shuffle(100).batch(2).prefetch(tf.data.AUTOTUNE)
prefetch overlaps data loading with model training
AUTOTUNE lets TF decide optimal prefetch size
You will usually see:
train_ds = (
    tf.data.Dataset.from_tensor_slices((X_train, y_train))
    .shuffle(1000)
    .batch(32)
    .prefetch(tf.data.AUTOTUNE)
)
2️⃣ Normalizing & Scaling Numerical Data
Neural nets love scaled input. If features are on very different scales, training becomes unstable or slow.
2.1 Why Normalize?
Example:
Feature 1: House size → 500–3000
Feature 2: Number of rooms → 1–5
The larger range (size) can dominate gradients.
Normalization fixes this.
2.2 Common Techniques
🔹 Standardization (Z-score)
[
x' = \frac{x - \mu}{\sigma}
]
Mean = 0
Std dev = 1
Good default for many ML models
🔹 Min-Max Scaling
[
x' = \frac{x - x_{min}}{x_{max} - x_{min}}
]
Scales to [0, 1]
Useful for bounded inputs
2.3 Normalizing with NumPy + tf.data
Compute normalization stats once:
import numpy as np

# Suppose X_train is shape (num_samples, num_features)
X_train = np.array([[750, 2],
                    [800, 2],
                    [900, 3],
                    [1000, 3],
                    [1200, 4]], dtype=np.float32)

y_train = np.array([[75],
                    [80],
                    [82],
                    [90],
                    [105]], dtype=np.float32)

# Compute mean & std per feature
mean = X_train.mean(axis=0)
std  = X_train.std(axis=0)

print("Mean:", mean)
print("Std: ", std)
Define a normalization function:
def normalize_features(x, y):
    x = (x - mean) / (std + 1e-8)   # add small value to avoid division by zero
    return x, y
Use it in a tf.data pipeline:
train_ds = (
    tf.data.Dataset.from_tensor_slices((X_train, y_train))
    .map(normalize_features)       # apply normalization
    .shuffle(100)
    .batch(2)
    .prefetch(tf.data.AUTOTUNE)
)
Now every batch x is normalized.
3️⃣ Hands-on Project: House Price Prediction (Simple Linear Regression)
Goal:
Predict house price (in lakhs) from size (sqft) and number of rooms using:
tf.data.Dataset
Normalization
Keras Sequential model with one Dense layer (Linear Regression)
3.1 Step 1: Prepare Data
Here we’ll use a tiny fake dataset just to demonstrate the flow:
import tensorflow as tf
import numpy as np

# Features: [size_in_sqft, num_rooms]
X_train = np.array([
    [600, 2],
    [800, 2],
    [850, 3],
    [900, 3],
    [1000, 3],
    [1200, 4],
    [1500, 4],
    [1800, 5]
], dtype=np.float32)

# Labels: price in lakhs
y_train = np.array([
    [35],
    [50],
    [55],
    [60],
    [65],
    [80],
    [95],
    [110]
], dtype=np.float32)
3.2 Step 2: Compute Normalization Stats
mean = X_train.mean(axis=0)
std  = X_train.std(axis=0)

print("Mean:", mean)
print("Std: ", std)
3.3 Step 3: Create tf.data Pipeline
def normalize_features(x, y):
    x = (x - mean) / (std + 1e-8)
    return x, y

train_ds = (
    tf.data.Dataset.from_tensor_slices((X_train, y_train))
    .map(normalize_features)
    .shuffle(100)
    .batch(4)
    .prefetch(tf.data.AUTOTUNE)
)
3.4 Step 4: Build Linear Regression Model
This is just:
[
price = w_1 \cdot size + w_2 \cdot rooms + b
]
In Keras:
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(2,))   # 2 features → 1 output
])
3.5 Step 5: Compile (Optimizer + Loss)
Problem type → regression
Loss → MSE
Optimizer → Adam (good default)
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
    loss='mse',
    metrics=['mae']     # mean absolute error for interpretability
)
3.6 Step 6: Train the Model
history = model.fit(
    train_ds,
    epochs=200,
    verbose=0
)

print("Final loss, MAE:", model.evaluate(train_ds, verbose=0))
3.7 Step 7: Make Predictions
Remember: normalize inputs before prediction using same mean & std.
# New house: 1300 sq ft, 3 rooms
new_house = np.array([[1300, 3]], dtype=np.float32)

# Normalize using training stats
new_house_norm = (new_house - mean) / (std + 1e-8)

predicted_price = model.predict(new_house_norm)
print("Predicted price (lakhs):", predicted_price[0][0])
You now have:
✅ tf.data pipeline
✅ Normalization with map()
✅ Simple Linear Regression using Keras
✅ House price prediction end-to-end
🔁 Mental Model (Flow)
Raw NumPy data
   ↓
Compute mean & std
   ↓
tf.data.Dataset.from_tensor_slices()
   ↓
.map(normalize)
   ↓
.shuffle().batch().prefetch()
   ↓
model.fit(dataset)
   ↓
model.predict(new_data_normalized)

Awesome, this is a perfect “end-to-end mini project” chunk 🤝
I’ll keep it focused and practical:
tf.data.Dataset – efficient data pipelines
Normalizing & scaling numerical data
Hands-on: House price prediction with Simple Linear Regression (Keras)
1️⃣ Using tf.data.Dataset for Efficient Data Pipelines
tf.data.Dataset lets you feed data to the model efficiently:
Works well with large datasets
Supports batching, shuffling, mapping, prefetching
Integrates directly with model.fit(...)
1.1 Create a Dataset from NumPy arrays
import tensorflow as tf
import numpy as np

# Example data: features (X) and labels (y)
X = np.array([[600], [800], [1000], [1200], [1500]], dtype=np.float32)  # e.g. size in sq ft
y = np.array([[30], [45], [60], [75], [100]], dtype=np.float32)        # e.g. price in lakhs

dataset = tf.data.Dataset.from_tensor_slices((X, y))
Now dataset yields (x, y) pairs.
1.2 Add batching, shuffling, prefetching
batch_size = 2

train_ds = (
    dataset
    .shuffle(buffer_size=100)           # randomize order
    .batch(batch_size)                  # group into batches
    .prefetch(tf.data.AUTOTUNE)         # overlap preprocessing and training
)
This train_ds is ready to be passed straight into model.fit(train_ds, ...).
2️⃣ Normalizing and Scaling Numerical Data
Neural networks train better & faster when inputs are scaled.
2.1 Why normalize?
Different feature ranges cause unstable training
e.g. size_in_sqft ∈ [500, 3000], rooms ∈ [1, 5]
Normalization makes features more “balanced”
Common approaches:
Standardization (Z-score):
[
x' = \frac{x - \mu}{\sigma}
]
Min-Max Scaling:
[
x' = \frac{x - x_{min}}{x_{max} - x_{min}}
]
We’ll use standardization.
2.2 Compute mean & std and use in a tf.data pipeline
# Suppose X has shape (num_samples, num_features)
X = np.array([
    [600, 2],
    [800, 2],
    [900, 3],
    [1000, 3],
    [1200, 4],
    [1500, 4]
], dtype=np.float32)

y = np.array([[35], [50], [55], [65], [80], [100]], dtype=np.float32)

# Compute statistics on training data
mean = X.mean(axis=0)
std  = X.std(axis=0)

def normalize_features(x, y):
    x = (x - mean) / (std + 1e-8)   # avoid division by zero
    return x, y

train_ds = (
    tf.data.Dataset.from_tensor_slices((X, y))
    .map(normalize_features)        # apply normalization
    .shuffle(100)
    .batch(2)
    .prefetch(tf.data.AUTOTUNE)
)
Now every batch that reaches your model is already normalized. ✅
3️⃣ Hands-on Project: House Prices with Simple Linear Regression
Goal: Predict house price from a couple of features using:
tf.data.Dataset
Normalization
Keras Sequential API with a single Dense layer (linear regression)
Let’s assume 2 features:
size_in_sqft
num_rooms
3.1 Data setup
import tensorflow as tf
import numpy as np

# Features: [size_in_sqft, num_rooms]
X_train = np.array([
    [600, 2],
    [800, 2],
    [850, 2],
    [900, 3],
    [1000, 3],
    [1200, 4],
    [1500, 4],
    [1800, 5]
], dtype=np.float32)

# Labels: price in lakhs
y_train = np.array([
    [35],
    [48],
    [52],
    [58],
    [65],
    [80],
    [95],
    [115]
], dtype=np.float32)
3.2 Normalization + tf.data pipeline
# Compute normalization stats
mean = X_train.mean(axis=0)
std  = X_train.std(axis=0)

def normalize_features(x, y):
    x = (x - mean) / (std + 1e-8)
    return x, y

batch_size = 4

train_ds = (
    tf.data.Dataset.from_tensor_slices((X_train, y_train))
    .map(normalize_features)
    .shuffle(100)
    .batch(batch_size)
    .prefetch(tf.data.AUTOTUNE)
)
3.3 Build the Simple Linear Regression model
Linear regression:
[
price = w_1 \cdot size + w_2 \cdot rooms + b
]
Keras version:
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(2,))   # 2 features → 1 output
])
3.4 Compile (Optimizer + Loss)
Since this is a regression problem:
Loss: MSE (Mean Squared Error)
Optimizer: Adam (good default)
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.1),
    loss='mse',
    metrics=['mae']   # Mean Absolute Error, easy to interpret (approx error in lakhs)
)
3.5 Train the model
history = model.fit(
    train_ds,
    epochs=200,
    verbose=0
)

print("Final loss & MAE:", model.evaluate(train_ds, verbose=0))
3.6 Predict for a new house
# Example: 1300 sq ft, 3 rooms
new_house = np.array([[1300, 3]], dtype=np.float32)

# IMPORTANT: use same mean/std as training
new_house_norm = (new_house - mean) / (std + 1e-8)

predicted_price = model.predict(new_house_norm)
print("Predicted price (lakhs):", predicted_price[0][0])
Mental Picture of the Whole Flow 🧠
Raw X, y (NumPy)
    ↓
Compute mean, std on X
    ↓
tf.data.Dataset.from_tensor_slices((X, y))
    ↓
.map(normalize_features)
    ↓
.shuffle().batch().prefetch()
    ↓
Keras model (Sequential + Dense(1))
    ↓
compile(optimizer='adam', loss='mse')
    ↓
fit(train_ds)
    ↓
predict(new_normalized_input)
If you’d like, next we can:
Upgrade this to a slightly deeper network (MLP)
Add a train/validation split + early stopping

Or turn this into “how I’d explain my preprocessing & modeling pipeline in an interview” answer.

1️⃣ Convolutional Neural Networks (CNNs)
CNNs are designed specifically for image data.
Why not just use Dense layers?
Because images have:
Spatial structure (height × width × channels)
Local patterns (edges, textures, shapes)
Translation invariance
CNNs exploit these properties.
🔹 Convolutional Layer
A convolution layer:
Applies filters (kernels)
Detects local patterns (edges, corners, textures)
Produces feature maps
Mathematically:
[
FeatureMap = Input * Filter + Bias
]
In TensorFlow:
tf.keras.layers.Conv2D(
    filters=32,
    kernel_size=(3,3),
    activation='relu'
)
Example:
model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(
        filters=32,
        kernel_size=(3,3),
        activation='relu',
        input_shape=(128, 128, 3)  # 128x128 RGB image
    )
])
filters=32 → 32 feature maps
kernel_size=(3,3) → 3x3 sliding window
activation='relu' → adds non-linearity
🔹 Pooling Layer
Pooling reduces spatial dimensions.
Why?
Reduces computation
Reduces overfitting
Makes features more robust
Most common: MaxPooling
tf.keras.layers.MaxPooling2D(pool_size=(2,2))
Effect:
If input is 128×128 → becomes 64×64
Example:
model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(128,128,3)),
    tf.keras.layers.MaxPooling2D((2,2))
])
🔹 Dropout (Regularization)
Dropout randomly turns off neurons during training.
Why?
Prevents overfitting
Forces network to learn robust features
tf.keras.layers.Dropout(0.5)
0.5 → randomly disable 50% of neurons
🔹 Complete Simple CNN Example
model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(128,128,3)),
    tf.keras.layers.MaxPooling2D((2,2)),
    
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D((2,2)),
    
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(1, activation='sigmoid')  # binary classification
])
Compile:
model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)
2️⃣ Image Augmentation
Problem:
Small datasets → overfitting.
Solution:
Generate variations of images during training.
🔹 Using Keras Image Augmentation Layers
TensorFlow provides built-in augmentation layers.
Example:
data_augmentation = tf.keras.Sequential([
    tf.keras.layers.RandomFlip("horizontal"),
    tf.keras.layers.RandomRotation(0.1),
    tf.keras.layers.RandomZoom(0.1),
])
Use inside model:
model = tf.keras.Sequential([
    data_augmentation,
    
    tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(128,128,3)),
    tf.keras.layers.MaxPooling2D((2,2)),
    
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
What each does:
Augmentation
Purpose
RandomFlip
Horizontal mirroring
RandomRotation
Slight rotation
RandomZoom
Zoom in/out
RandomContrast
Change lighting
Augmentation happens:
✔ Only during training
✔ Automatically inside model pipeline
3️⃣ Transfer Learning
This is extremely important in real-world CV.
Instead of training CNN from scratch:
Use a pre-trained model
Trained on ImageNet (millions of images)
Examples:
MobileNetV3
EfficientNet
ResNet
These models already learned:
Edge detectors
Texture patterns
Object features
You reuse them.
🔹 Using MobileNetV3
Step 1: Load Pre-trained Base Model
base_model = tf.keras.applications.MobileNetV3Small(
    input_shape=(224,224,3),
    include_top=False,
    weights='imagenet'
)
include_top=False → remove classification head
weights='imagenet' → use pre-trained weights
Freeze it:
base_model.trainable = False
🔹 Add Custom Classification Head
model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
Compile:
model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)
Train normally with your dataset.
🔹 Fine-Tuning (Advanced)
After initial training:
base_model.trainable = True

model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-5),  # smaller learning rate
    loss='binary_crossentropy',
    metrics=['accuracy']
)
Now the model slightly adjusts pre-trained weights.
This usually boosts performance.
🔹 EfficientNet Example
base_model = tf.keras.applications.EfficientNetB0(
    input_shape=(224,224,3),
    include_top=False,
    weights='imagenet'
)
EfficientNet is:
Smaller
More efficient
Often better accuracy per parameter
🔥 When to Use What?
Scenario
Approach
Small dataset
Transfer Learning
Huge dataset
Train from scratch
Overfitting
Add Dropout + Augmentation
Mobile deployment
MobileNet
Best accuracy
EfficientNet / ResNet
🧠 Big Picture Pipeline
Image Dataset
    ↓
Data Augmentation
    ↓
CNN (Conv + Pool)
    ↓
Flatten / GAP
    ↓
Dense Layers
    ↓
Sigmoid / Softmax
OR (Transfer Learning)
Image
    ↓
Pretrained Model (Frozen)
    ↓
Custom Dense Head
    ↓
Fine-tune
Excellent 🚀 This is a full deep-learning stack topic.
We’ll structure this properly:
1️⃣ NLP Basics
 • Tokenization
 • Text Vectorization
2️⃣ Word Embeddings
3️⃣ RNNs, LSTMs, GRUs
4️⃣ Model Monitoring
 • TensorBoard
 • Callbacks (EarlyStopping, ModelCheckpoint)
5️⃣ Hands-on Project → Sentiment Analysis (Movie Reviews)
(Perfect practical example for everything above)
1️⃣ NLP Basics
Unlike images, text must be converted to numbers before feeding into a model.
🔹 Tokenization
Tokenization = Breaking text into smaller units (tokens).
Example:
"I love deep learning"
Word-level tokens:
["I", "love", "deep", "learning"]
Character-level tokens:
["I", " ", "l", "o", "v", "e", ...]
Using Keras TextVectorization
TensorFlow provides:
tf.keras.layers.TextVectorization
Example:
import tensorflow as tf

texts = [
    "I love deep learning",
    "Deep learning is powerful",
    "I love NLP"
]

vectorizer = tf.keras.layers.TextVectorization(
    max_tokens=1000,
    output_sequence_length=5
)

vectorizer.adapt(texts)

vectorized_text = vectorizer(texts)
print(vectorized_text)
What happens internally:
Builds vocabulary
Assigns integer index to each word
Converts sentences to integer sequences
Example output:
"I love NLP" → [3, 5, 9, 0, 0]
(0 = padding)
2️⃣ Word Embeddings
Problem:
Integer encoding doesn't capture meaning.
Example:
"king" and "queen" are related
But integers like 45 and 92 don't represent meaning
Solution → Word Embeddings.
Embeddings convert words into dense vectors:
king  → [0.2, -0.4, 0.8, ...]
queen → [0.21, -0.39, 0.81, ...]
Similar words → similar vectors.
Keras Embedding Layer
tf.keras.layers.Embedding(
    input_dim=vocab_size,
    output_dim=embedding_dim
)
Example:
embedding_layer = tf.keras.layers.Embedding(
    input_dim=1000,
    output_dim=64
)
Each word → 64-dimensional vector
These embeddings are learned during training
3️⃣ RNNs, LSTMs, GRUs
Text is sequential data.
Order matters:
"not good" ≠ "good not"
Dense layers ignore order.
RNN-based models capture sequence information.
🔹 RNN (Recurrent Neural Network)
Basic RNN:
tf.keras.layers.SimpleRNN(64)
Problem:
Vanishing gradient
Struggles with long sequences
🔹 LSTM (Long Short-Term Memory)
Most popular sequence model.
tf.keras.layers.LSTM(64)
Advantages:
Handles long-term dependencies
Solves vanishing gradient problem
Standard for NLP tasks
🔹 GRU (Gated Recurrent Unit)
Simpler version of LSTM:
tf.keras.layers.GRU(64)
Faster
Slightly less complex
Often similar performance to LSTM
Simple NLP Model Example
model = tf.keras.Sequential([
    vectorizer,
    tf.keras.layers.Embedding(1000, 64),
    tf.keras.layers.LSTM(64),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)
4️⃣ Model Monitoring
Training deep learning models blindly is dangerous.
🔹 TensorBoard
TensorBoard helps visualize:
Loss curves
Accuracy curves
Weights histograms
Graph structure
Step 1: Create log directory
import datetime

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(
    log_dir=log_dir,
    histogram_freq=1
)
Step 2: Add to model.fit()
model.fit(
    train_ds,
    epochs=10,
    validation_data=val_ds,
    callbacks=[tensorboard_callback]
)
Run in terminal:
tensorboard --logdir logs
Open browser:
http://localhost:6006
🔹 Callbacks
Callbacks allow actions during training.
1️⃣ EarlyStopping
Stops training if validation loss stops improving.
early_stop = tf.keras.callbacks.EarlyStopping(
    monitor='val_loss',
    patience=3,
    restore_best_weights=True
)
2️⃣ ModelCheckpoint
Saves best model automatically.
checkpoint = tf.keras.callbacks.ModelCheckpoint(
    "best_model.h5",
    monitor='val_loss',
    save_best_only=True
)
Use:
model.fit(
    train_ds,
    validation_data=val_ds,
    epochs=20,
    callbacks=[early_stop, checkpoint]
)
5️⃣ Hands-on Project: Sentiment Analysis (Movie Reviews)
We’ll use binary classification:
Positive (1)
Negative (0)
Step 1: Load Dataset
import tensorflow as tf

(train_data, train_labels), (test_data, test_labels) = \
    tf.keras.datasets.imdb.load_data(num_words=10000)
Step 2: Pad Sequences
train_data = tf.keras.preprocessing.sequence.pad_sequences(
    train_data,
    maxlen=200
)

test_data = tf.keras.preprocessing.sequence.pad_sequences(
    test_data,
    maxlen=200
)
Step 3: Build Model
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(10000, 64, input_length=200),
    tf.keras.layers.LSTM(64),
    tf.keras.layers.Dense(1, activation='sigmoid')
])
Step 4: Compile
model.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy']
)
Step 5: Add Monitoring
early_stop = tf.keras.callbacks.EarlyStopping(
    monitor='val_loss',
    patience=2,
    restore_best_weights=True
)

history = model.fit(
    train_data,
    train_labels,
    validation_split=0.2,
    epochs=10,
    batch_size=64,
    callbacks=[early_stop]
)
Step 6: Evaluate
model.evaluate(test_data, test_labels)
🔥 Full NLP Pipeline Overview
Raw Text
   ↓
Tokenization
   ↓
Text Vectorization
   ↓
Embedding Layer
   ↓
LSTM / GRU
   ↓
Dense Output
   ↓
Binary Crossentropy

1️⃣ Functional API & Custom Models
Sequential API is limited.
Functional API allows:
Multiple inputs
Multiple outputs
Skip connections
Shared layers
Complex graphs
🔹 Functional API Example (Multi-input Model)
Example: House price model using:
Image of house
Numerical features (size, rooms)
import tensorflow as tf

# Input 1: Image
image_input = tf.keras.Input(shape=(128,128,3), name="image_input")
x1 = tf.keras.layers.Conv2D(32, (3,3), activation='relu')(image_input)
x1 = tf.keras.layers.GlobalAveragePooling2D()(x1)

# Input 2: Tabular data
tabular_input = tf.keras.Input(shape=(5,), name="tabular_input")
x2 = tf.keras.layers.Dense(32, activation='relu')(tabular_input)

# Combine
combined = tf.keras.layers.concatenate([x1, x2])
output = tf.keras.layers.Dense(1)(combined)

model = tf.keras.Model(
    inputs=[image_input, tabular_input],
    outputs=output
)
That’s impossible with Sequential.
🔹 Custom Layers (Subclassing)
You create your own layer by subclassing tf.keras.layers.Layer.
class MyDense(tf.keras.layers.Layer):
    def __init__(self, units):
        super().__init__()
        self.units = units

    def build(self, input_shape):
        self.w = self.add_weight(
            shape=(input_shape[-1], self.units),
            initializer='random_normal',
            trainable=True
        )
        self.b = self.add_weight(
            shape=(self.units,),
            initializer='zeros',
            trainable=True
        )

    def call(self, inputs):
        return tf.matmul(inputs, self.w) + self.b
Now use:
x = MyDense(64)(input_tensor)
🔹 Custom Loss Function
def custom_mse(y_true, y_pred):
    return tf.reduce_mean(tf.square(y_true - y_pred)) + 0.01 * tf.reduce_mean(y_pred)
Pass into compile:
model.compile(optimizer='adam', loss=custom_mse)
🔹 Custom Training Loop with tf.GradientTape
Full control over training.
optimizer = tf.keras.optimizers.Adam()

for epoch in range(10):
    for x_batch, y_batch in train_ds:
        with tf.GradientTape() as tape:
            predictions = model(x_batch, training=True)
            loss = tf.reduce_mean(
                tf.keras.losses.mse(y_batch, predictions)
            )

        gradients = tape.gradient(loss, model.trainable_variables)
        optimizer.apply_gradients(zip(gradients, model.trainable_variables))

    print("Epoch:", epoch, "Loss:", loss.numpy())
This is required for:
GANs
Reinforcement learning
Custom research models
2️⃣ Generative AI & Modern Architectures
🔹 Autoencoders
Architecture:
Input → Encoder → Latent Space → Decoder → Reconstructed Output
Used for:
Dimensionality reduction
Anomaly detection
Denoising
Simple autoencoder:
encoder = tf.keras.Sequential([
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(32)
])

decoder = tf.keras.Sequential([
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(784, activation='sigmoid')
])

autoencoder = tf.keras.Sequential([encoder, decoder])
🔹 Variational Autoencoder (VAE)
Adds probabilistic latent space.
Key idea:
Instead of encoding to a fixed vector → encode mean & variance.
Uses reparameterization trick.
Loss:
Reconstruction loss
KL divergence
Used in:
Generative modeling
Image synthesis
🔹 GANs (Generative Adversarial Networks)
Two networks:
Generator vs Discriminator
Generator → Creates fake data
Discriminator → Detects fake data
Training requires custom loops:
with tf.GradientTape() as tape:
    fake_images = generator(noise)
    real_output = discriminator(real_images)
    fake_output = discriminator(fake_images)

    gen_loss = generator_loss(fake_output)
Used in:
Image generation
Deepfakes
Super-resolution
🔹 Transformers & Attention
Attention solves:
Long-range dependencies
Parallel training
Better NLP performance
Core idea:
Attention(Q, K, V) = softmax(QK^T / sqrt(d)) V
In TensorFlow:
tf.keras.layers.MultiHeadAttention(
    num_heads=8,
    key_dim=64
)
Basic Transformer block:
x = tf.keras.layers.MultiHeadAttention(
    num_heads=8,
    key_dim=64
)(query, value)

x = tf.keras.layers.LayerNormalization()(x)
x = tf.keras.layers.Dense(256, activation='relu')(x)
Used in:
BERT
GPT
Vision Transformers (ViT)
3️⃣ Performance Optimization
🔹 Mixed Precision Training
Uses float16 instead of float32.
Benefits:
Faster training
Lower memory usage
Uses Tensor Cores (NVIDIA GPUs)
Enable:
from tensorflow.keras import mixed_precision

mixed_precision.set_global_policy('mixed_float16')
Important:
Final layer may need dtype float32
Works best on RTX / A100 GPUs
🔹 Distributed Training (Multi-GPU)
Using MirroredStrategy:
strategy = tf.distribute.MirroredStrategy()

with strategy.scope():
    model = build_model()
    model.compile(optimizer='adam', loss='mse')
Automatically:
Replicates model on GPUs
Synchronizes gradients
For multi-node clusters → use MultiWorkerMirroredStrategy.
4️⃣ Deployment
Now production-level ML.
🔹 TensorFlow Lite (Mobile)
Convert model:
converter = tf.lite.TFLiteConverter.from_saved_model("saved_model")
tflite_model = converter.convert()

with open("model.tflite", "wb") as f:
    f.write(tflite_model)
Used for:
Android apps
iOS apps
Edge devices
🔹 TensorFlow.js (Browser)
Convert:
tensorflowjs_converter \
  --input_format=tf_saved_model \
  saved_model \
  tfjs_model
Deploy:
In-browser inference
Client-side ML
🔹 TensorFlow Serving (Cloud)
Export model:
model.save("saved_model")
Serve via Docker:
docker run -p 8501:8501 \
  -v "/path/saved_model:/models/model" \
  -e MODEL_NAME=model \
  tensorflow/serving
Production REST API ready.
🔥 Level 3 Mental Architecture
Custom Models
    ↓
Custom Training Loops
    ↓
Modern Architectures (GANs, VAEs, Transformers)
    ↓
Performance Optimization (Mixed Precision, Multi-GPU)
    ↓
Deployment (Mobile / Browser / Cloud)
🧠 Interview-Level Understanding
If someone asks:
“What differentiates beginner vs advanced TensorFlow engineer?”
Answer:
Beginner → Sequential + model.fit
Intermediate → Functional API + callbacks
Advanced → Custom layers + GradientTape + distributed training + deployment
If you want next, we can:
Build a mini GAN from scratch
Build a Transformer block from scratch
Design a scalable production ML architecture
Or prepare a FAANG-level ML system design answer
Tell me your direction 🚀