🔥 Deep Learning

Deep Learning uses artificial neural networks with many layers to learn representations from raw data. We use PyTorch — the preferred framework in research and industry.

---

📋 What You'll Learn

Notebook	Topics Covered	Difficulty
Neural Networks (Coming Soon)	Perceptron, activation functions, forward/backward pass	🟠 Advanced
PyTorch Basics (Coming Soon)	Tensors, autograd, nn.Module, datasets, dataloaders	🟠 Advanced
Training Loop (Coming Soon)	Complete training loop, loss functions, optimizers, GPU	🟠 Advanced

---

🔑 Key Concepts

• Tensors: PyTorch's core data structure (like NumPy + GPU)
• Autograd: Automatic differentiation for backpropagation
• nn.Module: Base class for all neural network layers
• Loss Functions: MSE, CrossEntropy, BCE, Huber
• Optimizers: SGD, Adam, AdamW, RMSProp
• Learning Rate Scheduling: StepLR, CosineAnnealing
• GPU Training: .to(device) — CUDA acceleration
• DataLoader: Batching, shuffling, parallel loading

---

🚀 Quick Example

import torch
import torch.nn as nn

# Define a simple neural network
class MLP(nn.Module):
    def __init__(self, in_features, hidden, out_features):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(in_features, hidden),
            nn.ReLU(),
            nn.Dropout(0.3),
            nn.Linear(hidden, hidden),
            nn.ReLU(),
            nn.Linear(hidden, out_features)
        )

    def forward(self, x):
        return self.net(x)

device = 'cuda' if torch.cuda.is_available() else 'cpu'
model  = MLP(784, 256, 10).to(device)
optim  = torch.optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()

print(f"Device: {device}")
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")

---

📐 Key Formulas

Sigmoid: [\sigma(x) = \frac{1}{1+e^{-x}}]

ReLU: [\text{ReLU}(x) = \max(0, x)]

Cross-Entropy Loss: [\mathcal{L} = -\sum_{c=1}^{C} y_c \log(\hat{y}_c)]

Gradient Descent Update: [w \leftarrow w - \eta \frac{\partial \mathcal{L}}{\partial w}]

---