AI Knowledge Repository

Your comprehensive, evergreen textbook for artificial intelligence

Welcome to your personal AI knowledge repository. This is not just another blog or tutorial site—it's a living, breathing textbook that grows with the field. Every concept, every algorithm, every breakthrough is documented here with the depth and rigor of academic literature, yet accessible enough for practical application.

150+ Concepts Covered

50+ Algorithms Explained

200+ Research Papers

∞ Always Updated

Neural Networks

The foundation of modern artificial intelligence

Perceptron: The Building Block

Beginner 8 min read

The perceptron is the simplest type of artificial neural network. It's a linear classifier that makes decisions by computing a weighted sum of inputs and applying a threshold function.

f(x) = 1 if w·x + b > 0, else 0

linear-classifier threshold-function weights

Backpropagation Algorithm

Intermediate 15 min read

Backpropagation is the cornerstone of training neural networks. It efficiently computes gradients by applying the chain rule of calculus, enabling networks to learn from their mistakes.

∂E/∂w = ∂E/∂a × ∂a/∂z × ∂z/∂w

gradient-descent chain-rule training

Activation Functions

Beginner 12 min read

Activation functions introduce non-linearity into neural networks, enabling them to learn complex patterns. Each function has unique properties that affect learning dynamics.

ReLU(x) = max(0, x) Sigmoid(x) = 1/(1 + e^(-x))

non-linearity relu sigmoid

Deep Learning

Neural networks with multiple layers and advanced architectures

Convolutional Neural Networks (CNNs)

Intermediate 20 min read

CNNs revolutionized computer vision by using convolutional layers to detect spatial patterns. They're particularly effective for image recognition, object detection, and image generation tasks.

y = σ(W * x + b)

computer-vision convolution feature-detection

Recurrent Neural Networks (RNNs)

Intermediate 18 min read

RNNs process sequential data by maintaining hidden states that carry information from previous time steps. They're fundamental to natural language processing and time series analysis.

h_t = tanh(W_hh * h_{t-1} + W_xh * x_t + b_h)

sequence-modeling nlp memory

Long Short-Term Memory (LSTM)

Advanced 25 min read

LSTMs solve the vanishing gradient problem in RNNs through gating mechanisms. They can learn long-term dependencies and are crucial for complex sequence modeling tasks.

f_t = σ(W_f · [h_{t-1}, x_t] + b_f) i_t = σ(W_i · [h_{t-1}, x_t] + b_i)

gating long-term-memory gradient-flow

Transformer Architecture

The revolutionary architecture that changed natural language processing

Attention Mechanism

Advanced 30 min read

The attention mechanism allows models to focus on relevant parts of the input sequence. It's the core innovation that made transformers so powerful, enabling parallel processing and capturing long-range dependencies.

Attention(Q,K,V) = softmax(QK^T/√d_k)V

attention transformer parallel-processing

Multi-Head Attention

Advanced 22 min read

Multi-head attention runs multiple attention mechanisms in parallel, allowing the model to attend to different representation subspaces simultaneously. This captures various types of relationships in the data.

MultiHead(Q,K,V) = Concat(head_1, ..., head_h)W^O

multi-head parallel-attention representation

Positional Encoding

Intermediate 15 min read

Since transformers don't have inherent sequence order like RNNs, positional encoding is added to input embeddings to provide information about token positions in the sequence.

PE(pos, 2i) = sin(pos/10000^(2i/d_model)) PE(pos, 2i+1) = cos(pos/10000^(2i/d_model))

positional-encoding sequence-order sinusoidal

Research Papers

Landmark papers that shaped the field of artificial intelligence

Attention Is All You Need (2017)

Advanced 45 min read

The seminal paper that introduced the Transformer architecture. This paper revolutionized NLP by showing that attention mechanisms alone could achieve state-of-the-art results without recurrent or convolutional layers.

transformer attention landmark-paper

ImageNet Classification with Deep CNNs (2012)

Intermediate 35 min read

AlexNet marked the beginning of the deep learning revolution. This paper demonstrated that deep convolutional neural networks could dramatically outperform traditional computer vision methods.

alexnet cnn computer-vision