Lesson 31: Introduction to Neural Networks

What Are Neural Networks?

A neural network is a computational model inspired by the human brain. It consists of layers of interconnected nodes (neurons) that learn patterns from data. Neural networks power image recognition, speech processing, natural language understanding, and many modern AI systems.

Structure of a Neural Network

• Input Layer — receives the data.
• Hidden Layers — perform transformations and learn features.
• Output Layer — produces predictions.

How Neurons Work

Each neuron performs a simple calculation:

output = activation(weighted_sum(inputs) + bias)

The activation function introduces non‑linearity, allowing the network to learn complex patterns.

Common Activation Functions

• ReLU — most common for hidden layers.
• Sigmoid — used for binary classification.
• Softmax — used for multi‑class classification.

Training a Neural Network

Training involves adjusting weights to reduce prediction error. This is done using:

• Loss Function — measures how wrong the model is.
• Backpropagation — computes gradients.
• Optimizer — updates weights (e.g., SGD, Adam).

Example: A Simple Neural Network with Keras

from tensorflow import keras
from tensorflow.keras import layers

model = keras.Sequential([
    layers.Dense(16, activation="relu", input_shape=(4,)),
    layers.Dense(3, activation="softmax")
])

model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])

print(model.summary())

Deep Learning

Deep learning refers to neural networks with many hidden layers. These models can learn highly complex patterns and are used in:

• Computer vision
• Natural language processing
• Speech recognition
• Autonomous systems

Why Neural Networks Matter

• They can learn from large amounts of data.
• They automatically extract features without manual engineering.
• They power state‑of‑the‑art AI applications.
• They scale well with modern hardware (GPUs, TPUs).

Next Steps

Now that you understand neural networks, you're ready to explore more advanced architectures in Lesson 32: Convolutional Neural Networks (CNNs).

← Back to Lesson Index