The first committee member
Build a single artificial neuron and watch inputs, weights, bias, and weighted sums become a first vote.
🧠 Neural Network Fundamentals
Part 2: The First Committee Member - A Single Neuron
"One brave committee member steps up to make the first attempt at our classification task."
In Part 0 & 1 (neural_network_fundamentals.ipynb), we learned:
- Why fundamentals matter and met "The Brain's Decision Committee"
- How matrices represent data (images are just grids of numbers!)
- The dot product - the heart of pattern matching
- How matrix multiplication lets multiple "detectors" work at once
Now it's time to meet our first committee member - a single artificial neuron. This humble neuron is the building block of all neural networks, from simple classifiers to the most advanced AI systems.
What You'll Learn in Part 2
- The Biological Inspiration - How real neurons inspired artificial ones
- Anatomy of a Neuron - Inputs, Weights, Bias, and the Weighted Sum
- Building Our First Neuron - From scratch, step by step
- Connecting to Our Mission - How a neuron "sees" vertical vs horizontal lines
Prerequisites
Make sure you've completed Part 0 & 1 first! You should understand:
- What a matrix is and how images are matrices
- The dot product and what it measures
- Why we flatten 2D images into 1D vectors
Setup: Import Dependencies
Run this cell first to import all required libraries.
2.3 Inputs: What the Neuron Sees
For our line detection problem, the input is a 3×3 image. But neurons work with 1D vectors, not 2D matrices. So we need to flatten the image.
Flattening: 2D → 1D
Position mapping
Row 0: positions 0, 1, 2
Row 1: positions 3, 4, 5
Row 2: positions 6, 7, 8
Why Flatten?
- Mathematical convenience - Dot products work on vectors
- Flexibility - The same neuron code works for any input size
- Standard practice - This is how all fully-connected layers work
Let's see this in action with our line images:
2.4 Weights: What Matters Most
Weights are the learnable parameters of a neuron. They determine what the neuron "pays attention to."
How to Think About Weights
| Weight Value | Meaning |
|---|---|
| Large positive (+1, +2) | "This input strongly suggests the answer is YES" |
| Small positive (+0.1, +0.2) | "This input slightly suggests YES" |
| Zero (0) | "I don't care about this input" |
| Small negative (-0.1, -0.2) | "This input slightly suggests NO" |
| Large negative (-1, -2) | "This input strongly suggests NO" |
For Our Vertical Line Detector
If we want a neuron to detect vertical lines, it should have:
- High weights for the middle column (positions 1, 4, 7)
- Low/zero weights for other positions
- Position 1: High weight
- Position 4: High weight
- Position 7: High weight
Committee Analogy
"The weights are the committee member's priorities. A vertical line detector has high priority for evidence in the middle column because that's where vertical lines appear."
The Problem: We Don't Know the Right Weights Yet!
In practice, we don't hand-design weights. We:
- Start with random weights (the neuron doesn't know anything yet)
- Show it examples with correct answers
- Let it learn the right weights through training
For now, let's manually set "good" weights to understand how they work:
2.5 The Weighted Sum: Combining Evidence
Now we combine inputs and weights using the dot product (which you mastered in Part 1!).
The Formula
z = w · x = w₁×x₁ + w₂×x₂ + ... + w₉×x₉
This gives us a single number that represents how well the input matches what the neuron is looking for.
Interpretation
| Weighted Sum (z) | Meaning |
|---|---|
| Large positive | Input strongly matches what neuron looks for |
| Near zero | Input is neutral (doesn't match or anti-match) |
| Large negative | Input is the opposite of what neuron looks for |
Let's Calculate It
Using our vertical detector weights and both line images:
2.8 Interactive Weight Explorer
Let's build an interactive tool to see how weights affect the neuron's output. Adjust the weights and watch how the neuron responds to different inputs!
Part 2 Summary: What We've Learned
Congratulations! You've met your first committee member - a single artificial neuron!
Key Concepts Mastered
| Concept | What It Is | Committee Analogy |
|---|---|---|
| Inputs (x) | Data fed to the neuron | Evidence to review |
| Weights (w) | Importance of each input | How much the member cares about each piece |
| Bias (b) | Threshold shift | The member's default disposition |
| Weighted Sum (z) | w dot x + b | The member's overall assessment score |
| Flattening | 2D to 1D conversion | How the member "reads" images |
The Complete Neuron Formula
z = w1*x1 + w2*x2 + ... + wn*xn + b
z = w . x + b
What We Haven't Covered Yet
Our neuron outputs a raw score (z), but we need:
- Activation Function - Convert z into a meaningful output (0 to 1)
- Training - Learn the right weights automatically
- Making Predictions - Turn outputs into actual decisions
The Committee Connection
"Our first committee member can now receive evidence (inputs), weigh its importance (weights), account for their personal threshold (bias), and produce a score (z). But they haven't learned to vote yet - that's what activation functions are for!"