UNDERSTANDING NEURAL NETWORKS

The Understanding Neural Networks infographic describes how neural networks work.

What is a neural network?
How does it learn?
Where does it fit in the curriculum?

^{_{See Gadanidis, Li, & Tan (2024)}}

1. WHAT IS A NEURAL NETWORK?

A neural network is an attempt to digitally mimic how the human brain works.

The human brain has over 80 billion cells called neurons, which communicate with one another to carry out various functions, like walking, talking, and learning.

Brain neurons have receivers (dendrites) that take in information from other neurons, and transmitters (axon terminals) that send information to other neurons.

Similarly, a neural network has layers of digital neurons (or nodes), which send and receive information.

As the human brain learns, it makes new connections of strengthens connections among neurons.

As a neural network learns, it adjusts the weights (the importance) of the various information it receives from digital neurons (nodes).

2. HOW HUMANS LEARN

Here is one way that human learn.

Look the puzzle on the right.

Study the numbers in the YES column.
Can you see why the belong together?
A set of numbers may belong together because they have something in common.
For example:
- they may all be odd (1, 3, 5, 7 …)
- or even (2, 4, 6, 8 …)
- or divisible by 3 (3, 6, 9, 12 …)
- or prime (2, 3, 5, 7 …)
- or composite (4, 6, 8, 9 …)
- or square (1, 4, 9, 16 …)
- or …

To solve the puzzle — to learn what the numbers in the YES column have in common — you start by making a guess.

Guess a number from the Data Set that might belong in the YES column:
- If you guess 15, I would respond NO.
- If you guess 5, I would respond YES.
The YES and NO feedback gives you additional clues to help you better decide what the YES column numbers have in common, and what they don’t have in common.
You would then use this information to learn to make better guesses.

WHICH ONES BELONG TOGETHER?

Work with a partner to determine what the left columns have in common.

3. HOW A NEURAL NETWORK LEARNS

A similar method may be used to train a neural network to recognize digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9.

Unlike our ability to see an image as a whole, a neural network would subdivide the image into a grid of smaller pieces, and assign a value to each piece:

For example, 0 may indicate white and 1 may indicate black.
Or, decimals 0-1 may indicate gray scale values.
Or, 3 numbers may be assigned to each piece, to indicate red-green-blue (RGB) colour combinations.

These values are fed to the input layer (or first layer) of the neural network, which passes them to the second layer.

The second layer predicts what weights may be assigned to the inputs, applies the weights, and passes the adjusted values to the next layer.

This process continues until the output layer (or last layer). This process, from input to output, is called forward propagation.

At the output layer, the neural network is given feedback on how accurate its current predictions are. Based on this feedback, the network works backwards to adjust the weights it assigned in previous layers.

This backward feedback/adjustment process is called backpropagation.

As this forward and back propagation loop is repeated with new inputs (new images of digits 0-9), the network progressively learns to better recognize digits 0-9.

4. OUTPUT, FEEDBACK & TRAINING

Shown on the right is one possible set of predictions (or probabilities) for each of the digits 0-9, when the input is an image of the digit 9.

Notice that the highest probability, 0.98 or 98%, is assigned to digit 9, which is represented by the last output neuron (circled in red).

Notice that some other digits, like 4, 7 and 8, which look a little bit like 9, also high probabilities.

As the network is trained for a longer period of time, the network predictions improve and the probabilities change to match.

The training involves using numerous variations of digits 0-9, available from the MNIST database. A sample is shown below.

REFLECT

What did you learn?

What did you learn about neural networks?
What was most interesting to you?
What surprised you?
What else do you want to know?
Conduct an Internet search to learn more about neural networks.