Artificial intelligence often uses an artificial neural network, to obtain an output, from input. This is a way of computing that is designed to loosely mimic the way the human brain processes information.
The human neural network
The human brain contains nerve cells known as neurons, which are connected together chemically via structures known as synapses.
Neurons can be connected to many others. Their purpose is to send and receive electrical signals.
When they receive a signal, a neuron can do one of two things.
- It can amplify and propagate the message to the next neuron
- Or it can inhibit or suppress the signal, meaning that the message may stop here, or in later steps.
These signals travel through the brain from neuron to neuron, and then across the wider nervous system. Eventually they can result in actions, such as causing our arm muscles to reach out to catch a ball, or turning words written on a page into thoughts in our head as we read a book.
It is fair to say though, that there is a lot about the way the brain operates that we still don’t understand.
Artificial neural networks
In AI, the name neural network is given to a particular way of organising data processing (creating an output, from an input). Unlike the cell-based approach in human brain, the data processing in an AI system is typically implemented using software.
This conversion of inputs into outputs is spread across a range of layers. Each layer is made of of nodes which mimic some of the behaviours of neurons in the human brain. It is important to understand that this is not a “digital brain”, it is not sentience, or deep understanding, this is just a way of processing data and identifying patterns.
Think of each neuron as a function, something that takes a value and has some dials that you can tweak, to adjust the output based on a formula. By feeding the data through a series of neurons, we get a set of new numbers – the outputs. By feeding them into the next layer, and the subsequent outputs into the next and the next, we can continue the process. The hope is this repeated processing can help achieve a particular output – e.g. detecting numbers in an image, by processing the value of the image’s individual pixels.
The first layer is known as the input layer and consists of a large number of nodes.
A node is a mathematical function and is the equivalent of a biological neuron. It receives one or more input signals (in the form of numbers).
The nodes then combine and process these inputs using a formula. Each node will have their own formula with different parameters (weights). The result of this is a single number which is the output. This may be higher than the inputs (akin to amplifying the signal) or a lower (inhibiting it). If this value passes some pre-defined threshold (known as the bias), it is passed to any node it is connected to in the next layer. This web of connections is like the synapses in the human brain.
Each node in the next layer combines these multiple inputs mathematically and in turn generates a new output for the following layer, again if they meet that node’s pre-defined threshold.
This process is repeated multiple times until the outputs reach the final layer. Known as the output layer, each node here corresponds to potential answers and for each one, there is a number which represents the probability of it being correct (the fit with the model). Typically models select the most likely output, though models sometimes use less-likely ones to add a bit of randomness – “creativity” to the output (e.g. if creating a nonsense poem).
Neural networks need to be trained to produce the desired output for a given set of inputs. This involves systematically adjusting the way the different nodes respond and how the values are combined and running the inputs through it again. This is an iterative process that may need to be repeated many, many times before usable results are obtained.
Learn more
Building a neural network
This excellent video by Grant Sanderson (aka 3Blue1Brown) explains this in much more detail. In it, he builds a neural network that recognises hand-written numbers between 0 and 9.
If you’d rather read than watch, his colleague and student intern Josh Pullen has written it out step by step in an interactive form: But what is a neural network?
Training a neural network
He then continues the series, exploring how such a neural network can be trained. In this case, Grant continues his example where the input is a large set of hand-written images and the corresponding numerical value (e.g. picture 345 is the number 9) and comparing the true value with the model’s output. He can then derive a cost function – a mathematical measure of on average, how well or badly the current network is performing when fed all the training data. The training is then repeated after adjusting the values on the dials – the “weights” and “biases” – to get a better output – informed by attempts to reduce the cost function. This improves the accuracy of the network’s ability to recognise the hand-written numbers.
Again we have Josh Pullen to thank for turning this into a written, interactive format: Gradient descent, how neural networks learn.
Both these examples use classic machine learning methods, in an attempt to keep things (relatively) simple. Recent machine learning approaches do things in an even more complicated manner, to achieve even higher levels of accuracy. If you are interested in these models, watch more of the videos in Grant Sanderson’s Deep Learning series.
For the purposes of this introduction to generative AI, it is enough to know that by using a trained neural network, given an input, the generative AI models can obtain a series of potential outputs, which each have an associated probability of being correct.
Back to How does generative AI work?