Our brain is made up of a bunch of neurons that communicate together through spikes. These spikes are composed of either electrical or chemical signals to transfer information. Each neuron modifies these signals and then passes the information onto the next neuron and the cycle continues. While the underlying information is not well understood, this behavior can be captured mathematically as it is similar to electrical circuitry used in computers and consumer electronics today.

Current state of the art machine learning models use a similar approach to this cascading neuron model to do common tasks such as recognizing a dog from a cat, understanding human speech, or predicting weather patterns. This is done by cascading a bunch of artificial neurons together creating an Artificial Neural Network (ANN). The most basic component of this is the neuron. Neurons are organized into sets called layers, and take in information from all neurons in the previous set. The ANN takes in different signals (such as all the pixels of a picture) and the cascades them through these layers. At each layer the neuron outputs a signal using a step function, sending either one or zero to be sent to all the neurons in the next layer(modern approaches even use other functions beside the step function!). After going through many layers and many functions, the network outputs a final value, which is then used to determine a value or probability distribution that is then used by a wider system to make a decision. Commonly these networks start off guessing randomly, but can slowly learn almost anything given enough time and enough examples.

This type of optimization is what’s known as a black box approach. Because each individual neuron or set of neurons doesn’t give information, there isn’t much to be gained from any intermediate step, and only a final output is used in a decision making process. Although we do not understand the fundamental meaning of any given neuron, different subsets of networks are well understood. Convolutional Neural Networks have been shown to have similar intermediate outputs to the brain, as convolutions are a good way to do visual signal processing through filtering, or looking at small bits of an image at a time. So while an individual filter may look like it is determining texture of an object, we aren’t always sure why that texture is important.