AI and machine learning tend to borrow heavily from biology. This can be in the form of genetic algorithms that mimic evolution, or neural networks that try to capture what our brains do to learn. Our last blog post was about general machine learning and how models interact with the environment, post is going to deep dive into deep learning, or the use of Neural Networks in machine learning and AI today.
Our brain is made up of a bunch of neurons that communicate together through spikes. These spikes are composed of either electrical or chemical signals to transfer information. Each neuron modifies these signals and then passes the information onto the next neuron and the cycle continues. While the underlying information is not well understood, this behavior can be captured mathematically as it is similar to electrical circuitry used in computers and consumer electronics today.
Current state of the art machine learning models use a similar approach to this cascading neuron model to do common tasks such as recognizing a dog from a cat, understanding human speech, or predicting weather patterns. This is done by cascading a bunch of artificial neurons together creating an Artificial Neural Network (ANN). The most basic component of this is the neuron. Neurons are organized into sets called layers, and take in information from all neurons in the previous set. The ANN takes in different signals (such as all the pixels of a picture) and the cascades them through these layers. At each layer the neuron outputs a signal using a step function, sending either one or zero to be sent to all the neurons in the next layer(modern approaches even use other functions beside the step function!). After going through many layers and many functions, the network outputs a final value, which is then used to determine a value or probability distribution that is then used by a wider system to make a decision. Commonly these networks start off guessing randomly, but can slowly learn almost anything given enough time and enough examples.
This type of optimization is what’s known as a black box approach. Because each individual neuron or set of neurons doesn’t give information, there isn’t much to be gained from any intermediate step, and only a final output is used in a decision making process. Although we do not understand the fundamental meaning of any given neuron, different subsets of networks are well understood. Convolutional Neural Networks have been shown to have similar intermediate outputs to the brain, as convolutions are a good way to do visual signal processing through filtering, or looking at small bits of an image at a time. So while an individual filter may look like it is determining texture of an object, we aren’t always sure why that texture is important.
There are many different types of ANN’s, there are Deep-NN’s (DNN’s, or ANN’s with multiple layers), Convolutional-NN’s (CNN’s, that use convolutions to analyze 1-D and 2-D signals) and many many more types. At Tanuki we are focusing on different Neural Networks that are generative, or have the ability to take a random input and be able to make something human discernable out of it.
The most popular approach to do this type of work are GAN’s or Generative Adversarial Networks. This type of network is actually comprised of two different Neural Networks, an artist and a critic. An artist or a generative network will generate a final art piece. A critic or discriminator network will then give it a score, with a high score meaning it is more likely to be real. The two networks train against each other the artist trying to fool the critic and the critic trying to outsmart the artist.
This has been used to generate purely new images, or more learn patterns and be able to apply styles from one picture to another, such as taking a picture of a beach and having it look like Van Gough’s starry night.
We at Tanuki want to use these types of generative models in order to create different AI smart contracts that will interact with the rest of the blockchain. Our first project of these will be to create AI-Driven art that will be given to holders of Tanuki in the form of NFT’s. We are working hard and very excited that we will be announcing more details about this soon!