The Models & API

A model describes a certain type of neural network architecture. Some models are better suited to certain types of problems than others. Here’s a breakdown of the models we provide.

Deep Feedforward Net: Can use with any data you can put in a CSV type format. This is the one that started it all–the humble but amazingly effective multi-layer perceptron (MLP). These networks are trained using gradient descent and regularized with dropout (among other techniques). This is probably the network to start with.

Convolutional Network: Most used for images, “convnets” for short. Convnets have been breaking records left and right and currently represent the absolute state of the art in vision tasks. In addition to being very good at classification, they can also be used as high level visual feature extractors. These extracted features can then be used with separate algorithms (such as a Q-Learning algorithm for those of you working with cameras and robots.)

Recurrent Neural Network: Used for timeseries prediction. Timeseries are all around us–text can even be thought of as a time series. Recurrent neural networks are special in that they have a sort of “memory state” that allows them to remember something important that happened a while ago. So if you have a long sequence of multivariate frames and you want to learn how the frames evolve over time, recurrent neural networks are a good way to do it.

Autoencoder: Used for dimensionality reduction and feature extraction. These are unsupervised variants of an MLP. It works just like a normal supervised MLP, but instead of having some kind of labeled output target, it instead tries to reproduce its own input as output. So basically, you give the network some input, and you ask it to output the input. Which isn’t very exciting.

The trick is, we add noise or transform the input in some other random/stochastic way. Then, we ask the network to output the original (un-altered) input. If the network does this enough times, it will learn to de-corrupt the data, which requires knowing something about how the data is supposed to look. And thus it learns.