If you followed Tutorial: Flexibility and Modularity, you know how to create modular and flexible models to handle hooks between their inputs, hidden representations, and parameters.

In this tutorial, you will use those hooks in action - you will quickly combine denoising autoencoders together to create a stacked denoising autoencoder (sDA), and then add a classification layer at the end to make a supervised MNIST classifier. (I gave in, this is the most like a multilayer perceptron you will get with these tutorials). Except you will also learn the concepts of unsupervised pre-training with supervised fine-tuning to make a better model than the MLP! Let's dig in.

Stacking layers without creating a new model

So you just want to run an experiment by combining some layers? No problem. With this example, you will see the basics of using hooks with layers to combine them in sequence.

With each additional model or layer you add, you can choose to give it an inputs_hook using the outputs of a previous model (by calling get_outputs()), or a hiddens_hook by using the hidden representation calculated by the previous model (by calling get_hiddens()). Our optimization algorithm takes care of the rest! It is able to create the fully-connected computation graph so that by training the final layer, you train all the model parameters. Here is some code to walkthrough stacking denoising autoencoders:

[ THIS TUTORIAL IS UNDER CONSTRUCTION ]