Q:

Autoencoders for feature creation

When using an auto encoder to create non-linear dimensional reduced featires, is it more common to use the output of the network (the prediction of the input features) or to use the weights from the (or 1 of the if there are multiple) hidden layers? If the hidden layer is used, do you use the hidden layer activation as features or weights from the hidden layer to the output?

A:

When you want to use Auto-Encoders (AEs) for dimensionality reduction, you usally add a bottleneck layer. This means, for example, you have 1234-dimensional data. You feed this into your AE, and - as it is an AE - you have an output of dimension 1234. However, you might have many layers in that network and one of them has significantly less dimensions. Lets say you have the topology 1234:1024:784:1024:1234. You train it like this, but you only use the weights from the 1234:1024:784 part.
When you get new input, you just feed it into this network. You can see it as a kind of preprocessing. For the later stages, this is a black box.
This is manly useful when you have a lot of unlabeled data. It is called Semi Supervised Learning (SSL).

