Neural Foundations · 2006
Deep Belief Nets / Reducing Dimensionality
Showed that a deep network can be initialized by greedily pre-training one layer at a time as restricted Boltzmann machines and then fine-tuned as a whole, addressing the difficulty of training deep networks from random weights and enabling autoencoders that learn low-dimensional data codes.
Editorial record
Plain-language summary
The method builds a deep belief network by training each layer in turn to model the activations of the layer below, giving the full network a sensible starting point before ordinary gradient fine-tuning. Applied to an autoencoder, this two-stage approach learns a compact code that reconstructs the input better than linear methods like PCA. The work provided a practical way to train deep networks before better initialization and activation functions existed, and helped renew interest in deep learning.
Source record
Provenance
- Record ID
- A-026
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.