Introductions to Neural Networks

• But what is a Neural Network? Deep learning, chapter 1

  Great video series on neural nets by YouTuber 3Blue1Brown.

• Fully Connected Deep Networks

  Freely available preview chapter from Tensorflow for Deep Learning. A nice introduction to fully connected deep networks with some brief mathematical theory.

• Coding Neural Networks,

  A series of posts that cover implementing neural network in numpy including forward and back propagation, gradient checking, parameter initialization, L2 regularization, and dropout.

  Getting Started

• Official PyTorch Tutorials

  Comprehensive and readable with examples and some of the math, showing how to solve common ML problems with PyTorch. Start with the Sixty-Minute Blitz a 1-hour tutorial that explains PyTorch at a high level and how to train a neural net to classify images.

• Keras Documentation

  A competing high-level API to build NN in Python. My feeling is PyTorch’s online documentation is maybe a little better, but the book Deep Learning With Python uses Keras and it is a really good book.

Presentations

• Reinforcement Learning

  Slides from a lecture of CS885 at UC Waterloo.

Best practices

• Don’t use sigmoid: Neural Nets

  I guess everyone should just use ReLU and tweak the learning rates to avoid the dying ReLU Problem. If you want to know why read above.

• Deep Learning: A Critical Appraisal, Gary Marcus

  Against a background of considerable progress in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, this paper presents ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach artificial general intelligence.

• A Gentle Introduction to Dropout for Regularizing Deep Neural Networks

  A nice introduction to drop-out. A way to prevent NN from overfitting.

---

Table of contents

• Sequence Modeling

---