Nice GitHub Deep Learning Summary and Notes

I can't say enough about the list of resources here.

Distilling the Knowledge in a Neural Network

http://arxiv.org/abs/1503.02531 

A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions. Unfortunately, making predictions using a whole ensemble of models is cumbersome and may be too computationally expensive to allow deployment to a large number of users, especially if the individual models are large neural nets. Caruana and his collaborators have shown that it is possible to compress the knowledge in an ensemble into a single model which is much easier to deploy and we develop this approach further using a different compression technique. We achieve some surprising results on MNIST and we show that we can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model. We also introduce a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse. Unlike a mixture of experts, these specialist models can be trained rapidly and in parallel.

CS231n: Convolutional Neural Networks for Visual Recognition Winter Course Project Report

There are lots of interesting reads on this page.   And this is a great course to take if you are research deep learning for image processing.

Google Tech Talk with Jeff Dean at Campus Seoul

DL4J

I have been using Java for a long time but I find DL4J to be a bit cumbersome to use.  I prefer Torch/Lua or Theano for deep learning.

However because Java has been such a significant part of my life for so long, I will not give up on DL4J.

More to come once I get this working.

In the meantime, here are a few links, so I can close those tabs:-)....

word2vec in DL4J
deep autoencoders in DL4J
nd4j

Deep Learning Resources

Great Papers:
http://www.iro.umontreal.ca/~bengioy/papers/ftml.pdf
http://deeplearning.net/reading-list/

Tutorials:

TensorFlow:
https://www.tensorflow.org/versions/r0.7/tutorials/index.html

Theano:
http://deeplearning.net/tutorial/

Matlab:
http://deeplearning.stanford.edu/tutorial/

Caffe:
http://caffe.berkeleyvision.org/tutorial/

Torch:
https://github.com/soumith/cvpr2015/blob/master/Deep%20Learning%20with%20Torch.ipynb

Important Names and associated tutorials/talks:

Hinton:
https://www.cs.toronto.edu/~hinton/nntut.html

LeCun:
http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf

Socher:
http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial

Common Datasets:

MNIST - http://yann.lecun.com/exdb/mnist/

CIFAR - https://www.cs.toronto.edu/~kriz/cifar.html

LSUN - http://lsun.cs.princeton.edu/

IMAGENET - http://www.image-net.org/challenges/LSVRC/

Others:http://deeplearning.net/datasets/

Courses:

https://www.udacity.com/course/deep-learning--ud730 - Basic but uses TensorFlow, good to get a basic understanding
https://www.coursera.org/course/neuralnets - Provides great intuition, a little more challenging
https://cs231n.github.io/ - Great for understanding deep learning for images

word2vec tutorials and references

This is a nice way to get exposed to word2vec and to reduce the number of tabs in my browser:-)

References	Reference Type
Making sense of word2vec tutorial	Blog
Insights on word2vec	Blog
word2vec Explained	Paper
word2vec an Introduction	Blog
Word2Vec Tutorial Part I: The Skip-Gram Model	Blog
Word2Vec Tutorial Part II: The Continuous Bag-of-Words Model	Blog
Evaluation methods for unsupervised word embeddings	Paper
Improving Topic Models with Latent Feature Word Representations	Paper
Deep Learning, NLP, and Representations	Blog
What Should I Know Before Taking On word2vec	Reddit Post
CS224d: Deep Learning for Natural Language Processing	Course
word2vec code	Code

Convolutional Nets

Resources that give great intuition:

Relates to NLP
To understand convolution
Tutorial

Deep Learning for Big Data

Deep Learning Tutorial

Beginning deep learning with 500 lines of Julia

The link.

Jeremy Howard: The wonderful and terrifying implications of computers that can learn

AAAI Symposium

I've been spending the weekend at the AAAI Symposium. There have been quite a few interesting talks.
John Laird gave an interesting talk on General Intelligence.
Andrew Ng also gave an interesting talk, on Deep Learning.