Malaikannan The Deep Learning way of life

How to Learn Deep Learning?

17 Jan 2017

I oscillated between different blogs and videos to become a deep learning practitioner. This blog is to document my learning and to follow an optimal path to become Deep Learning practioner faster

It is all about Math

Don’t be shy if you haven’t brushed your Math skills for a while. When you are programming for while bad habits creep in , it takes time to unlearn and learn new things. I had a tough time initially then started refreshing my Math again. I used Khan academy , i like how most of the sessions are only 10 minutes long. I followed below order

  1. Algebra – Yes you have to refresh Algebra. Remember the equation for straight line y = mx + b. That is the best equation you learned in your life. Most of machine learning is about finding the value of “m” called weights and “b” called biases.
  2. Trigonometry
  3. Differential Calculus – Machine Learning/Deep Learning is all about finding slope aka derivatives, hence do it thoroughly
  4. Partial Differential Equations
  5. Integral Calculus
  6. Probability and Statistics – this is important for anything in Machine Learning.
  7. Linear Algebra – Most of calculations are done using Matrix multiplication, dot products, transpose so learn this well.
  8. Linear Algebra Advanced – Yes it is that important. I referred to Prof Gilbert Strang lectures from MIT.

Intro into Machine Learning

I took Prof Andrew Ng’s Coursera Machine Learning course in 2012. It is the bible if you are starting with Machine Learning. Take your time and learn the basics.

Mining Massive Datasets

This was one of the best courses i took, it helped me to understand Mathematical intuition behind lot of Machine Learning algorithms.

Deep Learning

This course is Math heavy, but Prof Ali Ghodsi lectures explains it well. It is one of the hidden gems there are quite a series of lectures in youtube, watch it all. Watch it in loop, till you get hang of every concept.

Convolution Neural Networks

Convolution Neural networks is a class of Deep Learning that is predominantly used for computer vision. AndreJ Karpathy and Justin Johnson taught a great course cs231n in Stanford on CNN. It gives lot of practical tips on building Deep Learning models. I wrote an intro level CNN tutorial for Keras.

Natural Language processing

Richard Socher’s class on Natural Language processing is must if you want to work on Unstructured text. CS224d is heavy on Recurrent Neural Networks. Recently Convolution Neural Networks are being used more for NLP.

Learn Python

Python is becoming a de-facto language for scientific and numerical computation. Most Deep Learning libraries have a python front end. If you are new to python then use Byte of Python book to learn. There are lot of good youtube tutorials too.

GPU’s

If you have managed to take all these classes mentioned in the list above, then you are a serious about being a Deep Learning practitioner. Invest in a good NVIDIA GPU for trying out different models. You can use AWS for training but you will end up spending lot of money to train different models, in a long run it will make sense to buy your own hardware. Hey you can use it for gaming too, if you feel bored about Deep Learning.

Must Buy/Read Books

Ian GoodFellow, Aaron CourVille, and Yoshua Bengio wrote an awesome Deep Learning Book. I bought it, since it is a text-book theory book. Another book i often referred to is Neural Networks and DeepLearning. This book explains [Backpropagation]((http://neuralnetworksanddeeplearning.com/chap2.html), one of the most important concepts in deep learning very well.

Blogs to Read

I read Machine Learning Mastery, it has practical tips and good blogs.

Deep Learning Frameworks

There are quite a few options when it comes to Deep Learning frameworks

  1. Tensorflow
  2. Theano
  3. Keras
  4. Caffe
  5. CNTK
  6. Lasagne
  7. Other

I am personally big fan of Keras (wrapper over Tensorflow and Theano), since it abstracts lot of complexity of building a Deep Learning model, i can build a model and test whether it works or not very fast. There are tons of online tutorials on Tensorflow and Theano.

Gokul Krishnan wrote a really good blog on Anatomy of Deep Learning Frameworks

Kaggle

Kaggle is a data science competition forum, lot of researchers compete there and share their approach they used for solving that problem. Compete actively to learn and improve.

Follow Researchers on Twitter

I used twitter recommendation engine (learning machine learning using machine learning) to keep myself updated with latest research papers. Check whom i am following , on my Twitter

Related Posts