Week 0 : fast.ai - 1 v2 - Setting Up, GPU instances and some housekeeping

I think fast.ai has been one of the many exciting developments in the Deep Learning / Machine Learning education space over the past 1.5 years. Fast.ai really champions the top-down approach where the goal is to make Deep Learning more accessible and get you to become productive using these tools to solve a variety of problems without spending years working on a Ph.D.

I've had some false starts with Deep Learning. I literally have stacks of books, papers and PDFs on the subject and a multitude of unfinished Coursera courses and MOOCs but I feel what's usually missing is how you distill all that knowledge to help you become productive right away.

I had the opportunity to watch several videos from the previous fast.ai MOOC and I really liked the teaching style so I decided to sign up for the second iteration of the MOOC and I must say I haven't been disappointed. Another added benefit is that the course is taught by Jeremy Howard.  The next series of posts will highlight and summarize learnings from the class sessions. My goal with this is to help spread fast.ai's message of making start-of-the-art Deep Learning Research / tools more accessible.

This is really powerful stuff and what you want is for people to use these Deep Learning techniques on some of the problems they're working on which may help push the envelope in many ways. 

Hardware Setup

A lot has already been written on why GPUs and specialized hardware are great for Deep Learning so I'll cut to the chase and run you through several setup options you could use for the course

AWS : This is a great guide to set up an AWS instance.  A p2.xlarge GPU instance is probably okay for the course and it runs about 90 cents an hour. If your p2 instance limit is 0, you may have to request an instance limit increase through the AWS support center which usually takes a day or two to be approved. Jeremy also set up a fastai Community AMI in the Oregon and N.Virginia regions and a few other regions around the world (Mumbai, Sydney, Ireland) with all the needed software pre-installed.

Once you're done setting up your instance, the next steps would be to get your ssh keys and then finish the ssh config. My current setup is shown below.  Notice the config file at  ~/.ssh/config and its contents.

The setup allows you to ssh into your AWS instance by using the command ssh fast-ai. Notice that the Hostname is blank, you'll have to fill that in with your instance IPv4 Publib IP which you find on your EC2 dashboard. Do keep in mind that this changes whenever you stop/start your instance. You can either update this manually or write a script to replace the hostname each time.

Also, notice the last line in the config file. LocalForward establishes tunneling between your system and the AWS instance allowing you launch a jupyter instance on the AWS instance and then launch the actual notebook on your local system. Running the command jupyter notebook gives the following output. We can then paste the url in a browser on our system to launch the notebook.

Crestle :  is the easiest way to get a GPU enabled notebook. It's built on AWS so you get access to an AMI via ssh. It also has the fast.at datasets loaded in by default and it was built by a former fast.ai student. The setup process is quick compared to AWS and Paperspace

Paperspace: Paperspace is a cloud platform that enables you to spin up a GPU enabled fully-fledged Ubuntu desktop in a browser. They have faster / cheaper GPUs than AWS and you're charged for compute (by the hour) and storage 

They also have a fast.ai public template so it's a really quick way to set things up for the course.

Python / Anaconda : I also finally switched my python version from 2.7 to 3.6. This is trivial to do via Anaconda and you can essentially switch between version. The fast.ai pytorch library requires python 3.6. 

Personal Workstation : It could make economic sense to build you own Deep Learning workstation. An NVIDIA 1080ti GPU would cost you around $700 or so and you can supercharge with multiple GPU cards. I have box with a 1060 card but I'm still running into some issues setting it up.

Year in Review

It's been quite an interesting year to say the least.

Lots of new Machine Learning and Deep Learning tools and libraries were released into the wild reducing the barriers to entry.

I'm really hoping to turn a corner and do more writing next year. I just can't seem to be able to shake off my writer's block.

Here again is Jeff Leek's Non-comprehensive list of awesome things other people did in 2016

Year in Review

It's been quite a year.

It appears we're moving closer to the Hardware + Software + AI singularity and all the stuff that comes with that... and it's kind of scary.

Here again is Jeff Leek's Non-comprehensive list of awesome things other people did in 2015.

Some of the majors like Google, Facebook, Baidu and Microsoft open sourced some of their internal Deep Learning tools / frameworks. Most of the value coming from these tools will be interesting and useful products built with / on them.

Also, this one literally sent chills down my spine - Landing of Falcon 9 first stage. I guess we're one step closer to Mars and becoming a multi planetary species.

I didn't quite put out much content out there this year. I'm hoping to do more writing next year.

Stay tuned...

Year in Review

It's been a really interesting year.. I moved to the Bay Area. It's one thing to read about Silicon Valley or visit briefly. It's another to actually live out here and experience all it has to offer. This is the center of this data revolution everyone seems to be talking about. Obviously, if you can manage the ridiculously expensive housing out here and how much more expensive everything is out here, then you should be fine.

To wrap up the year, here is Jeff Leeks' Non-comprehensive list of awesome things other people did in 2014 . It has an rlang slant since he's a statistician.

I had more blog posts and traffic this year than each of the previous 3 year combined. Hoping this trend continues. Just looking at my traffic, it does appear there is a lot more interest in Data Science Education and immersive experiences like boot camps.

Going forward, I plan to do more tutorial style posts showing side projects or other interesting tech I encounter. 

I do want to spend more time delving into Deep Learning. Starting with the nuts and bolts and then moving to available libraries / implementations and sharing some of what I learn along the way... stay tuned 

Some more interesting links-4, Machine Intelligence, TDA, ipython notebooks

Most Topological Data Analysis tools are either stuck in academic research papers or Company intellectual property. DataRefiner might help to change that

Python for Exploratory Computing : Collection of ipython notebook showing python basics, statistics and advanced python topics

A collection of ipython notebooks on hacking security data 

This is the future of education Open Loop University, where your education is spread over several years. You'll have periods of work with schooling interlaced inbetween

Detailed infograph showing major players in the Machine Intelligence space

You should look at this if you're interested in the Quantified Self space

I've been looking for something like this. Instant temporary ipython notebooks hosted in the cloud

An extensive Deep Learning Reading list

Nice reading on Generative vs Discriminative Algorithms (Naive Bayes - Logistic Regression)

Week 7 : Zipfian Academy - Advanced Machine Learning and Deep Learning

The week started with a detour into more advanced machine learning algorithms. We covered Logistic Regression, SVM, Naive Bayes and $k-$Nearest Neighbors. We compared these algorithms on several datasets to see which situations one would perform better than the other. Before now, I usually pick my favorite algorithms and apply them on a datasets -  that's the wrong way to approach things. You need to be more strategic in choosing which algorithm you use (Use the right tool for the job).  Machine learning algorithms are broadly classified as either generative or discriminative and you have MCMC (generative) and  Neural Networks (discriminative) at the opposite ends of this spectrum respectively.

Next day, we moved to Decision Trees, Random Forest and ensembles  We used BigML to visualize decision trees and i must say they have one of the best visualizations for Decision Trees. It comes in pretty handy when you're just doing EDA. Building Decision Trees can be slow and prone to over fitting but Random Forest solves a lot of these issues as the modeling process can be parallelized. They also tend to give you models with low bias and variance.

By mid-week, we delved into Deep Learning and built a Deep Belief Network using this library with several hidden layers (Restricted Boltzmann Machines) to classify digits from the popular MNIST dtataset. We had a feed-forward setup with no back propagation. Neural Networks have been around for a while but they were put back on the map with the advent of Deep Learning about a decade ago. In academic circles, most of the hottest Deep Learning research is going on at places like NYU/Facebook (Yann LeCun), Toronto/Google (Geoff Hinton), Montreal (Yoshua Bengio) and Stanford/Google (Andrew Ng) (..of course, not listed in order)

Towards the end of the week, we took another detour into time series analysis and worked on some trend and seasonality analysis using pandas. Friday was more of a catchup day. There were no official sprints. We worked on project proposals and had a Deep Learning and git breakout. We're slowing nearing the end of the structured curriculum and everyone is slowly moving into project mode. I'll say, there were a lot of 'aha' moments for me this week.

Highlights of the week:

• We had a guest lecture from Allen on multi-layer perceptrons. He talked about some of the interesting research he worked on at Nike and how they used Neural Networks and other machine learning tools to design better footwear.
• Chief Data Scientist  at @Personagraph  gave an interesting lecture on how they're using machine learning
• We closed out the busy week with a Mentor Mixer. A lot of industry practitioners attended. The goal was to match current students with practicing Data Scientists. There were a variety of mentors who showed up, mostly Data Scientists, some Chief Data Scientists and a two-time Kaggle Competition Winner (yes, they are a rare breed but they do exist)

Real Time Speech Translation

Now this is pretty cool, real time language transalation...

Language translation has come a long way from Hidden Markov Models, Pattern Recognition and now Deep Learning Neural Networks.  These deep neural networks are also behind the new Google 'Hummingbird' Algorithm