I’ve also written a prelude, I’ll share it later. Now the fun part. Work, work, work. There is no magic formula. I am tired of living paycheck to paycheck. I’ve been homeless. I’ve watched my increasingly unhealthy wife be forced to work for scraps. I’ve had enough. I’m ready for my life to begin. I’m hungry and I know I can learn it and make it. With a smattering of ancient computer science background I see a talk about Tensorflow and something clicks. I decide I amgoing to become an Artificial Intelligence developer.
Day 0: May 2018, binge tutorials and learn about the jobs in the field of data science. I want a strong fundamental standing in mathematics. My time at university, before failing/dropping out, is over a decade in the past at this point. I still love math though, so prepare to reeducate myself. I think about going back to school and moocs. Ultimately I learn better on my own. University is full of mandatory courses and busywork unrelated to the goal, and moocs are too light and not well respected enough. I will build a portfolio while learning and that will be the resume. So I look at the requirements for a facsimile of my dream job, data science researcher at Google or Facebook. The requirements seem to be divided into math and computer science. For math it is linear algebra, calculus, and statistics. For computer science it is machine learning, deep learning algorithms, and Python or R. I build a 3 months path for myself to follow. Starting at deep learning professional and working backwards to noob. I have very time consuming responsibilities with my wife’s business and being the only one who can do most household chores due to her chronic illness. I will spend 30 hours a week learning, no TV, no sleeping in, no vacations. Thank you HBO for not airing Game of Thrones in 2018.
Days 1–42: Math and learning a new programming language. At this point I think linear algebra is just a fancy name for algebra. Oh boy. But I feel I have a strong background in calculus and statistics left over from my time at university and college. I will just need a refresher on those. It also seems like everyone is doing Python or R for deep learning and my knowledge of C++ and Java will be useless. Thank you Siraj Rival for publishing a 3 months to machine learning curriculum which I borrow heavily from. https://github.com/llSourcell/Learn_Machine_Learning_in_3_Months
Trending AI Articles:
1. Investigating Focal and Dice Loss for the Kaggle 2018 Data Science Bowl
2. My solution to achieve top 1% in a novel Data Science NLP Competition
3. Application of Deep Learning in Front-end Web Development (HTML + CSS)
4. Paper repro: “Self-Normalizing Neural Networks”
For three weeks evenings are spent watching the MIT Linear Algebra 1806 course by professor Gilbert Strang. Each week I do one homework and one quiz to ensure it sinks in. I also play around on Khan Academy. https://www.youtube.com/playlist?list=PLE7DDD91010BC51F8 https://www.khanacademy.org/math/linear-algebra
One week watching the 3 Blue 1 Brown calculus playlist and feel like I remember most of it pretty well from 15 years ago. Again with Khan Academy in the mix. https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr
For one week of statistics I just pick a few Youtube videos, read some Medium articles on Bayesian statistics, and poked around on Wikipedia.
For these five weeks I also learn Python. I go through almost every tutorial on learnpython.org and really get a lot out of py.Ceckio.org
In the last week of this first six week sprint I build my first neural network using Tensorflow. Using Google Colab it’s fashion MNIST, the hello world of deep learning. I don’t just cut and past it though. I look up each line in the documentation and the code base. Not satisfied until I know exactly how and why it works. The tutorial I start with seems to be gone now, but you can find a similar one. https://www.tensorflow.org/tutorials/
Days 43–57: Setting up my workflow. I plan to spend only a morning getting Python and Tensorflow working on my Windows 7 desktop. Over 100 hours later it’s working. Over 100 days later and it’s broken again. Today I’m waiting on the delivery of parts for a 3000 Euro custom design deep learning workstation that I’ll base on Linux Mint. To get it Python and a deep learning framework running on Windows 7 forget it and use Google Colab. I’ve got no help to offer here. For more about how I’m building this deep learning dream machine check out Tim Dettmers and Jeff Chen. http://timdettmers.com/2018/12/16/deep-learning-hardware-guide/ https://medium.com/the-mission/why-building-your-own-deep-learning-computer-is-10x-cheaper-than-aws-b1c91b55ce8c
During these weeks I read on Medium and Reddit about Deep Learning algorithms, watch Youtube tutorials on CNNs, RNNs, and LSTMs as well as vanilla NNs. It is also when I read the first 5 chapters of the Deep Learning Book. https://www.deeplearningbook.org/
Up until now I have also been proactive about joining machine learning communities online. I’ve made a Twitter and started posting on Reddit. I start a Github account and attempt to contribute to some open source. Even just on the documentation side it’s very educational seeing how the sausage is made. I make a sad looking Linkedin profile https://www.linkedin.com/in/gary-butler-644ab098/ I’ve joined slack channels and discords. I’m offering help where I can as much as I can. When I’m wrong someone corrects me and I’m learning more. I’m on the lookout for actual meetups in my area. I start waking up at 6 am and staying in my office until midnight. Responsibilities still take me away from studies and I like to spend a day each week off with my lovely wife. Now I’m studying for 40–50 hours a week.
Days 58–99: The final six weeks, or so I think. I didn’t know it at the time, but this part of the story ends with my first paid gig and meeting my future partner in my startup. Again thank you Siraj Rival (sorry haters, he’s awesome and you’re just jelly) for posting your Deep Learning in six weeks curriculum. https://github.com/llSourcell/Learn_Deep_Learning_in_6_Weeks
Code from these weeks is on Github. It’s a shameful mess, but it’s part of the journey. https://github.com/gary-butler/Learning_Deep_Learning
Week 1: Make a vanilla RNN in numpy. Look at the code for this. It’s sloppy, but it sort of works. Here’s a highlight, it it’s supposed to be a character level generator of Rumi style poetry. I know where I went wrong, but that’s for another article.
uut ledand looned the aning Love of Love.”Houngs seaverus,
int the kn roulds the morthen I tod risa nole
In’the bus ery wice cart dones herlen inot whec thisit of Hey sauts thrulntout o dild
Week 2: CNN cats vs dogs image classifier in Keras on Google Colab. Got the data and inspiration for my code on kaggle. https://www.kaggle.com/c/dogs-vs-cats/data
Week 3: LSTM I failed to make a lyric generator this week. The code is still on Github. Maybe one day I’ll revisit it.
Week 4: Redo MNIST in Pytorch, Tensorflow, and Keras to get a better grasp on each of these tools. I later turn this exercise into a presentation for a School of AI meetup in Geneva, Switzerland. https://colab.research.google.com/drive/15T-qMk_u679W87a9KIb8OLc9TkTmZvPV
Week 5: Build my first GAN in 4 hours. Generating 8 bit sprites for a 48 hour game jam game. It is terrible. Here’s a link anyway. https://gm48.net/game/1072/gans-realm
The code is inspired by the PokeGAN. I just make it more vanilla. Starting with my MNIST code in Pytorch and rework it until it’s a generative adversarial network. https://github.com/moxiegushi/pokeGAN
Week 6: Spend 7 days getting OpenAI Gym to work and about an hour making a vanilla Deep Q Network for Cartpole using NumPy.
Day 99: My first meetup. The School of AI Geneva. Thanks again Siraj Rival for driving this community. Pretty amazing icebreaker. I talk way too much, but my family is tired of hearing about AI. I have a lot to be excited about. I talk casually about working together with a few people. After the meetup, a quiet guy talks to me a bit about his AI ideas. He’s a tax lawyer and has some ideas about AI for tax law, we exchange info. Cool guy, but I donn’t know why he’s talking to me about these ideas. I’m just learning and trying to get my feet wet freelancing.
Days 86–100: October 2018 I create a profile on both Upwork and Freelancer. Upwork rejects me. I’ll be trying again soon, now it’s early 2019. I’m applying for research grants from Google Cloud and National Geographic, a kid can dream. I apply for data science and data scraping jobs. I get a job scraping Twitter data about a Deep Learning conference in Africa, DLIndaba. Amazing experience doing sentiment analysis and data visualization. I have no clue what I’m doing. It pays 150 Euros and is supposed to take 10–15 hours. I happily spend 45 hours on it and get a 5 star review.
Beyond 100, days 101–200ish: I take Siraj Rival’s free mooc finally, Move 37, it’s way more work than the advertised 10 hours a week. I’ve heard complaints that it’s not formal enough, but I’m very self motivated and getting a lot out of it. Almost done with that. Still freelancing on Freelancer, not getting a lot of gigs, but I’m winning some contests. Now a salaried (read small salary) CTO of the legal tax AI company NickyTax. Still learning like crazy. Wake up at 6 am everyday working on a NumPy implementation of the NeuroEvolution of Augmenting Topologies (NEAT) research paper for my next Medium article. Starting next month I’m not staying in the office until midnight anymore. I’m going to leave at 6:30 pm to spend time with the incredible people who have supported me on this journey.
Success in 100 days for me means getting paid to do deep learning, so I can keep learning deep learning. This is my dream come true. It’s not $100,000/year playing Foosball. I’m no renegade hot shot. Hard work. Waking up early. No money. Getting Python environments running in time for fictional deadlines. “Working” (Read unpaid studying) weekends alone when there are friends visiting from out of town. I’m not there yet, but I don’t see another way for a college drop out in rural France to make it as a data scientist. I’ve built my life around this. Maybe that’s not for you, but that’s the truth of my journey. Be sure to check out the prelude whenever I post it.
Acknowledgements: Thanks Google for making Tensorflow and reminding me of my dreams. Thanks to my wife and family for putting up with my rants about installing Python on Windows. Thank you Siraj Rival for being awesome, and sharing your passion.
Finally thank you Phil for really getting into the nitty gritty with Deep Learning tutorials.