It All Started From My Personal Struggle
To tell you the truth, the fear of not being good enough at math has been hunting me from day one when I started my Machine Learning journey. It first struck me when I tried to apply the chain-rule while doing the back-propagation of neural networks. It took me a good several days before the assignment deadline until I finally figured out how to calculate the gradients. Man, that felt hard! I can almost still taste the bitterness and hardship in my mouth. Little did I know, this probably was the easy part. The real nightmare started when I first try to read a machine learning paper… 🤮
If you can relate to my experience, you’ve been there. What I am trying to tell you today is: It doesn’t need to be that way, and you can have a totally different angle to approach your machine learning journey, without worry too much about math. Let me explain.
Math Needed for Applied Deep Learning is very Limited
In Machine Learning, too many people have the impostor syndrome of not being able to get to the bottom of the math behind the model. It’s understandable, and we felt safe and confident about ourselves when we know we are in control. But not being able to achieve a full understanding of the math is a harsh reminder that we are not sufficient.
This might not be necessary 90% of the time, especially for Deep Learning. Remember the back-propagation chain-rule hell I mentioned above? It hunted me too long until one day, when I was learning PyTorch, I found something called ‘auto-grad.’ It blew my mind. It basically means whatever loss function you come up with, the framework will track it and automatically calculate the gradient for you. What?! Then why I need to do all those back-prop chain rule calculations by hand?
And even if you have to do this by hand, the essential math you need for Deep Learning is matrix multiplication and derivative calculus. Once you have these two under your belt(which is absolutely not hard), you should feel confident when facing most of the challenges. The rest are more of an experiment of what works and what not—things like AutoML and grid search.
🧗Two Languages, Two Sides of the Mountain, One Same Peak🧗♀️
Another thing about Machine Learning is that it requires you to speak two languages: programming languages like Python, R, and math. And sometimes, to tackle one challenge, you need the combined strength, and master one will definitely help the other. So if you’re not the ‘math head’, all hope is not lost, because you can still try doing it with the code, which is not vague at all, and very ‘tinkerable’. Unlike math, you can always experiment some more and get feedback.
In the coding world, when teaching recursion algorithms, the biggest hurdle is to ‘trust’ the simple recursion codes that can actually solve the seemingly difficult problem. It’s called the ‘recursion leap of faith’. First-time recursion learning often had a hard time getting through the uncomfortable feeling that something is still missing their code to solve the problem. It’s somewhat unnatural, but you just have to trust the code, trust the algorithm and let it unfold the problem in an elegant way. Only after practice will you comfortably and confidently solve problems using recursion.
I felt deep learning also has similar things. The ‘auto gradient’ feature most frameworks and some languages possess is one of them. You don’t need to know how to calculate the gradient of your loss function, just trust the framework/language can do it. Maybe it should be called ‘the auto-grad leap of faith’. You won’t be bothered if you can’t use assembly to write your Python code, right? So why making it a big deal when you can’t do the gradient by hand?
You Should not Let Yourself Stuck Anyway🚦
Another point worth noticing is related to learning and education in general. I wanted to call it ‘Don’t get stuck.’ When encountering an issue or challenge, maybe a math equation, maybe a bug, don’t let yourself stuck in it and put all the time into solving it. Instead, step back, breathe, take a break, walk away, find some way around. Or ignore it if you can for now and come back later. You’ll find out that with more context, a better-rested mind, and a clearer view, the issue will look different, and the solution will often emerge by itself. So don’t get stuck in math or anything else, really. Step back, see the big picture. 🥾
What Should You Do Instead?
Having said all this, I want to clarify that I’m not against learning math, nor do I suggest math is not important for your machine learning journey. Math is definitely at the core of most new impactful machine learning research progress. What I was saying is don’t let the obsession with math or fear of math get in the way of your learning. Just learn along, experiment with code if you are stuck, and get some intuition first. You’ll gradually wrap your mind around the math part of machine learning over time. The learning of math is necessary, but the extra worry is NOT. 👨💻
Conclusion
Hopefully, these tips will help you with your deep learning journey, be it learning something new or solving some nasty problem. And who knows, after tackled some challenges and see the beauty of it, you might fall in love with math, and this will never be an issue for you! Good luck and happy learning!
Written on February 1, 2021 by Michael Li.
Originally published on Medium