2023-04-08

The fast.ai book goes into much more detail than the course. It's quite cool.

High level learnings from Chapter 5:

• Multi-class classifiers use cross-entropy loss which you get by softmax followed by negative log likelihood
• fastai uses different learning rates, slower for base layers and faster for newer layers. This is called discriminative learning rates.
• if selecting number of epochs, a good idea is to train it for as long as you can wait and then see how good the results are.
• Kaggle is slow as fuck. Things that take 20s in the fastai notebook take 2 minutes on the kaggle notebook.

I still need to figure out a good small project to start with.

Ideas: Re-build something like AlphaGo from scratch, should be a nice learning exercise.: https://arxiv.org/pdf/1712.01815.pdf

Had a pretty long conversation with Shikhar yesterday (we met after an year). One thing that I'll note down is that optimizing for money and making money your goal seems to demonstrate lack of meaning in life to me. Money isn't really anything other than a number on a screen. I don't really know what I want to do either, but I think I've made progress in knowing that saying "I'll make X money and then I'll actually do what I want to do" is a huge trap.

However, another good point was that I still need to sustain myself. So I need to figure out a way to be able to do cool things while still being reasonably ok moneywise.