This is an overdue recap of the lecture I had 2 weeks ago, where we went through Stochastic gradient descent, momentum, and Adam. I’ve learned the benefits of SGD before, so I won’t go that deeply into it. But I think what’s good about the lecture is how it anchors it in the mathematics. Stochastic Gradient descent To avoid being trapped in a local minimum, at each iteration, the SGD algorithm choose a random subset of training data and computes the gradients from these examples alone. This is known as a minibatch. The parameter update will then consider only that batch alone. ...

The second class of FT5011 took us through loss functions. To be honest, my knowledge of loss functions is a little wonky. Like, I know the general idea, but beyond the simplistic model of “loss as sum of squared errors”, I really don’t know that much. After being completely lost in the class, and subsequently reviewing the chapter on loss functions, I am happy to say that my mental model of loss functions has been sucessfully updated. ...

I’ve always been a little skeptical about technical trading. One of the things that I find hard to buy is the notion that ‘risk’ can be measured purely by the standard deviation of the price of the stock. Surely this only measures how ‘volatile’ a stock is, but it is different from the notion of ‘risk’? Because a stock is risky if it goes in the opposite direction than the position you took and you lose capital, not that its price action may be a bit more neurotic. ...

One of the SMA (simple moving average) strategies proposed in the Hilpisch book goes like this: we construct 2 SMA indicators, one short (42 days) and one long (252 days), and whenever the short indicator is above the long indicator (indicative that recent tide is bullish), we take a long position, and whenever the long indicator is above the short indicator (the reverse), we take a short position. In the book example, they looked at 10 year data of EUR/USD pair. Whenever the green line is above the red line, we maintain a long position; whenever the green line dips below the red line, we maintain a short position: ...

I’ve just begun my second deep learning course with Prof Stanley Kok at NUS - for FT5011. Based on the first lecture alone, I have a feeling that this is going to be a great course. Stanley seems to be a very good explainer of concepts, and despite having learnt deep learning and neural networks before, there were still nuances from the first lecture that I want to write about. ...

And so, I am reproducing it here for reference. My prompt: What are some best methods of gaining mastery in doing deep learning, for instance, dealing with quirks of different environments, shapes, data types, etc? Its response: Great question—and honestly, the fact that you’re thinking about quirks (envs, shapes, dtypes) is already a sign you’re moving from “tutorial deep learning” toward real mastery. I’ll break this into mindsets, habits, and deliberate practice loops—because deep learning skill compounds less from reading papers and more from debugging scars. ...

Training a CNN from scratch only got my accuracy up till 70%, whereas with transfer learning, I managed to get my accuracy up to 90%. What do I mean by transfer learning here? It means that I used a Resnet50 pretrained model as a departure point, and only fine tuned the final linear layers. The entire model implementation looks like this: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 def build_model(num_classes=2, freeze_backbone=True): transfer_model = models.resnet50(weights='DEFAULT') if freeze_backbone: for param in transfer_model.parameters(): param.requires_grad = False transfer_model.fc = nn.Sequential( nn.Linear(transfer_model.fc.in_features, 500), nn.ReLU(), nn.Dropout(), nn.Linear(500, num_classes) ) for param in transfer_model.fc.parameters(): param.requires_grad = True device = torch.device("cuda" if torch.cuda.is_available() else "cpu") transfer_model = transfer_model.to(device) return transfer_model Here, I rely on a build_model function to instantiate my model. ...

Learnings: Using Alexnet model with kernel settings of size=11, stride=4, padding=2, and image size of 227x227, I could reach 70% accuracy with training of 30 epochs. This, unfortunately, is the same level of accuracy as my Simplenet in the previous post Training Using a simple neural network from the previous post, I could reach an accuracy level of 0.7. However whilst implementing the following CNN from chapter 3 of the Pytorch book, I realized that I had quite a few issues trying to get it to converge. ...

After having completed my CS5242 course on neural network and deep learning, I realized that there’s a lot more practice that I’ll need to do in order to deploy useful models out into the wild. Which means that I would really want to work on and fully master the fundamentals. One of them being Pytorch. And so, I began on a Pytorch book by Ian Pointer. In the first exercise, I implemented a simple 2 layer neural network to classify whether an image is a cat or a fish. This is not new to me, but still, I think there’s a couple of things about this implementation that I should learn about: ...

So I’ve decided to revive this project after a long while haha. What exactly is this project? Idk, technical writing, as well as sporadic posts about life. I have a substack, but that’s a little more for polished posts about general topics in life, whereas this one is for rough sketches of everything I find interesting, as well as an online documentation of my journey with certain things. And, as the current domain name suggests, I am officially in my deep learning phase in my life, and I want to carve out a space to write about everything that I learn on that front. ...