Posts

We are living in such crazy times. Why? Let me take you through how I am working on a web app project these days. First, I write down on a high level what I want to build. Then, I pass these instructions to Claude design for it to design the overall wireframes for the app. Typically, its first few attempts are not bad, but a bit of prompting is required to get it to truly satisfactory levels. Then, I ask it to generate high-fidelity wireframes. It intuites what I need, implements best UI/UX practices, finds good color schemes for me, and generates a good looking hi-fi wireframe. At this stage, I’ll ask it to generate a design handoff folder to be passed into LLMs. ...

This is such a sad novel. I cried at so many parts. Perhaps what was the most heart wrenching was the story of Chin Chin Man - the father of our protagonist, Gifty - a towering, dignified man from Ghana, who upon going to America, faces an implied racism which causes his gait to shrink. Rejected and dejected, the Chin Chin man had to settle for jobs such as a janitor. In Gifty’s memories, the Chin Chin man is still a friendly giant amongst the kids, never losing much of his cheer, until one day, amidst periods of intense conflict, he just disappeared. ...

At the start of the year, there was a few things that I wanted to do. First, complete my course - that I’ve done. Even if I still need to do another semester, I’ll treat it as effectively done, and that any future involvement will be way lower touch. Second, I wanted a change in job scope. That, I’ve also done. Oh, did I mention, I am now working in secops in my company? Security operations. I’ve managed to successfully apply for a job switch. I am grateful that it came through. It’s only been one week in my job, but so far, I am really liking it. Cybersecurity. This field, for now, is deep enough, and the level of craft here is really endless. It’s fascinating and also powerful. Fortunately, despite having only a lay person’s knowledge of this field, it’s one of those roles that allows you to pick up on the job and to learn by doing. For that, I count myself truly lucky. ...

The last few months has been crazy. Driven by the need to get an A for both of my courses in order to graduate, my life assumed a certain relentless discipline. Work, home, school, studying in the evenings, studying over the weekends, hanging out with Shaune, exercising once in a while. It was just so long. A full 13 weeks of this schedule. Towards the end, I did feel extremely tired. I miss hanging out with my friends. I miss the feeling of just switching off your mind, turning off the need to constantly ask, “what’s next”. I miss going on purposeless meanders, to dwell, and to take luxurious indulgent breaks. To breathe slowly, and to soak up each moment of the day. ...

For my assignment, I was training a 1D CNN model to predict stock prices. Whilst most of the models gave 0.5% accuracy, there was one version of the model trained with a learning rate of 1e-3 that was giving me 80% accuracy. I am a little skeptical. It feels a little too good to be true. A model that gave 0.8% accuracy would mean that this model would be extremely profitable. Sounds a litle too good to be true? ...

I wanted to see what are the effects of training some simple models on this dataset, namely, a Shallow 1-layer neural network, a deep 3-layer neural network, a variant of the deep neural network with wider dimensions, as well as a simple 1-layer CNN. The results of the models are as follows: Model Dimensions Accuracy Shallow NN 16 78.4% Deep 3-layer NN 8-8-8 79.2% Deep 3-layer NN 16-12-8 80.6 CNN 8 82.6% These corresond to models #1-4 on this page: https://models.minimumloss.xyz/ ...

Regularization is the process of ‘smoothening out’ your loss function, such that it is less responsive to the peculiarities of a specific training dataset and therefore does better at interpolating to unseen data. It attempts to resolve the problem of fitting well on training data but doing badly on test data. This process involves adding a new term to the loss function. In machine learning, the term regularization is also generally used to refer to any strategy that helps improve generalization. ...

When we’re training our models, we’re minimizing loss. Yet, not all loss are created equal. Broadly speaking, there are 3 sources of errors: Noise Noise is the inherent randomness of the test data. Assuming that you have a model that perfectly fits the true underlying function, the test data you draw will still lie within the SD of the true data. This is error that cannot be gotten rid of. ...

Typically when initializing parameters ($\beta$, $\Omega$) for a network, we choose values from a standard normal distribution (with mean 0 and variance $\sigma^2$). Now, the most important factor in preserving the stability of the network as we move past different layers (or functional transformations) is the variance of the initialization. This affects the magnitude of your preactivation f, activations h in the forward pass, as well as your gradients in your backward pass. This almost solely determines whether you’ll suffer from vanishing or exploding gradient problems. ...

It’s incredible how entire models can be represented as a series of equations, and the entire process of back propogation (the spirit matter of deep learning) can also be represented as a series of equations. I’ll type this out fully when I have the time, but here’s what most neural networks are, abstracted: The math of back propagation On the top left (1), you see a typical 3 layer neural network represented by a series of preactivations f and hidden units h functions. That’s all there is to it! It’s a very compact representation of a complex series of functional compositions. ...