miniblog.

← Back to all posts
May 4, 2019 at 12:11
An excellent overview of the different gradient descent algorithms, and a nice example of content that is available as both a responsive website and a PDF on arXiv:
An overview of gradient descent optimization algorithms
An overview of gradient descent optimization algorithms
Gradient descent is the preferred way to optimize neural networks and many other machine learning algorithms but is often used as a black box. This post explores how many of the most popular gradient-based optimization algorithms such as Momentum, Adagrad, and Adam actually work.

Related Posts

Apr 2, 2026 at 09:44
2
Some excellent safety improvements in C++26: hardening the standard library against out-of-bounds bugs, and a safer semantics for uninitialised local variables:
C++26 is done! — Trip report: March 2026 ISO C++ standards meeting (London Croydon, UK) – Sutter’s Mill
Mar 20, 2026 at 12:02
223
Excellent series on Self, both the programming model and the tool chain. I love how opening an inspector on a value (e.g. nil) then shows a link to all occurrences of the value in other open inspectors!
Environment and the programming language Self (part one; environment)
Environment and the programming language Self (part one; environment)
I bring you a message about a language that has been at the birth of many others, but almost no one knows it. A rumor of a graphical environment offering options like no other, but almost no one using it. I also bring information about a virtual machine supporting reflection almost to infinity, reaching almost half of the operating power of C in numerical calculations in its time, yet today forgotten.
Aug 17, 2025 at 08:05
3
The Matter protocol is excellent for puns. "Does it Matter? It doesn't Matter."