WritingMachine learning, mostly
Notes on how models learn, and how to make them run anywhere.
I'm Surya. I write long-form, interactive notes on transformers, learning theory and machine learning for audio, with the math, the code and a demo you can poke at in each one.
Writing
4 posts · newest first
- 004 Attention, from the softmax up Scaled dot-product attention one piece at a time, with a temperature dial you can turn to watch a query make up its mind.
- 003 A language model in this tab A small instruction-tuned model running inside the page, on your GPU through WebGPU or on your CPU, with nothing sent to a server.
- 002 Low-rank updates, plotted What LoRA's rank actually buys you, as pixels and as curves. A sketch of a sweep, with charts you can hover, filter and smooth.
- 001 Watching an optimiser move Gradient descent with momentum on a surface you can click, and a short film of a diffusion model turning noise into a spiral.
About
I'm Surya Kant Sahu. I'm impatient with slow programs, so I tend to end up with a pile of second-hand machines, a local network and NCCL, and a job that runs faster than it did yesterday.
I'm interested in transformers, learning theory, reinforcement learning and machine learning for audio and music. I work mostly in Python and C++, with PyTorch, ONNX, Numba and Cython.
- Paper
- On the Connection of Benford's Law and Neural Networks (2021), with A. Java and A. Shaikh
- Code
- github.com/ojus1
- Feed
- RSS