p(x|y) as a nonlinear correlation metric?
Correlation captures a narrow kind of dependence: linear association between variables. Could comparing conditional distributions, \(p(x \mid y)\), provide a more general way to character...
Loose Threads
A working map of random ideas and thoughts: questions, half-formed connections, things I’m reading, and small experiments. Leaving this deliberately unpolished to see how my thoughts evolve.
A map of loose connections
Ring colour is the note’s major topic; the ring’s line style is what kind of note it is. Hover a node for its title, or filter below to highlight a group.
Type
Topic
What does the retina choose to keep?
The retina is not simply a camera that forwards pixels. Its output is already a transformed representation of the visual scene. What are the relevant constraints behind that transformatio...
What does the Schur complement have to do with information?
The Schur complement appears in the conditional covariance of jointly Gaussian random variables. It seems to isolate the part of \(X\)’s variability that remains after accounting for \(Y\).
What does invariance cost?
A representation that ignores nuisance variation can make downstream decisions easier. But invariance is also information loss: distinctions in the input are deliberately collapsed.
Can you compute a price?
Mises argued in 1920 that a planned economy can’t work: with no prices there’s no way to express what anything costs in terms of everything else. Lange said fine, the planner solves the e...
Notation decides what's easy to think
Try long division in Roman numerals. The algorithm isn’t harder, the notation just won’t hold the intermediate state.
Effective inductive biases are the path to efficient models
Scale works and it costs more every year.
Everyone kept their own noon
Until the 1840s every English town ran on local solar time. Bristol sat about ten minutes behind London and nobody minded, because nothing moved fast enough for it to matter.
Noise as a probe of representation
Rather than treating noise only as something to remove, use it as a way to interrogate a representation. Perturb the input, intermediate features, or readout and ask which changes matter ...
Why did the spreadsheet win?
Excel probably has more users than every programming language combined, and most of them wouldn’t say they write code.
Apart from scale, what do transformer models get us?
Transformers showed up at the same time as a jump in data and compute, so the two are hard to pull apart.
The Tube map is wrong on purpose
Beck’s 1933 diagram is wrong about distance nearly everywhere and is the better map for it. Riders want stop counts and interchanges, not geography.
How do we get closer to the biology we actually care about?
Model systems get picked for how easy they are to work with. Immortalized lines, 2D monolayers, one cell type, no mechanical load. Each is a step away from the thing you’re modeling, and ...
No notes in this thread yet.