# Notes: Zoom In circuits Article
Just finished reading this and wanted to write down what stuck with me. This came out in 2020 and is apparently the first in a whole series they've been publishing, so I'm curious how these ideas held up and where they went with it.
Main idea
The article is basically arguing that we can understand neural networks by zooming in on small pieces - like looking at individual neurons and how they connect. They call these little connected groups "circuits."
Instead of trying to explain the whole network at once (which seems impossible), they're saying we should treat it more like biology - look at small parts, run experiments, build up understanding piece by piece.
The three claims
They make three main claims:
Features exist and make sense - Individual neurons actually detect real things. Not just edges in early layers, but stuff like curve detectors, dog heads, car parts in later layers. They're saying most neurons can be understood if you put in the work, not just a few lucky examples.
Circuits are real - Neurons connect in meaningful ways that implement actual algorithms. You can literally look at the weights and see what computation they're doing.
Universality shows up across models - Different networks trained on similar tasks learn similar features. If this is true it's a big deal, if not then we're stuck understanding each model individually.
What actually clicked for me
The curve detector example - They show how curve detectors work in detail. The network builds curve detectors by combining earlier, simpler features. And when you visualize the weights themselves, you can see the curve shape in the numbers. That was wild. The weights aren't random - they have structure you can see and understand.
The dog head detector - This one was interesting because it's more complex. The network maintains separate pathways for left-facing and right-facing dog heads, then combines them into a pose-invariant detector. It could have just done something messy but it actually carved the problem into cases and handled them separately. Seems like a smart solution.
Polysemantic neurons are annoying - Some neurons respond to multiple unrelated things (like cat faces AND car fronts). They call this "superposition" - the network is packing multiple features into limited neurons to save space. Makes sense why it would do this but makes interpretability way harder.
Stuff I'm skeptical about / want to see more on
The universality claim seems pretty speculative here. They show a few examples of similar features across models but straight up admit they haven't done enough work to know if it holds broadly. That's kind of a huge open question - does every model learn its own thing or is there a shared "periodic table of features"?
Also wondering how well this scales. They're looking at InceptionV1 from 2014. Does this approach still work on transformers? On way bigger models? On models doing different tasks? They seem aware these are open questions but I'm curious what they found in the follow-up work.
The polysemantic neuron problem also seems like a real blocker. If lots of neurons are doing multiple unrelated things, how far can you really get with this? They mention maybe "unfolding" networks to separate these out but that sounds hard.
What I want to know from the later work
Since this is from 2020 and they've published more:
- Did the universality hypothesis hold up? Do we actually see the same features across models?
- How does this work on modern architectures? Transformers vs CNNs?
- Did they make progress on polysemantic neurons / superposition?
- Can you actually use circuit understanding to predict or control model behavior?
- How far did they get on mapping out all the features in even one model?
Random thoughts
They compare it to microscopes and cellular biology which feels right. Not trying to explain everything at once, just building up understanding of small pieces.
The field seemed kind of messy in 2020 (they call it "pre-paradigmatic") - no standard definitions or ways to evaluate work. Wonder if that's still true or if circuits became a real paradigm.
I appreciate that they're honest about the hard parts. Not overselling, just saying "here's what we've found, here's what we don't know."
The idea that you can read algorithms off the weights is pretty cool. Makes the network feel less like a black box and more like something with internal logic.
Where this leaves me
Mostly just thinking neural networks might be more interpretable than I assumed. Still hard, still lots we don't understand, but not fundamentally unknowable. There's actual structure in there.
But also these were pretty specific claims about vision models from 2014. Need to see if this actually generalized or if it was just nice examples that looked good. Curious to dig into the follow-up papers and see what actually panned out vs what turned out to be wrong or limited.