Curve Detecting Neurons

(distill.pub)

54 points | by sdenton4 149 days ago

4 comments

  • sdenton4 149 days ago

    This is a really cool dive into the interpretability of intermediate-level neural network activations. I really like the way that they've identified a core debate in the ML conversation, and created a relatively simple test-space to better understand it and (hopefully) move the discussion forward.

    • rjeli 148 days ago

      Very nice. Was a bit confused by the jump to splines and fitting splines to butterflies, more explanation would be appreciated

      • bionhoward 148 days ago

        They write about neurons having “meaning” without a strict definition. What’s the quantitative meaning of meaning?

        • MrMontyBurns 148 days ago

          They fulfill a task, if you want the short answer. No idea how to define 'task' here

        • nextaccountic 149 days ago

          This isn't immediately clear on whether it's discussing biological neurons or artificial neurons. I suppose it's the latter.

          • clemParis 149 days ago

            All the authors are part of OpenAI, and the very first sentence talks about computer vision and a parallel with real neurons. So "This isn't immediately clear" should be read as "I only read the title and didn't even try to understand what the article is about".