While I appreciate people trying to publish their own explanations of how various NN/Deep Learning architectures work, I think a fully coded article would be much better for the people these are aimed at.
It's like with research papers: The intuitive part isn't that hard to grok (sometimes they are remarkably intuitive) - it's the translation from equations to working code.
I have one friend that's trying to learn ML, and he has a solid mathematical background, but mostly on a theoretical level (i.e not applied). He could explain perfectly well how various neural networks worked, but he really, really struggled with coding them.
In fact, I wish every researcher would put a link to their code repository
It depends on where you are in terms of your learning and background. Some people will find value in intuitive blogposts which complement research papers. Others can “grok” this as you say and will find more value in code. Yet others can “grok” the code if only they had the intuitive understanding. You can see where I am getting: everyone has different backgrounds and needs and different articles will appeal to different audiences. There is no one audience to appeal to.
On the note of code: this tends to be more subjective than the maths in a paper. There are lots of design decisions that can affect the overall structure of the code. Then there is the trade off between readability and performance. Readable code for blogs is probably not the code you would use in practice.
Personally I find it easier to establish the ground truth in the mathematics than to try and figure out the underlying truth from the code. And if you can’t do it yourself, then that’s where the intuitive blogposts help. After that you can choose your own libraries, frameworks, programming design decisions and teachers (including other blogposts) to help you get the code.
I was thinking, can we invent a more formal, common graphical representation of how neural networks work, just like how we have a graphical representation of electronic networks?
We do. If you’re interested in the architecture particulars (I.e. which layer types, layer shapes, layer parameters, activation functions, etc) you’ll commonly see these in deep learning papers as a directed acyclic graph with layers represented by a collection of vertices or rectangles. Other architecture specifics are sometimes noted with text alongside layers. We also have more general representations of deep models which look like, and sometimes are exactly, Probabilistic Graphical Models. PGMs have their own formal language which makes it quite easy to write complex models describing a joint distribution very simply.
However, this is different than interpreting how a deep model works. “Interpreting” is an overloaded and poorly-defined term of active research. The sense I get from my friends in the interpretability/explainability research world is that despite the buzz, there is no common definition that lends to a clear set of requirements for an acceptable interpretation of a neural network.
One of the major selling points of neural networks is that you can model/learn relationships that are very hard to explicitly write down. Also keep in mind that neural networks often work with floating point values to scale input values. Sure, you can write down these floating point values in a diagram, but the numbers themselves will not tell you why they have the magnitude that they have.
With that being said, there is research happening that tries to find better tools to understand the reasoning of neural networks. And sometimes we might actually see and understand a general decision pattern of a NN that we could adopt for a simpler model. But in many cases it will not be possible to come up with a simpler model because some phenomena have an intrinsic complexity.
On a related note, there's a "Interpretable Machine Learning" approach that tries to provide more insight into the "whys" of ML. (I'm not en expert and I've learnt about this quite recently)
My list of "don't even bother" websites is growing. It basically includes anything owned by oath/verizon media and I guess medium is on the list now too.
And someone took the opportunity to sign me up for a bunch of unwanted newspapers... My email hasn't been private for a long time, and so are many other people's emails. That is not an invitation to be an asshole, nor am I a 'stupid fuck'.
It's like with research papers: The intuitive part isn't that hard to grok (sometimes they are remarkably intuitive) - it's the translation from equations to working code.
I have one friend that's trying to learn ML, and he has a solid mathematical background, but mostly on a theoretical level (i.e not applied). He could explain perfectly well how various neural networks worked, but he really, really struggled with coding them.
In fact, I wish every researcher would put a link to their code repository
On the note of code: this tends to be more subjective than the maths in a paper. There are lots of design decisions that can affect the overall structure of the code. Then there is the trade off between readability and performance. Readable code for blogs is probably not the code you would use in practice.
Personally I find it easier to establish the ground truth in the mathematics than to try and figure out the underlying truth from the code. And if you can’t do it yourself, then that’s where the intuitive blogposts help. After that you can choose your own libraries, frameworks, programming design decisions and teachers (including other blogposts) to help you get the code.
https://youtu.be/o64FV-ez6Gw
However, this is different than interpreting how a deep model works. “Interpreting” is an overloaded and poorly-defined term of active research. The sense I get from my friends in the interpretability/explainability research world is that despite the buzz, there is no common definition that lends to a clear set of requirements for an acceptable interpretation of a neural network.
With that being said, there is research happening that tries to find better tools to understand the reasoning of neural networks. And sometimes we might actually see and understand a general decision pattern of a NN that we could adopt for a simpler model. But in many cases it will not be possible to come up with a simpler model because some phenomena have an intrinsic complexity.
https://i.imgur.com/E0MYinp.png
To blog writers, please reconsider hosting your blog on Medium.