I love the idea of autogenerating diagrams from either a schema or from some sort of actual definitions (terraform/cloud inspection/etc), but I've generally found that the layout can be pretty poor for any semi-complex system.
Even worse, a small change to the structure will typically cause a full re-layout, and place nodes completely differently to the previous output, making visual/mental comparisons between versions quite taxing.
Given that autolayout is likely never going to be as good as someone doing some amount of manual layout or at least hinting, it'd be nice if there was some way to allow explicit placement of items and track it as metadata independently of the model changes.
I think Structurizr[1] has some capabilities along these lines, but the editor wasn't great the last time I tried it, and I think largely requires their SaaS for tracking that layout info.
Another option might be to feed it all into some flavour of graph DB with interactive querying/filtering and layout, and let users produce just the diffs / bits that they care about, if practical.
It depends. We are an MSP and we generate the same "building block" network topologies for each of our clients. We are in process of adding auto diagramming, and it will work just fine because we have standardized cloud topology for each client. This allows us to take each security zone of their solution, manually lay out the base and then figure out how to layout optional attached components. Each diagram will look quite different when done, but you will be able to tell they all were done from the same template.
However, I agree auto diagraming some random topology doesn't work nearly as well, and worse if it is a complex one. They tend to look like a spaghetti mess.
That's a good point, if you have a fairly small set of structures and you're mostly just adding/removing connections and changing labels, I can definitely see that working well.
How are you generating the templates/base topology structures for that?
Templates will be manual, so in reality we are just gluing together various parts to make a whole, with an understanding of how big the topology can get so we can do nice layout.
I agree with this. I'm usually not a fan of auto-generated diagrams at all.
I would rather spend the extra time making something that's more readable and has a better layout.
The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)
There's absolutely value in being able to partially generate (maybe just the nodes/edges, but without final layout), or possibly even just validate that a given diagram does actually match a given schema/definition, without necessarily solving the extremely hard problem of a good looking auto-layout.
I've dabbled in the past with trying to do auto-layout of new items while preserving explicit changes by hacking at some of the Cytoscape.js layout providers to heavily penalise moves to manually positioned nodes, but I've never managed to make anything close to what I'd call acceptable for general use.
> The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)
I think there is something of a truism here... hand-made diagrams don't get updated because they are so hard to maintained.
Pretty cool. The biggest thing I think is missing from many popular diagramming tools is the ability to easily diff the architecture across changes. Although that can already be done with graphviz source files.
However, these days I prefer to just have diagrams generated from the source code itself, like:
In order to make a compelling use case out of having to maintain a parallel definition like this, I think it needs to be able to contribute to analysis like formal verification or, despite them stating that it's not a goal, being able to create the described architecture. Or even generate something like a terraform plan. Otherwise it falls victim to the same problem as any other method of creating diagrams, that of them falling out of sync with the system.
They are a long way there already by providing a way to use a programming language backed by a large ecosystem of other tools that can be used to work with e.g. terraform, like python/go.
Isn't it too late for the diagram if the code is already written? I think many diagram use cases are about informing which code should be written to begin with.
For understanding an existing thing, diagram from code seems great.
Depends on the reason for having a diagram. If you need to work on a piece of existing code you’ve never seen before, it can be very helpful to have a diagram as a guide.
If I’m prototyping, I’m probably just going to draw a diagram on a whiteboard, not write code to create one. That’s bringing way too much cognitive overhead into the process that will distract from the task at hand.
I tried doing an AWS systems architecture and it looked so ugly, so, we switcehd to Cloudcraft [0], which does both isometric 3D and 2D. All Terraform tools like inframap were terrible, too. It's 2022 and we still don't have pretty autodiagraming tools - we have ML generating unimaginable in terms of complexity photorealistic images, but it can't generate a pretty diagram yet!
I'm always on the lookout for more diagram as code authoring options. Mermaid is pretty decent, if frustrating at times. Looks like there's a port of it to TypeScript: https://www.npmjs.com/package/diagrams-ts
I'm not digging this. I don't think a full Turing complete language syntax adds much to a diagram tool. I much prefer the conciseness of the syntax of Mermaid compared to this.
Airflow uses it for task ordering within a DAG, but that's used in the parsing step that happens before/outside of any actual task execution so it's not really in the source code of tasks at least.
The Airflow 2.0 TaskFlow API[1] doesn't use the bitshift operators any more -- if you pass the result of one task into another, the second task is automatically downstream of the first. So it all just looks like function calls, despite using XComArgs behind the scenes.
from diagrams import Diagram
from diagrams.aws.compute import EC2
with Diagram("Simple Diagram"):
EC2("web")
This has a very odd API. It's using (abusing?) context managers and contextvars to do weird spooky things that you could just as easily do with ordinary objects or functions.
But wondering why it's using ">>". I'm not great with Python. How does this ">>" thing work and why does it make sense to associate elements with ">>".
I worked in CASE product team at Oracle last century. We had a significant development effort around autolayout algorithms. It's funny seeing all the same discussion decades later.
Even worse, a small change to the structure will typically cause a full re-layout, and place nodes completely differently to the previous output, making visual/mental comparisons between versions quite taxing.
Given that autolayout is likely never going to be as good as someone doing some amount of manual layout or at least hinting, it'd be nice if there was some way to allow explicit placement of items and track it as metadata independently of the model changes.
I think Structurizr[1] has some capabilities along these lines, but the editor wasn't great the last time I tried it, and I think largely requires their SaaS for tracking that layout info.
Another option might be to feed it all into some flavour of graph DB with interactive querying/filtering and layout, and let users produce just the diffs / bits that they care about, if practical.
[1] https://structurizr.com/
However, I agree auto diagraming some random topology doesn't work nearly as well, and worse if it is a complex one. They tend to look like a spaghetti mess.
How are you generating the templates/base topology structures for that?
Diagrams as Code 2.0 • Simon Brown • GOTO 2021: https://www.youtube.com/watch?v=Za1-v4Zkq5E&t=1847s
The C4 model for visualising software architecture: https://c4model.com/
Intro to Structurizr: https://www.baeldung.com/structurizr
I would rather spend the extra time making something that's more readable and has a better layout.
The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)
I've dabbled in the past with trying to do auto-layout of new items while preserving explicit changes by hacking at some of the Cytoscape.js layout providers to heavily penalise moves to manually positioned nodes, but I've never managed to make anything close to what I'd call acceptable for general use.
I think there is something of a truism here... hand-made diagrams don't get updated because they are so hard to maintained.
The Python→Go port was discussed on Sep 27, 2020 — https://news.ycombinator.com/item?id=24603926
Someone was also working on a web editor (maybe with Electron), but I forgot the name of the repository.
However, these days I prefer to just have diagrams generated from the source code itself, like:
- https://github.com/cycloidio/inframap
- https://graphviz.org/Gallery/directed/bazel.html
In order to make a compelling use case out of having to maintain a parallel definition like this, I think it needs to be able to contribute to analysis like formal verification or, despite them stating that it's not a goal, being able to create the described architecture. Or even generate something like a terraform plan. Otherwise it falls victim to the same problem as any other method of creating diagrams, that of them falling out of sync with the system.
They are a long way there already by providing a way to use a programming language backed by a large ecosystem of other tools that can be used to work with e.g. terraform, like python/go.
For understanding an existing thing, diagram from code seems great.
If I’m prototyping, I’m probably just going to draw a diagram on a whiteboard, not write code to create one. That’s bringing way too much cognitive overhead into the process that will distract from the task at hand.
[0]: https://www.cloudcraft.co/
Surely, my diagram shouldn't have programming crud around it
EDIT Actually, might be different...
Not doing crap like that is one of the things that I loved about Python...
[1] https://github.com/tomerfiliba/plumbum
1. https://airflow.apache.org/docs/apache-airflow/stable/tutori...
no
But wondering why it's using ">>". I'm not great with Python. How does this ">>" thing work and why does it make sense to associate elements with ">>".
In this case they’re overriding bitshifting. In terms of why, it does feel like a natural visual choice.
https://github.com/mingrammer/diagrams/blob/master/diagrams/...
Of course C++ has it, and it is one of the great footguns in its arsenal.
I think those garbage "academia" sites are constantly emailing, asking me to "claim" it.