Diagram as Code

(github.com)

151 points | by sebg 12 days ago

15 comments

  • shabble 12 days ago
    I love the idea of autogenerating diagrams from either a schema or from some sort of actual definitions (terraform/cloud inspection/etc), but I've generally found that the layout can be pretty poor for any semi-complex system.

    Even worse, a small change to the structure will typically cause a full re-layout, and place nodes completely differently to the previous output, making visual/mental comparisons between versions quite taxing.

    Given that autolayout is likely never going to be as good as someone doing some amount of manual layout or at least hinting, it'd be nice if there was some way to allow explicit placement of items and track it as metadata independently of the model changes.

    I think Structurizr[1] has some capabilities along these lines, but the editor wasn't great the last time I tried it, and I think largely requires their SaaS for tracking that layout info.

    Another option might be to feed it all into some flavour of graph DB with interactive querying/filtering and layout, and let users produce just the diffs / bits that they care about, if practical.

    [1] https://structurizr.com/

    • nu11ptr 12 days ago
      It depends. We are an MSP and we generate the same "building block" network topologies for each of our clients. We are in process of adding auto diagramming, and it will work just fine because we have standardized cloud topology for each client. This allows us to take each security zone of their solution, manually lay out the base and then figure out how to layout optional attached components. Each diagram will look quite different when done, but you will be able to tell they all were done from the same template.

      However, I agree auto diagraming some random topology doesn't work nearly as well, and worse if it is a complex one. They tend to look like a spaghetti mess.

      • shabble 12 days ago
        That's a good point, if you have a fairly small set of structures and you're mostly just adding/removing connections and changing labels, I can definitely see that working well.

        How are you generating the templates/base topology structures for that?

        • nu11ptr 5 days ago
          Templates will be manual, so in reality we are just gluing together various parts to make a whole, with an understanding of how big the topology can get so we can do nice layout.
      • adolph 12 days ago
        I just watched the below video from last year partly about Structurizr. Below are other tabs about the topic I still have open.

        Diagrams as Code 2.0 • Simon Brown • GOTO 2021: https://www.youtube.com/watch?v=Za1-v4Zkq5E&t=1847s

        The C4 model for visualising software architecture: https://c4model.com/

        Intro to Structurizr: https://www.baeldung.com/structurizr

        • TranquilMarmot 12 days ago
          I agree with this. I'm usually not a fan of auto-generated diagrams at all.

          I would rather spend the extra time making something that's more readable and has a better layout.

          The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)

          • shabble 12 days ago
            There's absolutely value in being able to partially generate (maybe just the nodes/edges, but without final layout), or possibly even just validate that a given diagram does actually match a given schema/definition, without necessarily solving the extremely hard problem of a good looking auto-layout.

            I've dabbled in the past with trying to do auto-layout of new items while preserving explicit changes by hacking at some of the Cytoscape.js layout providers to heavily penalise moves to manually positioned nodes, but I've never managed to make anything close to what I'd call acceptable for general use.

            • Veuxdo 12 days ago
              > The only exception for me would be for something that changes frequently where any diagram made by hand would go out of date quickly or be burdensome to maintain (not that diagrams usually get maintained...)

              I think there is something of a truism here... hand-made diagrams don't get updated because they are so hard to maintained.

          • guessmyname 12 days ago
            A year ago, someone ported this Diagrams (Python) into Go — https://www.reddit.com/r/golang/comments/is8eo9/godiagrams_c...

            The Python→Go port was discussed on Sep 27, 2020 — https://news.ycombinator.com/item?id=24603926

            Someone was also working on a web editor (maybe with Electron), but I forgot the name of the repository.

            • sixstringtheory 12 days ago
              Pretty cool. The biggest thing I think is missing from many popular diagramming tools is the ability to easily diff the architecture across changes. Although that can already be done with graphviz source files.

              However, these days I prefer to just have diagrams generated from the source code itself, like:

              - https://github.com/cycloidio/inframap

              - https://graphviz.org/Gallery/directed/bazel.html

              In order to make a compelling use case out of having to maintain a parallel definition like this, I think it needs to be able to contribute to analysis like formal verification or, despite them stating that it's not a goal, being able to create the described architecture. Or even generate something like a terraform plan. Otherwise it falls victim to the same problem as any other method of creating diagrams, that of them falling out of sync with the system.

              They are a long way there already by providing a way to use a programming language backed by a large ecosystem of other tools that can be used to work with e.g. terraform, like python/go.

              • beardedwizard 12 days ago
                Isn't it too late for the diagram if the code is already written? I think many diagram use cases are about informing which code should be written to begin with.

                For understanding an existing thing, diagram from code seems great.

                • sixstringtheory 12 days ago
                  Depends on the reason for having a diagram. If you need to work on a piece of existing code you’ve never seen before, it can be very helpful to have a diagram as a guide.

                  If I’m prototyping, I’m probably just going to draw a diagram on a whiteboard, not write code to create one. That’s bringing way too much cognitive overhead into the process that will distract from the task at hand.

              • nikolay 11 days ago
                I tried doing an AWS systems architecture and it looked so ugly, so, we switcehd to Cloudcraft [0], which does both isometric 3D and 2D. All Terraform tools like inframap were terrible, too. It's 2022 and we still don't have pretty autodiagraming tools - we have ML generating unimaginable in terms of complexity photorealistic images, but it can't generate a pretty diagram yet!

                [0]: https://www.cloudcraft.co/

                • bilalq 12 days ago
                  I'm always on the lookout for more diagram as code authoring options. Mermaid is pretty decent, if frustrating at times. Looks like there's a port of it to TypeScript: https://www.npmjs.com/package/diagrams-ts
                  • deltree7 12 days ago
                    The next step would be to make it declarative like PlantUML or Ditaa.

                    Surely, my diagram shouldn't have programming crud around it

                    • abraxas 12 days ago
                      I'm not digging this. I don't think a full Turing complete language syntax adds much to a diagram tool. I much prefer the conciseness of the syntax of Mermaid compared to this.
                      • SaulJLH 12 days ago
                        Already posted https://news.ycombinator.com/item?id=31807021

                        EDIT Actually, might be different...

                      • wmil 12 days ago
                        Is it common for python libraries these days to overload the >> operator for oddball things like C++ does?

                        Not doing crap like that is one of the things that I loved about Python...

                        • shabble 12 days ago
                          if you liked that, you'll love Plumbum[1] :)

                          [1] https://github.com/tomerfiliba/plumbum

                          • marginalia_nu 12 days ago
                            I'm waiting for Guido to rush in any moment going "Stop right there, criminal scum! Nobody breaks the law on my watch!"
                          • tgarv 12 days ago
                            Airflow uses it for task ordering within a DAG, but that's used in the parsing step that happens before/outside of any actual task execution so it's not really in the source code of tasks at least.
                          • neuronexmachina 12 days ago
                            The Numpy and Tensorflow libraries do a fair bit of operator overloading, but it's IMHO more sensible than this example.
                            • detaro 12 days ago
                              > Is it common for python libraries these days to overload the >> operator for oddball things like C++ does?

                              no

                            • j4mie 12 days ago

                                from diagrams import Diagram
                                from diagrams.aws.compute import EC2
                              
                                with Diagram("Simple Diagram"):
                                    EC2("web")
                              
                              This has a very odd API. It's using (abusing?) context managers and contextvars to do weird spooky things that you could just as easily do with ordinary objects or functions.
                              • VectorLock 12 days ago
                                I think you can still use ordinary functions to string things together as well. I also don't like the abuse of context managers in this way.
                              • zander312 12 days ago
                                Love the idea, definitely will use.

                                But wondering why it's using ">>". I'm not great with Python. How does this ">>" thing work and why does it make sense to associate elements with ">>".

                                • aidos 12 days ago
                                  Python has dunder methods that allow you to override behaviour (eg operator overloading, hashing, object identity etc).

                                  In this case they’re overriding bitshifting. In terms of why, it does feel like a natural visual choice.

                                  https://github.com/mingrammer/diagrams/blob/master/diagrams/...

                                  • AtlasBarfed 12 days ago
                                    operator overloading. One of the great holy wars of PL.

                                    Of course C++ has it, and it is one of the great footguns in its arsenal.

                                    • aidos 12 days ago
                                      With great power comes great responsibility.
                                • AlbertCory 12 days ago
                                  I was ahead of my time. My M.S. thesis was "A flowchart-to-source-code conversion program for GPSS/360."

                                  I think those garbage "academia" sites are constantly emailing, asking me to "claim" it.

                                  • abraae 12 days ago
                                    I worked in CASE product team at Oracle last century. We had a significant development effort around autolayout algorithms. It's funny seeing all the same discussion decades later.
                                    • AlbertCory 11 days ago
                                      We might have overlapped at Oracle. I left in Jan. 1998.
                                  • Yhippa 12 days ago
                                    I actually kind of like this. Would be neat if there was a way once the initial diagram is generated to visually edit it.
                                    • mkl95 12 days ago
                                      Interesting, although I'm way more excited about the inverse (code as diagram).
                                      • alexott 12 days ago
                                        What is the advantage over PlantUML, for example? Only syntax?