Lightweight UI component tree definition syntax, DOM creation and differential updates using only vanilla JS data structures (arrays, iterators, closures, attribute objects or objects with life cycle functions, closures). By default targets the browser's native DOM, but supports other arbitrary target implementations in a branch-local manner, e.g. to define scene graphs for a canvas element as part of the normal UI tree.
Benefits:
- Use the full expressiveness of ES6 / TypeScript to define user interfaces
- No enforced opinion about state handling, very flexible
- Clean, functional component composition & reuse
- No source pre-processing, transpiling or string interpolation
- Less verbose than HTML / JSX, resulting in smaller file sizes
- Supports arbitrary elements (incl. SVG), attributes and events in uniform, S-expression based syntax
- Supports branch-local custom update behaviors & arbitrary (e.g. non-DOM) target data structures to which tree diffs are applied to
- Suitable for server-side rendering and then "hydrating" listeners and components with life cycle methods on the client side
- Can use JSON for static components (or component templates)
- Optional user context injection (an arbitrary object/value passed to all component functions embedded in the tree)
- Default implementation supports CSS conversion from JS objects for style attribs
- Auto-expansion of embedded values / types which implement the IToHiccup or IDeref interfaces (e.g. atoms, cursors, derived views, streams etc.)
- Only ~5.5KB gzipped
I'm interested to see this how this combination of ideas spreads outside the browser and into other applications of UI. As for myself, I've been experimenting with a writing a terminal emulator/component lib[3] using Clojure that is inspired by Reagent/Om and React Native. It's been satisfying seeing the way these ideas compose.
[1] - https://reagent-project.github.io/
[2] - https://github.com/omcljs/om
[3] - https://github.com/aaron-santos/zaffre/blob/master/src/examp...
Quote: "The syntax is inspired by Clojure's Hiccup and Reagent projects, which themselves were influenced by prior art by Phil Wadler at Edinburgh University, who pioneered this approach in Lisp back in 1999. hdom offers several additional features to these established approaches."
However, Reagent also still relies on React under the hood whereas hdom is completely standalone. Also, hdom has a lot more features in terms of what kind of values are supported within an hiccup tree and additionally, in this latest version (5.0), hdom already has been extended to support non-DOM targets to update. Currently, the only user of this feature is the hdom-canvas package (https://github.com/thi-ng/umbrella/blob/master/packages/hdom...), which allows you to define canvas scene graphs in the same way as the rest of the UI, but those virtual shape elements are translated into canvas draw calls. A WebGL version of this is in early progress. In general, by implementing a certain interface you can support any other target context/data structure.
Ps. I've never used Om so didn't list it as an active influence.
I always wondered: if you use reasonable compression (what is reasonably expected to be provided by CDNs these days? brotli? I'm so out of the loop..), if you use this, how much does file size actually matter? Isn't it about entropy, rather than size? Say, if every HTML tag required 136 <'s to open, and 136 > characters to close. How would it actually affect different compression algorithms?
My intuition says that we're all on this wild goose chase for smaller file size while it may (should) not matter at all.
For example, I took Wikipedia's page on entropy and replaced each < by 136*<, same for >. I bring you the file sizes for different algorithms:
I don't know what compression algo is de rigeur these days, and 136 <s is obviously not the same as substituting double tags for sexprs. Still, I hope we can put this file size boondoggle in the perspective of entropy one day, instead of just mindlessly chasing the character count dragon.EDIT: turns out you can convert HTML to pseudo sexprs with some regex. here are more realistic numbers:
Less radical, but still following the trend.I'm sure it's fine for commercial JS web apps, but those are bad for the web too.
HTML is the web. All this pure es6/dom stuff is a cancer.
It all depends what one wants to do with a browser, doesn't it? For some of my projects the browser is merely a sandbox for delivering design tools and I really don't care about the HTML aspects of it at all.
For other projects, I only care about HTML generation and use hdom's sister library to generate static HTML from the same components:
https://github.com/thi-ng/umbrella/tree/master/packages/hicc...
If a user has JS disabled in the browser, fine. If not, then the browser app can hydrate the static HTML and add interactive features and cause cancer :)
I call BS on that. The accessibility tools that people actually use only work with "proper" browsers (counting in IE here) that all support Javascript. Modern browsers don't even officially support disabling Javascript. For most web apps, the only feasible way to degrade from "no Javascript" is to display "this page requires Javascript". Spending any more effort here because it's "good practice" is a complete waste of time. Put some effort into testing screen readers instead.
> HTML is the web.
Nonsense.
> All this pure es6/dom stuff is a cancer.
Hey, who are you to tell people what to use the web for? If somebody wants to serve as static HTML page, nobody is stopping them. Nobody is forcing people to use some Javascript framework for that.
That's right. The big browsers have all been co-opted (even firefox) and now target Grandma browsing Facebook and other SPA as their demographic. This is bad for the web but very good for corporations making money. These two things aren't very compatible no matter how much most web devs are invested in not realizing there's a difference.
I'm telling you how I see it and how I design my sites. None of them require javascript to function (even for my comment system). I can do this because I'm not being paid I'm just doing it for fun. I understand people being paid have to make bad websites. But that doesn't make it okay.
my favourite way of demonstrating this was actually to turn js off and still see everything working. the progressive enhancement flag is still flying, and tools like this make it easier by far.
in this case, the server could be (soon!) java/clojure, which is insanely hard to do in an idiomatic, pleasant way with react components. really looking forward to getting this all wired up, and if anyone is interested we're discussing how to get hdom talking to CLJS here: https://github.com/thi-ng/umbrella/issues/36
Decompressing a larger amount of content on the client side, and then parsing it, is going to take more resources. Now, it may be a minuscule amount for one page load, but if you multiply it by hundreds of requests and thousands of users, I can see why smaller file sizes can be still be important.
I think there's fine line between aiming to be lightweight and aiming for smallest whatever by sacrificing all other aspects . I'm definitely in the former camp...
That demo will need to be refactored too and the plan is to extract those SVG components into their own package sometime during the winter break...
[1] https://www.flickr.com/photos/toxi/albums/72157627351329656
[2] https://github.com/thi-ng/umbrella/blob/master/packages/rstr...
This is why I prefer something like lit-html with actual markup.
Example from hdom readme:
In lit-html: Yeah, there some extra weight to the close tag, but it looks like HTML.Disclaimer: lit-html author. https://github.com/Polymer/lit-html
The benefits of the component as plain, non-stringified data are not the avoidance of closing tags, but what you can do with those components computationally. E.g. instrumentation, behavior/attribute injections, configuration, filtering etc. An approach based on string interpolation can only go this far, whereas having everyting as arrays, objects, iterators your possibilities of composition & transformation are almost endless...
Also, take for example the hdom-canvas support library which doesn't target the DOM itself, but defines virtual elements translated into drawing calls. Using the string interpolation approach would be complete overkill since these elements would have to be re-parsed again, in which case SVG would be faster. But I can easily manage 10k+ moving particles @ 60fps with the hdom approach:
https://demo.thi.ng/umbrella/hdom-canvas-shapes/#points-10k
FYI, lit-html isn't string interpolation and templates are data, so you can transform and compose templates, operate on arrays/iterables of them, or Promises or async iterables of them, etc.
And it doesn't re-parse HTML (because it's not string interpolation), so that particle demo could also be fast - elements can also represent draw calls like a scene graph. I've been meaning to do a similar demo actually...
From a quick glance at the docs, though it seems to me there're a lot of other differences in approach (apart from the syntactic differences), especially WRT state. I will have to study these further before being able to comment properly...
Just one more note about XML vs. JS syntax to describe elements. As I said earlier, am sure there's ton of people who prefer the familiar way, but objectively, do you really find hdom's approach less readable? Honest question!
Check out this app component in PWA Starter Kit: https://github.com/Polymer/pwa-starter-kit/blob/master/src/c...
Or the template from the js-framework-benchmarks: https://github.com/krausest/js-framework-benchmark/blob/mast...
Or the template from one of the wired-elements: https://github.com/wiredjs/wired-elements/blob/master/packag...
Given HTML-specific syntax-highlighting and large templates, I really think the HTML format is easier to read.
Isn't also a large appeal of working with components to compose UIs in a more abstracted manner, with different semantics? The hiccup format doesn't use anything alien to web developers and is (objectively!) less noisy in terms of punctuation to express the same concepts. And array manipulation in JS is (again objectively!) easier than manipulating the same kind of data in some <template> DOM. Hdom's syntax is arguably also less tainted by HTML, and that makes sense and was desired, precisely because it is NOT only aimed at the browser DOM, which is just one target (the default). Since the syntax is just `[tag, attribs, ...body]` it's sufficiently general to work equally well for many other use cases not aimed a browser DOM, e.g.
- Server-side rendering
- Static site generation
- WebGL UIs, shader definitions, entire scenes
- Tagged data interchange format between different apps (on top of JSON)
- 3D printing (use as interim format before generating final G-code)
- ...
Lastly, from your new examples given, in my mind, it seems your project is simply aimed at not just somewhat different use cases (albeit there are of course large overlaps), but also a somewhat different audience. Note, I'm not saying that hdom is worse or better than lit-html, it's simply based on a (quite) different philosophy. So please don't compare apples with pears!
I think JSX is popular because of historical reasons. Not the history of JSX, but the Web in general, specifically about HTML being the primary API to build web interfaces since the early days. Everyone feels at home with HTML because of this. We all started with it and it is unquestionable for many.
DOM is the second way to build a web interface. But nobody in their right mind would prefer DOM over HTML as long as HTML is sufficient for the task at hand.
The advent of stuff like Hyperscript and virtual DOMs of today introduced a new alternative to HTML and DOM: Now we can build our interfaces using simple data structures (POJOs). This was already possible from the beginning, but the interactivity requirements of the modern application forced us to invent the "UI as a function of the state" paradigm and this approach became mainstream after that (AFAIK).
But wait... Isn't HTML a data structure as well? Not sure. It clearly started as some kind of DSL for building web interfaces. Then XML is invented and HTML was declared as being an extension of XML; thus some kind of data structure (roughly). Then it got complicated. Today I have no idea about what HTML is.
This is the first part of the story.
The second part is about RPC. This one is short. XML used to be everywhere until the advent of JSON. Suddenly everybody started to embrace JSON and hate XML; for several reasons most of which are reasonable.
The question is....
Why isn't the same thing happening with user interfaces? HTML is a subset of XML (or something like that), and the syntax proposed in this article is clearly JSON. Why is everybody embracing JSX like crazy. Do they not find it extremely ugly, as XML is, compared to a POJO?
Is HTML really a better way to represent user interfaces than a solution such as the one presented in this article?
It may be hard to answer as we are all biased in favor of HTML.
I'm not experienced enough to make big claims, I'm just thinking out loud, I find this topic interesting. Corrections and feedbacks are welcome. Not trying to start a flame war.
To everyone outside of that bubble, the following is obvious: Web technology sucks. HTML sucks. CSS sucks. Javascript sucks. They're inadequate solutions, which is why people keep reinventing the way to do web frontend every two years.
> It may be hard to answer as we are all biased in favor of HTML.
Not me.
But this problem is not specific to our generation as far as I understand. Maybe the thing that sucks the most that became big in our generation is the Web, but weren't there similar scenarios before us as well?
Doesn't Linux suck? Systemd? Java?
Doesn't every freaking mainstream technology suck? I regularly come across rants from significant developers regarding this issue. There probably is no way to prevent this from happening. Because worse is unfortunately better.
I regularly and seriously consider to leave software development because of this (as I'm really suffering) but then I remember how much power it gives me despite the horrible faults it has.
I think it's very important to understand that the tools we work with mostly suck. In my opinion this is something that is missing from software education curriculums. People need to know and understand how and why everything is messed up and how can we work around this issue while preserving our mental health and the intellectual satisfaction we are supposed to get from the craft.
Web technology is fundamentally different in that you have a big standards body regulating everything from layout down to color blending, then leave it up to multiple vendors to implement all that stuff at great expense and with long delay. Worse, it's all shoehorned on top of some "document model" than was never designed with applications in mind. It's a waste.
To see what a better platform could look like, consider this:
https://mickens.seas.harvard.edu/publications/atlantis-robus...
https://www.youtube.com/watch?v=1uflg7LDmzI
The key here is reducing surface area. It's entirely possible that WASM will bring us closer to that, but the web developers of today probably won't like that.
> I regularly and seriously consider to leave software development because of this (as I'm really suffering) but then I remember how much power it gives me despite the horrible faults it has.
I don't know what you do, but I find web development to be significantly more frustrating than other areas. That doesn't mean there isn't frustration elsewhere though and it's definitely part of the job.
http://homepages.inf.ed.ac.uk/wadler/papers/next700/next700....
1. http://danlec.com/blog/xss-via-a-spoofed-react-element
I used to be a game developer from an OOP background. I find this really hard to read.
https://github.com/thi-ng/umbrella/blob/master/examples/hdom...
Would anyone be able to give 2 cents about where the industry is headed in this regard? Will I be looked down on that my codebase is OOP in the future? Also for those who were cut from my cloth, did you find the transition difficult?
Re: the above example - you didn't really explain which aspects you find alienating, but finding this hard to read, could have to do with the fact that this demo uses a few other concepts & libs not very popular in the JS world (specifically transducers)? I totally grant that without prior knowledge of those constructs this specific code is not the easiest to comprehend (though I've tried to comment every step).
In general though, IMHO this approach is a lot more direct and lightweight than any OOP solution. Having your entire UI defined using native ES6 primitve data structures, provides huge potential and endless options to transform raw app state into UI components. If you ever worked with a Lisp, you'll also find the nested inline definitions not that hard to read. To me it's also more legible than having an (often) over-engineered, multi-layered OOP architecure, riddled with inheritance and so on... Though again, this all is highly subjective. There's certainly ample opportunity to refactor this example into smaller functions and I might as well do that when I get a chance...
All successful mainstream languages have embraced a mix of OOP and FP.
OOP is not a synomim for Java and C++ way of programming with classes.
There are other ways of doing OOP, even all those map/filter/fold patterns were already present in Smalltalk.
I learned OOP with Turbo Pascal 5.5, followed by C++, Clipper, Smalltalk, Eiffel, Oberon, Component Pascal, and many others.
Parallel to that, also SWI Prolog, Lisp and Caml Light as introduction to other paradigms.
It helps to keep an open mind, and approach learning by thinking about paradigms in abstract and not getting too focused on language details.
Don't worry about being looked down on. Worry about improving yourself as our field figures out how we can improve our craft. There's a lot of noise, which makes it hard to tell which things offer the most improvement, but knowledge is never a setback. You will never learn something new and be worse off than you were before. And learning something new doesn't require you to rewrite your codebase to use it, either—of course, you can if you so chose, but that's not a decision you can make until you've grokked the newness ;)
As for the difficulty of the transition: I didn't find it difficult, but I also had the luxury of time. I think it could potentially be difficult for someone thrown into the deep end (e.g., "go into the `hdom-canvas-draw` example and implement new features X, Y, and Z by Thursday!"). People often talk about FP being a different way of thinking about programming. That's true to some extent, but it's not entirely different from what you're used to. I think the thing that seems strangest for newcomers to FP isn't the way of thinking, it's the terminology. Names like `filter`, `map`, `mapcat`, `partition`, &c. seem odd and incomprehensible at first. But they're just operators, like `add`, `multiply`, `insert`, `append`. They're the kind of operator that make functional programming functional: they operate (at least in part) on functions.
If FP is something you want to learn, my advice is to start slow and simple. Some people can probably dive into that example code, tear it apart, figure out how it works, and learn that way. Perhaps I could if I were motivated enough, I don't know. What I do know is that I can't be arsed to do something like that. It's frustrating. Annoying. I have better things to do with my time.
The common advice is Abelson & Sussman's «Structure and Interpretation of Computer Programs», which is pretty good, though it has some problems. Felleisen & al. wrote «How to Design Programs» to address some problems with SICP, and wound up introducing even more problems. The choice between those two boils down to whether you want a book that addresses you like a 13-year-old with a good grasp of the infinitesimal calculus (SICP) or one that addresses you like an 11-year-old that barely understands elementary algebra (HtDP). Personally, I rather like Paulson's «ML for the Working Programmer», which assumes you're an adult who probably doesn't hold a degree in mathematics. Unfortunately, it's not freely available like the other two.
If you decide to go with SICP, I recommend using either MIT Scheme or Racket with sicp mode (you can install the mode through the menus in the DrRacket IDE). For MLftWP, I'd recommend using PolyML (any Standard ML implementation compatible with the revised definition will work fine, but PolyML is actively maintained and easy to set up).
From there, you should be pretty versed in the foundations of FP, and it's just a matter of practice to get comfortable. Of course, there will always be more to learn—the student may never rest :)
ETA - My best advice would be this: if you have the money and the time, read both the Paulson book and SICP; if you have the money but not the time, read the Paulson book; if you don't have the money, read SICP, which you can find in many formats on the Web (it's CC BY-SA licensed). I would not recommend HtDP to a professional programmer, and only maybe to a neophyte.
I found some docs:
http://docs.thi.ng/umbrella/hdom/
https://github.com/thi-ng/umbrella/blob/master/packages/hdom...
https://github.com/thi-ng/umbrella/commits/master/packages/h...
The idea is that all program listings are essentially just lists of things, including other lists: A Javascript source file can be thought of as a list of lines, or more semantically a list of statements and expressions. Function calls are just the name of a function and a list of arguments. Each different kind of statement or expression has its own kind of syntax: a `function(baz) { yadda(baz); }' expression uses different syntax than a `if (something) { yadda(foo); } else { blah(bar); }' statement. S-expression languages just use one kind of syntax for everything, so you can do `(lambda (baz) (yadda baz))' and `(if something (yadda foo) (blah bar))' instead.
This uniformity of syntax allows most of the interesting features of lisps, like that everything is an expression (returns a value), and macros, which are kind of a souped-up version of C preprocessor `#define's.
Particularly like the Estuary and chart samples.
You mentioned you still use ClojureScript - but I'm guessing you're not using CLJS for front-end stuff anymore and just stick to your ES6 libs?
You were one of the reasons why I got interested in Clojure / CLJS. Even visited a workshop of yours a couple of years ago ;) I'm still in dabbling mode unfortunately.
So my understanding after reading your tweets - is that for your use cases ES6/TS is better suited for front-end stuff. But Clojure is still worth digging into - if only for learning the philosophy, principles, ideas and simplicity of FP.
For what use cases would you use ClJ / CLJS nowadays?
Anyway, thanks again for your work - I like the hiccup style of building UIs - so will be checking out hdom.
Edit: typo
Those tweets really weren't meant to discourage people from using CLJ(S) at all!! I was merely trying to explain why, personally, I think TS is better suited for my own use cases (and those of my job)... I'm not building "standard" websites and for the tools I'm dealing with, CPU performance is more important than for most other web apps. I sometimes just felt constrained by some of the features & indirections people usually come to Clojure/Script for in the first place. For example, when generating or processing large amounts of geometry I really don't want or even need immutable data structures by default and so I wanted to have the freedom of being able to break out from this more easily. In some cases I too wanted to have more of a 1:1 correspondence between the code I write and what it ends up as in JS, something the combo of CLJS + GCC made hard(er). ES6 syntax offers many niceties (e.g. array and object destructuring, iterators, async fns) and without those I'd probably be still using CLJS more.
I was in your last Clojure for beginners course. I remember your dog was very friendly and liked to play with the half chewed tennis ball ;)
I really loved your geom library btw, thank you!
https://twitter.com/toxi/status/1041143431786115072
https://twitter.com/toxi/status/1041144813092044800
https://www.reddit.com/r/Clojure/comments/9deyxe/thinggeom_a...
Hth!
I funny enough came at CLJ in part for the math functionality - but I might hit the awkwardness you mentioned. Mostly I'm quite interested in Neanderthal which runs native (MKL) and OpenCL. But living outside of the browser and JS/CLJS presents other limitations (like mixing geom with JavaFX for GUIs has some clunkyness - and the JavaFX wrapper libraries like fnfx are great but not polished)
It's great to see the no-org branch! I love org-mode and use it a lot (for templates and examples) but it always ends up a bit tricky for me to massage the code layout to fit the right narrative structure - so I get why you're moving away to encourage contribution :)
Thanks again for this awesome piece of tech. The API is just so fun to use once you learn how it works (it's so different and ergonomic from anything else I've used in C++/Cinder/etc.), and the end result looks very sexy. I'll keep using it and maybe add some PRs with examples once no-org merges in
- hdom does NOT care about state, where it comes from or how you manage it
- hdom uses vanilla arrays(*) instead of `h()` function wrappers, which not just create more legible (IMHO) style, but also open components up for more options in terms of construction/composition and also transformation.
I think that last aspect is seldom talked about in all those discussions and comparisons w/ React & co. Entire 3rd party frameworks are invented & maintained just to provide augmentation, configuration & injection features for React style components, issues which can be much easier solved if your components are in a less abstract & more native data format already. ES6 provides a ton of nice features to construct and transform arrays without any additional support structures. hdom also supports & fully embraces ES6 iterators and the more modern ways of transforming data they support.
There's a blog post about this forthcoming, so apols for not going into more detail here now...
Looking forward to that post as I’m always happy to be proven wrong!
Win10/Chrome/i7 4790k/GTX 970
https://demo.thi.ng/umbrella/hdom-canvas-shapes/#points-10k https://demo.thi.ng/umbrella/hdom-canvas-shapes/#points-50k
There's also a stress test here which updates several hundred nodes each frame:
https://github.com/thi-ng/umbrella/tree/master/packages/hdom...
E.g. each cell shown requires 3 DOM updates per frame, so in the 256 cells config it updates 768 DOM nodes. On my MBP2016 this config runs at ~35 fps, 192 cells (576 real DOM updates) at 59-60fps