Hey HN, this is a side project that we made after realizing that a solid chunk of the time and code spent doing full-stack development was spent writing and debugging RPC layers - REST, GraphQL, GRPC, you name it.
The fact that the frontend and the backend are separate services leads to a bunch of cruft (https://thume.ca/2020/05/17/pipes-kill-productivity/ is a perfect description) and security holes, so we had the idea of merging them. It turns out that Phoenix LiveView already did server-side execution with changes streamed over WebSockets, but we weren't the biggest fans of having to learn an entirely new ecosystem and paradigm. So, we built a framework that allows (at the moment, admittedly simple) vanilla React apps to run on Node.js, with DOM updates being streamed to the client. That way, we can delete the RPC layer and greatly reduce development complexity and total code.
I had this same inspiration from Chris McCord's talk about LiveView ~1.5 years back, and created Purview: https://github.com/karthikv/purview - we've been using it in production at the company I work at for over a year now.
It's great to see a similar mindset here, and I hope an approach like this becomes more widely used. I appreciate how Caldera is built using React's reconciler and is centered around hooks, both of which are different than the approach Purview takes.
Two of the challenges we've experienced here:
- Transferring server-side state between processes: in production, you'll likely have multiple Node processes handling requests via the cluster module or other load balancing. If a client were to disconnect their websocket temporarily (e.g. put their laptop to sleep), they can be reconnected to a different process, which doesn't have the state of their components. As a result, you'll need some way to transfer/persist component state between processes. This same issue will also arise during deploys--you'll spawn new processes with your new code, but you don't want to disrupt existing clients, so those new processes need to load existing component state from old processes. Even more subtly, this issue can occur on page load, since two requests need to be made: one to fetch the initial page and one to initiate the websocket connection. Those two requests can be sent to two different processes/boxes.
- Working with existing client-side libraries, like selectize or date range pickers. We've opted to use the Web Component spec to create custom tags that can be sent down just like normal tags from the server, but making sure these interoperate nicely with server-side state is challenging.
Very excited to see how Caldera tackles these problems!
Great minds think alike! I had a feeling this was a relatively straightforward insight, and Purview looks like a pretty cool implementation.
To answer your questions:
1. We have a mechanism to serialize the React state tree, as long as it can be copied with the HTML structured clone algorithm (https://github.com/calderajs/caldera-react/blob/master/src/s...) which has always seemed to be the case in practice. Each tree is associated with a unique token which is stored on the client, but the current storage implementation is in-memory only as we haven't addressed the application upgrade issue. Outside of that, there's nothing stopping us from adding basic versioning (along with react-hot-loader-like reconciliation behavior) and moving the state storage to a database that is shared among application servers.
2. This isn't really fleshed out yet, but we'd likely take inspiration from React Native's native module approach where lifecycle events for mounting/unmounting/updating a component are implemented on the client, with bindings being called in the server application code (a React-specific version of your web component implementation). I know we're against high-level RPC, but React props naturally have a data/async callback structure that is amenable to being represented as RPC calls (which works for React Native), which is why we're leaning towards that approach.
For the first problem what about the option of making the session:backend-instance mapping sticky? Seems it could be simpler than the implementing the state handover.
Can you add a flappy bird demo? I find that flappy bird demos for state-over-wire tech like LiveView and Blazor to be a great gauge for performance and latency. If there is the slightest delay or inconsistency in performance you will be able to tell immediately.
I can't tell if you're joking, but just in case you're serious... the only thing benchmarking flappy bird demos tells you is how fast your internet connection is.
I am preeminently qualified to beg people to judge frameworks on API surface and architecture because I have written all of the so-called hi-perf frame rate demos for StimulusReflex and one of the main reasons we took them down is because it is only a distraction from the incredible force for good these libraries actually are.
How's this for a demo: when you embrace the Rails + StimulusReflex + Turbolinks stack, you don't need to introduce React in 90% of the scenarios it's improperly used today. You end up with UIs that run faster, are smaller over the wire, drastically simpler and more fun to build and easier for newcomers to contribute to.
I'm surprised (and happy) that you added ts-liveview. I'm working on 1.0 moving to use primus.js instead of raw websocket for better (dis)connection handling, and trying to make the integration of s-js less noisy.
one of the things I worry about other platforms attempting to do phoenix LiveView is that the BEAM vm gives you process isolation. If there's a miscode, then the process crashes and you're all good. How does JS handle such a situation, and does it result in a malicious attack vector?
We are able to catch errors in the React tree (which is per-client) and restrict the crash to the specific client instance. Unfortunately, we haven't gotten around to implementing it yet
That's exactly my starting point to work on ts-liveview. I want to have the runtime property of phoenix liveview (being lightweight and deliver first meaningful paint ASAP) while enjoying the type-checked dev-experience.
Erlang, Elixir, and Lisp(s) show great concepts but missing dev-time type checking (from IDE and compiler) is my road blocker.
Edit: I know there are typed-racket and dialyzer for erlang but my experience with they are not reactive as I do dev / refactoring with TS so far.
One of the contributors here - not entirely sure if you're asking for pros or cons, but the entire point of Caldera is to execute/keep the React code on the server and only stream DOM updates. It's set up such that existing React web apps can be easily ported over (essentially just move all client logic to the server), so you can definitely make normal web apps with the framework! If you're asking about drawbacks, one of the biggest issues is latency, since events need to be sent to the server to be processed, but this can be relatively easily alleviated by hosting servers closer to clients.
The fact that the frontend and the backend are separate services leads to a bunch of cruft (https://thume.ca/2020/05/17/pipes-kill-productivity/ is a perfect description) and security holes, so we had the idea of merging them. It turns out that Phoenix LiveView already did server-side execution with changes streamed over WebSockets, but we weren't the biggest fans of having to learn an entirely new ecosystem and paradigm. So, we built a framework that allows (at the moment, admittedly simple) vanilla React apps to run on Node.js, with DOM updates being streamed to the client. That way, we can delete the RPC layer and greatly reduce development complexity and total code.
Would love to hear your thoughts!
It's great to see a similar mindset here, and I hope an approach like this becomes more widely used. I appreciate how Caldera is built using React's reconciler and is centered around hooks, both of which are different than the approach Purview takes.
Two of the challenges we've experienced here:
- Transferring server-side state between processes: in production, you'll likely have multiple Node processes handling requests via the cluster module or other load balancing. If a client were to disconnect their websocket temporarily (e.g. put their laptop to sleep), they can be reconnected to a different process, which doesn't have the state of their components. As a result, you'll need some way to transfer/persist component state between processes. This same issue will also arise during deploys--you'll spawn new processes with your new code, but you don't want to disrupt existing clients, so those new processes need to load existing component state from old processes. Even more subtly, this issue can occur on page load, since two requests need to be made: one to fetch the initial page and one to initiate the websocket connection. Those two requests can be sent to two different processes/boxes.
- Working with existing client-side libraries, like selectize or date range pickers. We've opted to use the Web Component spec to create custom tags that can be sent down just like normal tags from the server, but making sure these interoperate nicely with server-side state is challenging.
Very excited to see how Caldera tackles these problems!
To answer your questions:
1. We have a mechanism to serialize the React state tree, as long as it can be copied with the HTML structured clone algorithm (https://github.com/calderajs/caldera-react/blob/master/src/s...) which has always seemed to be the case in practice. Each tree is associated with a unique token which is stored on the client, but the current storage implementation is in-memory only as we haven't addressed the application upgrade issue. Outside of that, there's nothing stopping us from adding basic versioning (along with react-hot-loader-like reconciliation behavior) and moving the state storage to a database that is shared among application servers.
2. This isn't really fleshed out yet, but we'd likely take inspiration from React Native's native module approach where lifecycle events for mounting/unmounting/updating a component are implemented on the client, with bindings being called in the server application code (a React-specific version of your web component implementation). I know we're against high-level RPC, but React props naturally have a data/async callback structure that is amenable to being represented as RPC calls (which works for React Native), which is why we're leaning towards that approach.
I have a longer list of such technologies here: https://news.ycombinator.com/item?id=19716696
I am preeminently qualified to beg people to judge frameworks on API surface and architecture because I have written all of the so-called hi-perf frame rate demos for StimulusReflex and one of the main reasons we took them down is because it is only a distraction from the incredible force for good these libraries actually are.
How's this for a demo: when you embrace the Rails + StimulusReflex + Turbolinks stack, you don't need to introduce React in 90% of the scenarios it's improperly used today. You end up with UIs that run faster, are smaller over the wire, drastically simpler and more fun to build and easier for newcomers to contribute to.
https://laravel-livewire.com/
Check out the console for DOM updates streamed from the server. React is running on Node.
[1] https://en.wikipedia.org/wiki/Caldera_OpenLinux
Although Phoenix is great, it's good to have similar things on Node in pursuit of velocity.
Erlang, Elixir, and Lisp(s) show great concepts but missing dev-time type checking (from IDE and compiler) is my road blocker.
Edit: I know there are typed-racket and dialyzer for erlang but my experience with they are not reactive as I do dev / refactoring with TS so far.