Show HN: A flow based data processing editor

(datablocks.pro)

188 points | by moklick 9 days ago

11 comments

  • moklick 8 days ago
    Hey! this is Moritz. one of the creators of datablocks. the project is in a very early stage and more like a showcase right now. If you have any feedback, please let me know :) It's very valuable for us. We open sourced the underlying library for creating the flows. It's called react flow https://reactflow.dev/. The backend is built with nhost and we can really recommend it -> https://nhost.io/! Thanks for the nice words here :)
    • nefitty 7 days ago
      Well I’ll be damned. I literally just discovered react-flow about two hours ago while spelunking some other app’s source. It has unlocked so many ideas in my head! I’ve been sitting with the whiteboard scribbling stuff to make it with it… it’s gonna be a long night lol
      • mrwnmonm 8 days ago
        I think people will accuse you saying "You stole the name from databricks"
      • dvt 8 days ago
        I'm working on a product that uses react-flow, and I just wanted to thank you for developing that awesome library! As a data professional, I also think Datablocks is pretty amazing. I would only add that you guys should consider adding "Output" blocks: Postgres, BigQuery, Mongo, etc. outputs would be super useful -- extra points for spinning up a temporary DB in the cloud.
        • moklick 8 days ago
          thanks :) database connectors are on our list!
        • iagovar 8 days ago
        • hermitcrab 8 days ago
          Looks slick. We are doing something similar but different with https://www.easydatatransform.com. Although data scientist seem mostly wedded to R and Python, I think no-code graph-based tools have a lot to offer to everyone else.
          • moklick 8 days ago
            looks interesting. Thanks for the link.
          • codeulike 8 days ago
            I guess this isn't an ETL but the look and featureset overlap with the ETL space quite a lot. Hence it would be interesting to know how this compares with the many graphical ETL tools out there, say SSIS, Pentaho, Talend, Stitch, Apache Airflow, Parabola.io
            • ska 8 days ago
              It might have more in common with that old (IBM?) too DataExplorer, later OpenDX iirc.
            • pupdogg 8 days ago
              Very cool! I use vector.dev on daily basis and this feels very much like a GUI version of it.
              • dvaun 8 days ago
                On a side note, thank you for referring to vector.dev. After skimming through their documentation that will be a useful tool as well.
              • nreece 7 days ago
                Pretty cool! I'll try it out in more detail.

                I also liked a similar product, aimed more at ETL/ML/AI segment, that was recently posted on HN: https://hal9.ai

                Question: I added a 'HTTP Request' which returns JSON data, but connecting it to the 'Sort' block shows a warning/error: 'Data input is not valid. Accepted types are: DATASET'. So how do you convert JSON to a dataset?

                • lettergram 8 days ago
                  This looks pretty awesome!

                  Something that might be cool is also identifying the schemas and the ranges. You would probably be able to automatically generate some of these blocks. You could use something like this:

                  https://github.com/capitalone/DataProfiler

                  With enough data, you could probably further automate this process with AI. Good luck! Feel free to reach out for ideas or anything.

                  • Spotted a bug. Using Firefox on Windows.

                    1. Add UFO example block.

                    2. Add Filter

                    3. Set filter options to Year, Equals, 1945

                    4. No results, and the Equals dropdown changes to Please Select.

                    • moklick 7 days ago
                      thanks for the hint! I will have a look.
                    • craig154 7 days ago
                      Thanks for sharing. I found a lot of interesting information here. A really good post, very thankful and helpful.

                      https://www.walgreenslistens.life/

                      • qainsights 8 days ago
                        Looks cool. Let me play with my performance data.