Facebook’s GraphQL gets its own open-source foundation


379 points | by bodecker 221 days ago


  • burtonator 221 days ago

    I did a huge heads down on GraphQL vs AppSync vs Firebase for an app I'm building around document collaboration, annotation and sync for people working with PDFs and web content (https://getpolarized.io/) - it's kind of like an offline web browser.... anyway.

    GraphQL is super awesome at what it does but it's definitely not designed for rapid prototyping applications.

    The thing about GraphQL is that it's middleware. It's designed to act as really nice glue between multiple backends.

    It solves a lot of nice problems like over-fetching too much data, calling too many APIs, etc.

    The problem is that you really don't need these to get an app shipped immediately.

    The REAL sweet spot for GraphQL is for a company like Netflix or Facebook where you have 1500 APIs and tons of problems with data over-fetch and you have the time to sit down and do things right.

    I think I'm going to end up going with Firebase just because you can bang something out FAST and get it shipped.

    It's not going to be perfect but you can ship an MVP and start making revenue and/or grow your user base while you figure things out.

    • 013a 221 days ago

      GraphQL to REST is a more typical comparison. In this case: designing a GraphQL API is substantially easier to both make and consume; while REST just tells you "here's some guidelines, now go do it however you want", GraphQL enforces a much more consistent view of how an API should look, while allowing clients much more freedom in how they get the data they need.

      Where you start running into issues is the surrounding tooling. Integrating a typical REST API into an APM monitoring solution is a cinch, because all of these tools know how to read the incoming requests, HTTP methods, paths, bodies, etc. With GraphQL, you might be left building glue for your APM tool of choice, or just using the highly limited, but at least specialized, Apollo Engine. Enforcing strict rate limiting is easy with REST; very difficult with GraphQL due to how complex and free-form queries are.

      Optimizing your backend to support those free-form queries is also (I dare say intractably) difficult; I haven't seen a single backend framework which doesn't actively encourage an N+1 problem on any query which returns multiple objects of data. AppSync as well, my god is that an evil play from AWS; if you've got separate lambda functions serving all the different nodes in your graph, a single query could trigger dozens, or even hundreds, of invocations. Combine that with their guidance to use Aurora Serverless and any casual observer might say that they're actively exploiting the unfortunate ignorance of an engineer trying to jump on the latest trends.

      I don't believe any of these things are problems with GraphQL. I think they're issues with ecosystem immaturity, and I hope they get better over time. Frankly, every single backend library I've used sucks; its designed to be awesome on the frontend, and it is.

      I think you're right that, right now, its best suited to large organizations. Large organizations can engineer around all of its issues and extract a LOT of value from it. Medium organizations are almost immediately going to run into ecosystem immaturity and scaling issues. Small organizations are going to get the most value from an "all in one" solution, whether that's Firebase, or a simple REST API on App Engine, or something like that.

      But I could be wrong in my analysis that its not a core issue with GraphQL, and there are subtle complexities with the API definition language which make scaling it for anyone who isn't Facebook intractable. Time will tell.

      • ruslan_talpa 221 days ago

        > I haven't seen a single backend framework which doesn't actively encourage an N+1 problem on any query which returns multiple objects of data

        Well then you haven't really looked :)





        These are all examples of tools/libs that implement a graphql api without a N+1 issue

        • 013a 220 days ago

          Those are great resources. But all of them appear to be tightly coupled with a SQL storage backend. That's valuable, but what I believe the ecosystem needs is a generic system which can "pre-compile" a GraphQL query into a backend language data structure which the developer can then transpose into a database query.

          I think Facebook's Dataloader is more close to a solution.

          • BenjieGillam 220 days ago

            PostGraphile is built on Graphile Engine which is completely backend-independent (like GraphQL itself). This is what we use in PostGraphile for the "look-ahead" functionality, allowing you to build a database/API/backend query that represents a full GraphQL query as one action. You can read more about it here, though it's currently not as well documented as PostGraphile itself. https://www.graphile.org/graphile-build/

            • macca321 220 days ago

              Almost like a Language INtegrated Query (LINQ) in C#?

              OData did exactly this years ago, and got heavily criticised for giving too much querying power to the client side, as (without expert usage on the server side to whitelist the query) a client could run queries that would overload the server.

              You're damned if you do, and damned if you don't.

            • tirumaraiselvan 221 days ago

              We at Hasura[0] use a GraphQL to SQL compiler approach[1] so its just ONE query no matter the size of the Graphql query.

              [0] https://hasura.io [1] https://blog.hasura.io/architecture-of-a-high-performance-gr...

            • hakanito 221 days ago

              Facebook released DataLoader [1] to help reduce the number of database invocations. As long as you write your queries to fit the loader pattern it's pretty straight forward. Been trying out a golang port [2] for a month or two and it really (really!) improves performance.

              [1] https://github.com/facebook/dataloader [2] https://github.com/graph-gophers/dataloader

            • cheriot 221 days ago

              GraphQL shifts complexity to the server instead of the client. That can be an advantage when there's multiple clients or just one client iterating faster than the underlying logic. There's certainly a short term cost to GraphQL compared to REST, but there's a lot of use cases with positive ROI outside FANGs.

              When read and write becomes real time sync, though, I've always thought Firebase was under appreciated.

              • vlozko 220 days ago

                I'm not entirely sure I agree that it shifts complexity. I think there's more complexity on both the client and the server end. I work extensively with GraphQL on the client end (iOS) and there's certainly more work involved compared to a REST endpoint for more larger applications. Notably:

                * The mapping of the response to the types used in your app balloons. There's heavy use of Swift's Codable to map the JSON result to objects. I'm finding in a lot of cases where I'd be making queries and I don't need the resultant root object but rather a value one or two levels deeper. This has caused me to write a number of "shell" types to help streamline the decoding process.

                * Different GraphQL requests can query for different fields on the same type, thus forcing your app to have to different types for arguably the same thing or have optional fields in more places that I would like.

                * There's security implications with exposing your backend to any kind of request. GraphQL supports hashed queries so that the entire request isn't sent over each time and prevent the abuse that can result from an exposed API. Setting up and supporting this infrastructure takes some amount of resources.

                * More complex to provide response metadata, such as cache control and request/response IDs. Granted, some of this could/should be moved to the response header but more complex types are trickier to handle.

                That all being said, I'm quite happy with the trade offs compared to using REST, including:

                * With REST, some of the endpoints ended up returning massive results as they had to support Android, iOS, and Web use cases. It's really hard to audit what fields were still in use by the app and the ROI on cleaning up the endpoints was minimal.

                * Related to the above, it's easier to deprecate certain fields and make the changes on all the platforms appropriately. Given that GraphQL supports tracing of queries/fields usage, it's a lot easier to know when a field is no longer in use and be able to clean it up. Granted, this is more of a backend plus vs a client but it provides a much more smoother migration process for the client.

                * Explicit declaration of non-null fields. Fantastic for mapping types. The entire GraphQL query fails if a resolver returns null for a non-null-defined field, giving the app a piece of mind with regards to type safety.

                I agree there's definitely a short-term cost but a big ROI.

              • jchb 221 days ago

                > tons of problems with data over-fetch

                What is the GraphGL story on caching and closest point of presence redirection? We build a mobile app that consumes various "enterprisy" HTTP-based APIs. Often, due to how the APIs are designed to support a range of different frontends, we have to either fetch more data than we need, or do a bunch of granular requests where we would prefer to do a single large one. But most of the time that is outweighed by the fact that many responses are cached in CDN (Content Delivery Network). Since our users are spread out globally, going to the origin server for every response would in many cases imply a latency of 100-200 milliseconds, which wouldn't be acceptable.

                • krschultz 221 days ago

                  It depends, the GraphQL spec isn't all that opinionated on caching. There's a little bit of guidance on the matter but not much. [1] The implementation of GraphQL that you choose might be more "batteries included" though.

                  [1] https://graphql.org/learn/caching/

                • baby 221 days ago

                  Agree with you, for small apps it looks like this is a lot of effort for little benefits. For large companies that deal with huge APIs, maybe it's good.

                  • cdaringe 221 days ago

                    Hard disagree for many cases. GraphQL APIs are free with db reflection tools--ga ga ga ga GREAT for prototyping. Shoutout to postgraphile.

                  • hamandcheese 221 days ago

                    GraphQL is not comparable to Firebase, it is comparable to implementing your own REST/gRPC/Thrift API.

                    When comparing apples to apples, GraphQL is amazing for rapidly iterating.

                    I think the biggest risk with GraphQL is it’s too easy to just mirror your data structures as an API.

                    Also, unrelated to the above, but we use Firebase for a few auxiliary real-time needs at work and it goes down constantly.

                    • he0001 221 days ago

                      >When comparing apples to apples, GraphQL is amazing for rapidly iterating.

                      Rapidly iterating what?

                      • hamandcheese 220 days ago

                        Mostly the frontend. I’m comparing to a React+Redux+REST app where for every new resource you have to:

                        - create a new action - turn that action into requests via some middleware - denornalize and put in your store - maybe write a selector to get the data you need from your store

                    • k__ 221 days ago

                      Did you use AWS Amplify?

                      I had the impression it was like Firebase for quick prototypes (saw someone build a product in only 4h once), but you get CloudFormation templates out of it, which makes it much more flexible in the long run, when you customize more and more.

                      • ivanhoe 221 days ago

                        Not necessarily, on small projects that are being rapidly developed it's very common that you need to change many times the data structures returned by api, as front-end is iterating on the design and ideas and figures out the new pieces of data that they need. Mapping the results of DB queries into complex structures & entities, and juggling it to fit the latest needs of front-end/app devs used to take a big chunk of my back-end dev time (and nerves). GraphQL can extremely simplify and speed up these iterations.

                        • davidjnelson 220 days ago

                          You may also enjoy heroku for hosting an mvp. Pricy, but straightforward to scale and set up a staging env for testing release candidates on, plus lots of great features like automatic db backups. And since it’s built on aws it’s easier to shift some microservices onto ec2 later as needed.

                          • alwaysreading 221 days ago

                            Totally agree. GraphQL makes perfect sense for a large app with many endpoints but there isn’t a lot of benefit for smaller apps. I’m involved in an app now that “has to use GraphQL” but could easily be done with a restful api without the additional overhead of what is essentially middleware.

                            • adamnemecek 221 days ago

                              I think it’s dope if you are fundamentally querying a graph. Doing it manually is repetitive.

                            • orta 221 days ago

                              As someone who knows a few of the folks involved in this foundation, I think this is a real net positive for everyone who builds GraphQL APIs and is involved in improving the ecosystem.

                              • xrd 221 days ago

                                I remember when I was at RealNetworks a long time ago and Microsoft and Real were competing over the future of multimedia languages (SMIL vs HTML+Time). I was on some of the standardization committees and remember being surprised when some of my colleagues suggested that standardization was used by companies like MS to slow down the innovation process (and catch up to faster starts like RealNetworks had made). I wonder how often open source foundations serve the same purpose, or if these are truly embraced by companies like Facebook? Is there a reason internally Facebook would have fought something like this? Is it all roses?

                                • vjeux 221 days ago

                                  Lee Byron who was one of the founders of GraphQL and has been actively driving its success in Open Source left Facebook recently (after a very long tenure). Making GraphQL part of a foundation is likely a way for him to be able to keep stewarding the project. He wrote a post with a bit more context: https://medium.com/@leeb/introducing-the-graphql-foundation-...

                                  • jahewson 221 days ago

                                    It’s common knowledge in the community that GraphQL already covers Facebook’s use cases and they’re wanting others to take up the reigns for new features, e.g. Apollo. A foundation puts all contributors on an even footing.

                                    • joshred 221 days ago

                                      There's a little of column A, a little of column B.

                                      Standards allow the information economy to function... but influential companies want their internal practices to become 'the standard'.

                                      • k__ 221 days ago

                                        It probably isn't all roses, but I have the feeling people today are more prone to "fork" if they feel belittled, because of the omnipresence of the Internet they feel less weak against big companies.

                                        • syn0byte 221 days ago

                                          Generalization of Hanlen's razor: Never attribute to malice what can be explained by laziness/greed/stupidity.

                                          FB has GraphQL stable and doing exactly what they want it to do. Now they can pass it off to a foundation for maintenance and blame without having to pay out of pocket.

                                          • krschultz 221 days ago

                                            GraphQL isn't a system per-say, it's a spec and many tools around it. Putting the spec in a foundation (with a few of those tools) gives the entire community certainty about the direction of the project. Hopefully that spurs more people outside of FB to commit to GraphQL.

                                            The existence of the foundation is orthogonal to FB's internal investment and usage of GraphQL, which was and will continue to be significant.

                                            • chrisweekly 221 days ago

                                              >"per-say" -> "per se"

                                              Right pronunciation, but it's Latin. :)

                                              • krschultz 221 days ago

                                                Thanks! At least I learned something today.

                                                • tannhaeuser 221 days ago

                                                  per se = "in and by itself"

                                                  Having given tutoring in Latin when I was 15, I'm always dying a little inside if I'm reading "per say".

                                            • superflyguy 221 days ago

                                              "Generalization of Hanlen's razor: Never attribute to malice what can be explained by laziness/greed/stupidity."

                                              I think OP was going for greed.

                                              • karmakaze 221 days ago

                                                That seems an over-generalization--greed should counts as malice in spirit. [nit: Hanlon's Razor]

                                          • mohsen1 221 days ago

                                            Both Swagger (Open API Spec) and GraphQL are now part of the Linux Foundation! That's cool!

                                            • jondubois 221 days ago

                                              I'm not a big fan of GraphQL. Its main benefit is for serverless which I am also not a fan of. It shifts all the control to the client and makes the server generic which actually takes away flexibility overall.

                                              • LeicaLatte 221 days ago

                                                Are there good alternatives to Apollo at all?

                                              • tw1010 221 days ago

                                                Oh great, an ad that covers the entire page on mobile. Guess I'm not reading this article then.

                                              • nikhilsimha 221 days ago

                                                About time.

                                                • beders 221 days ago


                                                  • leddt 221 days ago


                                                    • mlevental 221 days ago


                                                      • andrewingram 221 days ago

                                                        It's a pretty simple library, does it need commits every month to prove that you should still use it?

                                                        • mlevental 221 days ago

                                                          code rot is a thing

                                                          • andrewingram 220 days ago

                                                            It is, but it’s not a universal rule. In the 2+ years I actively used Dataloader (not using GraphQL-JS right now), I had one feature request which got implemented, but other than that it was perfectly stable and fit for use.

                                                  • degyves 221 days ago

                                                    Great. Another solution for an already-solved, already-standardized query API language (SPARQL).

                                                    • i_phish_cats 221 days ago

                                                      SPARQL demands a certain data model (triple stores). Also, everything about it smells like ivory-tower academic snobbery + ultra-large-enterprise crap. GraphQL does neither.

                                                      • degyves 221 days ago

                                                        Funny that ultra-large-enterprise crap could be applied to Facebook and GraphQL?

                                                        • degyves 221 days ago

                                                          GraphQL also demands a certain data model. Just that it is not standardized. And no, RDF/XML is not the only representation for SPARQL. It can deal also with JSON.

                                                          • nevir 221 days ago

                                                            GraphQL has no demands on your data model. You can do whatever you want behind the schema you provide to client apps

                                                            • degyves 221 days ago

                                                              I think you should read it directly from the website: "GraphQL is a query language for your API, and a server-side runtime for executing queries by using a type system you define for your data." You need a type system to bypass the datamodel. Just pushing the issue around.

                                                            • i_phish_cats 221 days ago

                                                              No, it doesn't, it is agnostic. There's an adapter for every major database out there. Oh, yeah, graphql also has momentum and strong adoption, unlike SPARQL.

                                                              • degyves 221 days ago

                                                                Oh yeah, GraphQL is backed by Facebook (may that be a hint of its momentum, like its support for React?) , unlike SPARQL which is a _standard_. You can also convert from PostgreSQL, MySQL, DB/2 and others to SPARQL. Still, only difference seems 1. Facebook pushing its non-standardized tech around and 2. Losing any benefits from HATEOAS from the usage of GraphQL.

                                                                • true_religion 221 days ago

                                                                  Sure but I use GraphQL as middleware when fetching from elasticsearch, or redis, or flat files. It is not something you hook up directly to your database, it's a way to help define an api for your application.

                                                                  Most of the time, in my app, resource names do not even correspond to single tables or backend data structures but are simply presented that way for ease of use for external consumers.

                                                                  Subjectively, Implementing it feels more like old school COBRA than REST or SQL.

                                                          • linkmotif 221 days ago

                                                            You’re getting caught up in the “graph” part. GraphQL is unfortunately named, because it doesn’t actually have any “graph” related semantics or features in the graph database sense. The “graph” in GraphQL just refers to the fact that you can use it to retrieve nested data. On the other hand, SPARQL and Tinkerpop (the “true GraphQL” imo) are actually about graphs and related data in the graph database/semantic web sense. GraphQL and SPARQL have nothing to do with each other.

                                                            • magic_quotes 221 days ago

                                                              I'm pretty sure the "Graph" part refers to the Facebook's social graph. They already have (REST'ish RPC) Graph API, so there is a precedent here.

                                                            • hn_throwaway_99 221 days ago

                                                              GraphQL is not a generic query API language. One of the most unfortunate things about GraphQL is the presence of "QL" in the name makes people think it is somehow comparable to SQL or SPARQL.

                                                              • degyves 221 days ago

                                                                Maybe because it IS a query language? From the website: "GraphQL is a query language for your API"... just that it is _not_ a standardized API, so we won't take advantage of HATEOAS.

                                                              • marktani 221 days ago

                                                                I recommend this talk to see what both technologies can still "learn" from each other - https://youtu.be/LUF7plExdv8. I don't think SPARQL or GraphQL completely replace one another.

                                                                • degyves 221 days ago

                                                                  I think GraphQL is just an way to bully the efforts of the W3C to standardize web technologies. It not just is non-standard, also is breaks any advantage of the HATEOAS principle.

                                                                  • halostatue 221 days ago

                                                                    I have written HATEOAS APIs (not just “RESTish” APIs). I prefer writing GraphQL APIs by far.

                                                                    If I have to write something that’s REST-ish, I’ll do my damnedest to make sure it’s HATEOAS, but in my experience, the number of developers who can consistently produce useful and consistent HATEOAS APIs is…vanishingly small.

                                                                    I can learn more about an API from its GraphQL schema than most of the other API documentation (like OAS/Swagger or RAML) out there.

                                                                    • tjpnz 221 days ago

                                                                      >the number of developers who can consistently produce useful and consistent HATEOAS APIs is…vanishingly small.

                                                                      Because relatively few developers truly understand HATEOAS let alone REST. I think that did play a small part in the increased popularity of GraphQL.

                                                                      • degyves 221 days ago

                                                                        Yeah, if you are writing HATEOAS APIs just by personal preference then you are using the wrong technology. You should write HATEOAS to make sure your API will be consistent, standard and automatically browsable on the long term.

                                                                  • lsh 221 days ago

                                                                    Fellow curmudgeon, I give you an upvote. SPARQL is a more elegant tool from a more civilised time.

                                                                    • rboyd 221 days ago

                                                                      that's awesome -- y'all are like the tech version of the Mad Max War Boys. sacrificed yourselves betting on the wrong tech.

                                                                      witness me!

                                                                      • degyves 221 days ago

                                                                        Can you elaborate why SPARQL is the wrong tech?

                                                                        • exogen 221 days ago

                                                                          As someone who used to write SPARQL queries for a living with some of the most established players in the Semantic Web game, the idea of SPARQL and GraphQL serving the same purpose is pure misconception (usually by folks who haven't yet taken the dive into GraphQL).

                                                                          GraphQL allows the backend to say "here's the data I offer and how it's structured, pick which fields you want from that" and the frontend query just specifies the "what." It's like picking from a menu (although fields can have parameters), and the results you get back are structured objects with nesting.

                                                                          On the other hand, the data offered by a SPARQL endpoint has no inherent structure, the query is what gives the data structure. You not only ask what you want (the projection) but how to resolve the fields using (often complex) relational logic. It's orders of magnitude more powerful, which is awesome, but also more work and more confusing. Its biggest downside is that, like SQL, the results are row-oriented. For anything other than completely simplistic queries, the resulting data often needs to go through a transform/reducer step instead of being used directly, because real-world needs aren't always row-oriented (especially on the frontend). If you want to request information about both entities and the "children" of those entities, the projection is going to be (1) incredibly messy and (2) full of duplicate data, or you're making multiple queries.

                                                                          I wouldn't want my frontend communicating with a raw SPARQL endpoint for the same reason I wouldn't want to expose a raw SQL endpoint (and basically nobody does). GraphQL puts the frontend "on rails" so to speak and lets the backend worry about the "how." That can be a positive or a negative depending on what you want to do, but in the frontend world (where REST is the norm), folks consider that a positive and it's the direction they've generally chosen.

                                                                          • degyves 220 days ago

                                                                            SPARQL results DO have an inherent structure. Even more, this is why projects like Eclipse Lyo can automatically convert SPARQL results directly to Java Objects. So no, SPARQL it is not intended to be used directly but as a product for a library to be converted into objects or rows or graphs or anything the client needs.

                                                                            On the other hand, GraphQL pushes its complexity (to worry about the "how") into the server's implementation. The complexity is still there. And it's impossible to be automatically browseable.

                                                                            Semantic Web is only an application of SPARQL, not the only one.

                                                                            • dalyons 221 days ago

                                                                              Very well articulated!

                                                                            • Meic 221 days ago

                                                                              I never met a SPARQL endpoint I couldn't crash with a trivial query.

                                                                            • lsh 221 days ago

                                                                              ... ?

                                                                          • atombender 221 days ago

                                                                            GraphQL isn't a query language, it's an API protocol language. Big difference.

                                                                            • aikah 221 days ago

                                                                              Well it is a query language, QL stands for query language.

                                                                              • atombender 221 days ago

                                                                                The name doesn't make it so. It defines a syntax and a single operator (nesting), but nothing else. You can't use it to "query" arbitrary data sources in the sense that other query languages such as SQL do, since there are no operators, functions, subqueries/arbitrary joins etc. It's an alternative to REST, not SQL.

                                                                            • degyves 221 days ago

                                                                              From the website: "GraphQL is a query language for your API"