Very cool and lightweight way to talk to PlanetScale but for now I'll stick with Prisma. Prisma is much heavier (engine weighs in at ~50MB) and that can be a non-starter for serverless in some cases but it works for me on AWS Lambda.
The nice thing about PlanetScale is you get nearly unlimited connections (soft limit of like 250K IIRC) so making 1 connection per active lambda isn't a problem at all.
I've been using PlanetScale since shortly after they went GA and I've been very happy so far. Cheaper than Aurora Serverless, less hassle, and the branching feature is super cool. Zero-downtime deploys, with rollback support, feel magical.
> This specifically is targeting environments where a MySQL client isn't able to run.
Oh, absolutely and I didn't mean to imply it wasn't useful, I was just saying I went down the Prisma path and will be sticking with that. Even so I'm glad this exists for times that I don't need the full weight/power of Prisma but do want to talk to a PlanetScale DB.
I went down the Prisma path investigating JavaScript ORMs, and... I have opinions. :)
I feel that what we have would be useful within a Prisma context, but given their complexity and how Prisma works, it's likely not very practical with their "engine".
That’s fair. They are the only game in town, last I checked, that has full (good) TypeScript support which is a must-have for me. TypeORM is the other I know about and I’ve used it as well but I greatly prefer Prisma.
Other than having to bundle the engine I’ve been very happy with Prisma + PlanetScale. I’m open to any other TypeScript ORMs if you know of them.
I've also been following Kysely and am a big fan, having used Objection.js and Knex a lot in the past. I consider Kysely to be the TypeScript-native spiritual successor to Knex.
For a theoretical new Node + TypeScript + SQL project, the three options I would consider are, from most-ORMy to least:
- Prisma
- Kysely + kysely-codegen (query builder)
- slonik + @slonik/typegen (raw postgres driver with good types; not sure if there's a good MySQL equivalent)
I would not consider myself a member of the JavaScript or TypeScript communities. I just build the infrastructure and platforms to support these kinds of things. :)
You can learn more about the zero-downtime deploys here [0] which also covers the rollback support ("Rewind"). You can also read more about Rewind in this blog post [1]
They say in the post to use environment variables when deploying. I think they’re talking about server-side JS, not JS running in an end-user’s browser.
This is great! Potentially the missing piece in the world of "serverless": a serverless SQL database accessible over HTTP with no minimum price. I saw AWS Aurora has a Data API but currently that does have a minimum monthly cost. Fauna is possibly the closest, but that appears to involve translating SQL to its FQL. How stable is the beta?
It's 90% there... but not allowing foreign keys [1] (=cascading delete) means that this solution becomes equal with DynamoDB (Serverless + Scale to 0 as well) because products will either be built for it from day one or never support it with a full fledged DB server instance that is 100% up in mind. Unfortunately, didn't find a lot in-between.
I like foreign key constraints as much as the next guy, but they are hardly the only difference between MySQL and DynamoDB. For example, you should be able to use Rails/Django/Laravel without foreign key constraints, you can use off the shelf SQL clients or BI clients, etc.
I mean I'm using PlanetScale with Prisma ORM on AWS Lambda right now without issue. Unlike many DB providers, PlanetScale supports nearly unlimited connections. I think I saw in their docs they have a soft-limit of something like 250K connections.
I used Aurora Data API before moving off Aurora Serverless (insane pricing) to Prisma and PlanetScale, I don't think I'd go back to the HTTP API as Prisma works very well and I enjoy using it. One downside to Prisma is the DB "engine" is a arch-specific (obviously) binary that is pretty hefty. I want to say ~50MB. That can be a killer in serverless but I was able to work around it without much issue. If I ever wanted to dive into the world of Lambda layers I could probably move it into it's own later (however that still counts towards your max lambda size).
Yeah, I think this new thing would be less useful for Lambda as that does support TCP connections. However a HTTP API is required for e.g Cloudflare Workers where you can't create a normal MySQL client. I think that's where this could shine.
We are hearing from some users that the performance with Lambda is really good with the new driver. We are going to do some comparisons soon to see if there is more worth sharing on it too. We are going to do a Twitch stream adding it to a Lambda on Tuesday at 11am CT if you are interested. https://www.twitch.tv/planetscale
Our beta for this is pretty stable. The only real instabilities are around the underlying APIs we use, which is part of why we're not ready to document it just yet.
But the underlying tech is exactly the same as we use for handling traditional MySQL connections, so there isn't anything to fear.
That's fair and since my use of PlanetScale is very burst-y I do wish there was a cheaper option for the vast majority of the time when I don't need to scale but at $30/mo it's much cheaper than Aurora Serverless and I'm not aware of many other "serverless" db's, let alone cheaper ones (that are still MySQL).
This is huge for me. I went down the path of trying to build an entire API on CF Workers and the biggest stumbling block was no easy access to external relational databases due to a lack of v8 compatible connectors. This was before the D1 announcement of course.
They still require external connections to be made over HTTP and not other networking protocols. This is part of why this new driver is useful. Before today, it would have been impossible to use PlanetScale directly from a worker without it.
To my knowledge you can use planetscale’s read replicas in different regions which the user is then routed to the closest one? I was going to use fly.io until I realized PlanetScale does this already - or do you mean writes as well?
Please consider using a custom string interpolation function to sanitize SQL, not an additional array of parameters. See styled components for inspiration.
Great question! Author of a lot of the infrastructure surrounding this here.
I plan on following up with a more technical deep dive on some of these aspects, but it's quite a bit hard to do a 1:1 comparison.
While, obviously, HTTP also uses TCP, I'm going to assume you're asking more about the binary MySQL protocol vs HTTP.
On the surface, yes, HTTP is going to have more overhead with headers and the other aspects that come with HTTP. Also, in this case, the payload is JSON, both on the request and response, which is going to be slightly more bulky than the payloads over the MySQL protocol.
So where things get interesting is the real world performance implications. Networks and CPUs are really fast. HTTP has been extremely scrutinized and JSON as well. While they are admittedly not the most efficient standards, they are used so heavily and parsers are heavily optimized for these protocols.
Mixing in modern TLS 1.3 with modern ciphers, which is something you're very unlikely to get with a traditional MySQL client, we can achieve a much faster first connection time. Pairing with modern ciphers that are demanded for TLS 1.3, and the transport itself can be significantly faster than a slower cipher. This isn't the best comparison since typically you're using MySQL without TLS, but when talking to a service provider like us, we require it for obvious reasons.
Next, with HTTP, we get to leverage compression. In our case, we can use browser level native compression with gzip, or if using something server side, we support compressions like snappy. Combine something like gzip or snappy with HTTP/2 now, and given that the bulk of a query result is typically the result itself, and not the protocol, compression can make up a decent amount of the difference. Again, I'm being hand wavy since every query pattern and results are going to be wildly different based on your data, so it's not anything fair to say "this is x% more or less efficient."
And lastly, with HTTP, especially HTTP/2, we can multiplex connections. So similar to the gains with TLS 1.3, you can do many many concurrent database sessions across 1 actual TCP connection with HTTP/2. And similarly to TLS 1.3, the cost here is in connection timings, and management of a connection pool. You don't need a pool of connections on the client, so your startup times can be reduced.
As a stretch goal, HTTP/3 (with QUIC) is in the crosshairs, which should eliminate some more transport cost since it uses UDP rather than TCP. My hunch is all of this combined together, an optimized driver leveraging HTTP/3 might beat out a MySQL client overall. Time will tell though!
So there's no simple answer here, everything has tradeoffs. :)
There are, in my opinion, a few other concrete benefits outside of performance too that I didn't touch on. I plan on touching on them more in my follow up blog post though.
A lot of this for me/us at this point is still theoretical since we don't have tons of application yet. Serverless was an easy first target since they require HTTP as the transport. The real experiments will come when we say, put together a Python or Go driver based on these APIs and compare side by side to a native MySQL driver.
Do you think, if this becomes the norm, that we could start seeing no-backend apps (thick client talking directly to DB)? Maybe with the occasional light worker/lambda
I'm going to use a big notice here: None of this is inherently safe to use as a client. This is similar security profile as a database driver. This API has no row level auth or anything like that. Unless this is explicitly what your product is doing, like within PlanetScale, this is what powers our Console, don't use this in a browser.
With that said, I personally have no real opinions on this topic. It's pretty far out of my wheelhouse. I think that's more something like Firebase and gang, but that's not something we're gunning for at the moment and aren't trying to replace that.
That's awesome, I was assuming a sizable overhead (even if it is for a sizable benefit). But I can see how lots of factors contribute to closing the gap, and HTTP/3 might even be a negative overhead since it almost cuts out a protocol layer by stepping down to UDP.
The nice thing about PlanetScale is you get nearly unlimited connections (soft limit of like 250K IIRC) so making 1 connection per active lambda isn't a problem at all.
I've been using PlanetScale since shortly after they went GA and I've been very happy so far. Cheaper than Aurora Serverless, less hassle, and the branching feature is super cool. Zero-downtime deploys, with rollback support, feel magical.
This specifically is targeting environments where a MySQL client isn't able to run.
Oh, absolutely and I didn't mean to imply it wasn't useful, I was just saying I went down the Prisma path and will be sticking with that. Even so I'm glad this exists for times that I don't need the full weight/power of Prisma but do want to talk to a PlanetScale DB.
I feel that what we have would be useful within a Prisma context, but given their complexity and how Prisma works, it's likely not very practical with their "engine".
Other than having to bundle the engine I’ve been very happy with Prisma + PlanetScale. I’m open to any other TypeScript ORMs if you know of them.
For a theoretical new Node + TypeScript + SQL project, the three options I would consider are, from most-ORMy to least:
- Prisma
- Kysely + kysely-codegen (query builder)
- slonik + @slonik/typegen (raw postgres driver with good types; not sure if there's a good MySQL equivalent)
How does this work?
[0] https://planetscale.com/docs/concepts/deploy-requests
[1] https://planetscale.com/blog/its-fine-rewind-revert-a-migrat...
import { connect } from '@planetscale/database'
const config = { host: '<host>', username: '<user>', password: '<password>' }
https://github.com/planetscale/database-js
The idea is these runtimes typically only allow HTTP outbound requests, not whatever protocol is typically used to connect to a planetscale db.
[1] https://planetscale.com/docs/learn/operating-without-foreign...
I used Aurora Data API before moving off Aurora Serverless (insane pricing) to Prisma and PlanetScale, I don't think I'd go back to the HTTP API as Prisma works very well and I enjoy using it. One downside to Prisma is the DB "engine" is a arch-specific (obviously) binary that is pretty hefty. I want to say ~50MB. That can be a killer in serverless but I was able to work around it without much issue. If I ever wanted to dive into the world of Lambda layers I could probably move it into it's own later (however that still counts towards your max lambda size).
Yeah, I think this new thing would be less useful for Lambda as that does support TCP connections. However a HTTP API is required for e.g Cloudflare Workers where you can't create a normal MySQL client. I think that's where this could shine.
But the underlying tech is exactly the same as we use for handling traditional MySQL connections, so there isn't anything to fear.
if so and you used it, what was you experience running fauna in CF workers?
It seems like you lose most of the benefit of your code running at the edge if your database is still in an AWS region.
https://planetscale.com/docs/concepts/read-only-regions
I plan on following up with a more technical deep dive on some of these aspects, but it's quite a bit hard to do a 1:1 comparison.
While, obviously, HTTP also uses TCP, I'm going to assume you're asking more about the binary MySQL protocol vs HTTP.
On the surface, yes, HTTP is going to have more overhead with headers and the other aspects that come with HTTP. Also, in this case, the payload is JSON, both on the request and response, which is going to be slightly more bulky than the payloads over the MySQL protocol.
So where things get interesting is the real world performance implications. Networks and CPUs are really fast. HTTP has been extremely scrutinized and JSON as well. While they are admittedly not the most efficient standards, they are used so heavily and parsers are heavily optimized for these protocols.
Mixing in modern TLS 1.3 with modern ciphers, which is something you're very unlikely to get with a traditional MySQL client, we can achieve a much faster first connection time. Pairing with modern ciphers that are demanded for TLS 1.3, and the transport itself can be significantly faster than a slower cipher. This isn't the best comparison since typically you're using MySQL without TLS, but when talking to a service provider like us, we require it for obvious reasons.
Next, with HTTP, we get to leverage compression. In our case, we can use browser level native compression with gzip, or if using something server side, we support compressions like snappy. Combine something like gzip or snappy with HTTP/2 now, and given that the bulk of a query result is typically the result itself, and not the protocol, compression can make up a decent amount of the difference. Again, I'm being hand wavy since every query pattern and results are going to be wildly different based on your data, so it's not anything fair to say "this is x% more or less efficient."
And lastly, with HTTP, especially HTTP/2, we can multiplex connections. So similar to the gains with TLS 1.3, you can do many many concurrent database sessions across 1 actual TCP connection with HTTP/2. And similarly to TLS 1.3, the cost here is in connection timings, and management of a connection pool. You don't need a pool of connections on the client, so your startup times can be reduced.
As a stretch goal, HTTP/3 (with QUIC) is in the crosshairs, which should eliminate some more transport cost since it uses UDP rather than TCP. My hunch is all of this combined together, an optimized driver leveraging HTTP/3 might beat out a MySQL client overall. Time will tell though!
So there's no simple answer here, everything has tradeoffs. :)
A lot of this for me/us at this point is still theoretical since we don't have tons of application yet. Serverless was an easy first target since they require HTTP as the transport. The real experiments will come when we say, put together a Python or Go driver based on these APIs and compare side by side to a native MySQL driver.
With that said, I personally have no real opinions on this topic. It's pretty far out of my wheelhouse. I think that's more something like Firebase and gang, but that's not something we're gunning for at the moment and aren't trying to replace that.
That's awesome, I was assuming a sizable overhead (even if it is for a sizable benefit). But I can see how lots of factors contribute to closing the gap, and HTTP/3 might even be a negative overhead since it almost cuts out a protocol layer by stepping down to UDP.
Thanks again!
So we work there just fine.
It's open-source. What's your concern now?
https://github.com/planetscale/database-js