Great piece. I think the future of FaaS architecture still isn't widely understood (see: slim vs fat function discussion below) mostly because people are going to great lengths to shoehorn old architectural styles into FaaS platforms.
To be honest, I think AWS Lambda (and Azure Functions / etc.) nailed the compute and economic model, but only got halfway there WRT abstraction layer and ideal development for serverless functions themselves. There's an opportunity here to introduce millions of new software engineers to a programming model where they no longer have to think about how their web services run and can treat remote function calls like first-class citizens of their development environment.
In fact, this is what we've built and we're continuing to iterate upon at StdLib [1]. One of the most common pieces of feedback we receive from our customers is; "this is everything serverless should be." We have an open source Gateway specification called FaaSlang [2] that makes local development a breeze and perfectly in sync with production functions, as well as providing automatic error (type, parameter) checking at the request level, before function execution. We've only been able to get here by "throwing the baby out with the bathwater" and reimagining backend web development with serverless functions from the ground up.
Disclaimer: I'm the founder of the company and pretty gosh darn excited about the future of this space. Check us out if you get the chance, you won't be disappointed. :)
>For example, a 200ms service task that needs to run every five minutes would need a dedicated service instance (or two!) in a traditional model
This feels like such a straw man. Under what traditional model would you spawn off an entire server instance for such a minor function? You'd slap it in as a service on an existing host. Hosts which for a long time now have been virtualized on consolidated hardware, not bare metal.
Serverless can bring big benefits for certain workloads, but hyping it up by misrepresenting the alternatives seems a dishonest attempt to justify investment in the big architectural changes you have to make to get there.
> Under what traditional model would you spawn off an entire server instance for such a minor function
I think that's the point - would you rather run your own VPS and manage the logs/errors from a cron job? Or simply write code that conforms to their contract and not worry about logs, errors, and whether it can scale or will fail to restart itself.
Or have another vps/server with spare capacity where this makes sense to run as well? In most cases, such a function will be closely related to other functions and can be grouped with them.
> "… serverless platforms today are useful for important (but not five-nines mission critical) tasks, where high-throughput is key, rather than very low latency, and where individual requests can be completed in a relatively short time window. The economics of hosting such tasks in a serverless environment make it a compelling way to reduce hosting costs significantly, and to speed up time to market for delivery of new features."
The crux so to speak. I wonder if this type of planning makes sense though? Can't I argue that good design, which imo has been around for ages and collected well in canonical books, aid in improving time to market for new features?
(I'm one of the coauthors of the paper referenced in the post)
For me, charging for actual usage instead of reserved capacity actually aids in implementing those good design guidelines that were collected in books, such as low coupling and high cohesion of code, and this is what we wrote about in the paper as well. charging for reserved capacity creates financial incentives to bundle tasks and features into applications, creating runtime coupling (eg sharing the same /tmp dir, reusing security roles etc), even though they are designed to be isolated. charging for actual usage removes that incentive, so stuff that was designed to be isolated stays isolated when deployed.
If you give me a monolith with well isolated packages vs a serverless app across 400 lambdas and ask me to fix a bug, i can bet you which one i can find it faster.
Separation of concerns is easy to do without lambda or microservices. If you want separation of concerns write good code. Don't move your environment to a locked in circus.
I've seen surprisingly little on running "fat" services in a serverless (like AWS Lambda) environment. We're doing this at work pretty successfully so far: Django projects with many endpoints within a single Lambda function. There are limitations to work with (50MB compressed code) but they're quite manageable for some time, and so far a nicer problem to deal with than all the things we give up by abandoning serverless. So try adjusting your example to something like a dozen lambdas for an engineering organization of 50 devs, with each lambda service boundary roughly matching the team and product structure.
Serverless enables the extreme (many tiny services with minimal upfront cost) but doesn't require it.
Are you doing zappa or something else for your Django?
This sounds better, and perhaps I could be on board with this. I went a different route, which is kubernetes for my Django + Celery. Over 75% of my load is "stable", so kubernetes + celery ends up cheaper than lambda for me. I can basically throw my web servers in my kubernetes cluster "for free".. but if I had just a webapp, I would feel better about my entire Django in one Lambda, or at least 1 entire "app" of my Django in one Lambda. My biggest gripe is warmup speed, 3 seconds for a cold request is pretty bad compared to the 100ms I have now in kubernetes.
We now use Zappa only for packaging the ZIP that we upload to Lambda. For deployment, we use CloudFormation (via Sceptre) which we've found to be a much more robust, declarative approach than Zappa's imperative-style deployment.
We use Lambda primarily to improve developer efficiency and to allow dev teams to own their own operations end-to-end, with cost efficiency only a secondary goal. It's been great for that as it's a small enough thing to integrate with that any given developer can learn the entire operations stack (for their team's services) well enough to work on it themselves.
3 seconds on cold request sounds unusual, it should be around a second or less - have you measured it? If you use VPC you'll unfortunately be in the realm of 20-30 seconds cold start, which is why we've avoided it for Lambda (and used stores like DynamoDB rather than RDS which work with IAM as an alternative to security groups). No Elasticache without VPC is going to be a big problem soon though...
That might be a catchy phrase. "Fat functions", or a single function that holds a lot of endpoints in a single lambda (or another platforms' version of cloud functions). Have you had any issues with requests getting timed out, or managing the 'health' of the fat function?
No unusual timeout issues - our "fat function" has only marginal overhead, just the WSGI layer and method dispatching code to route the request to the appropriate view/controller. This should be single digit millisecond overhead.
For health, there's certainly more noise now: we can look at overall invocation error rates (a metric lambda gives us), but they're across several endpoints within the lambda function. This is still an open question for us, but solvable.
The serverless graphic had more arrows than the server based one. I'm not sure that is an argument for serverless being simpler.
The serverless functions take up to 1s to boot. It is not fair to compare that to a VM that is on all the time. We should compare it to a container that is booted on demand. Right now only Heroku does that. At GitLab we're considering adding idling by sponging requests https://gitlab.com/gitlab-org/gitlab-ee/issues/3712
To be honest, I think AWS Lambda (and Azure Functions / etc.) nailed the compute and economic model, but only got halfway there WRT abstraction layer and ideal development for serverless functions themselves. There's an opportunity here to introduce millions of new software engineers to a programming model where they no longer have to think about how their web services run and can treat remote function calls like first-class citizens of their development environment.
In fact, this is what we've built and we're continuing to iterate upon at StdLib [1]. One of the most common pieces of feedback we receive from our customers is; "this is everything serverless should be." We have an open source Gateway specification called FaaSlang [2] that makes local development a breeze and perfectly in sync with production functions, as well as providing automatic error (type, parameter) checking at the request level, before function execution. We've only been able to get here by "throwing the baby out with the bathwater" and reimagining backend web development with serverless functions from the ground up.
Disclaimer: I'm the founder of the company and pretty gosh darn excited about the future of this space. Check us out if you get the chance, you won't be disappointed. :)
[1] https://stdlib.com/
[2] https://github.com/faaslang/faaslang
>For example, a 200ms service task that needs to run every five minutes would need a dedicated service instance (or two!) in a traditional model
This feels like such a straw man. Under what traditional model would you spawn off an entire server instance for such a minor function? You'd slap it in as a service on an existing host. Hosts which for a long time now have been virtualized on consolidated hardware, not bare metal.
Serverless can bring big benefits for certain workloads, but hyping it up by misrepresenting the alternatives seems a dishonest attempt to justify investment in the big architectural changes you have to make to get there.
I think that's the point - would you rather run your own VPS and manage the logs/errors from a cron job? Or simply write code that conforms to their contract and not worry about logs, errors, and whether it can scale or will fail to restart itself.
The crux so to speak. I wonder if this type of planning makes sense though? Can't I argue that good design, which imo has been around for ages and collected well in canonical books, aid in improving time to market for new features?
For me, charging for actual usage instead of reserved capacity actually aids in implementing those good design guidelines that were collected in books, such as low coupling and high cohesion of code, and this is what we wrote about in the paper as well. charging for reserved capacity creates financial incentives to bundle tasks and features into applications, creating runtime coupling (eg sharing the same /tmp dir, reusing security roles etc), even though they are designed to be isolated. charging for actual usage removes that incentive, so stuff that was designed to be isolated stays isolated when deployed.
Separation of concerns is easy to do without lambda or microservices. If you want separation of concerns write good code. Don't move your environment to a locked in circus.
Serverless enables the extreme (many tiny services with minimal upfront cost) but doesn't require it.
This sounds better, and perhaps I could be on board with this. I went a different route, which is kubernetes for my Django + Celery. Over 75% of my load is "stable", so kubernetes + celery ends up cheaper than lambda for me. I can basically throw my web servers in my kubernetes cluster "for free".. but if I had just a webapp, I would feel better about my entire Django in one Lambda, or at least 1 entire "app" of my Django in one Lambda. My biggest gripe is warmup speed, 3 seconds for a cold request is pretty bad compared to the 100ms I have now in kubernetes.
We use Lambda primarily to improve developer efficiency and to allow dev teams to own their own operations end-to-end, with cost efficiency only a secondary goal. It's been great for that as it's a small enough thing to integrate with that any given developer can learn the entire operations stack (for their team's services) well enough to work on it themselves.
3 seconds on cold request sounds unusual, it should be around a second or less - have you measured it? If you use VPC you'll unfortunately be in the realm of 20-30 seconds cold start, which is why we've avoided it for Lambda (and used stores like DynamoDB rather than RDS which work with IAM as an alternative to security groups). No Elasticache without VPC is going to be a big problem soon though...
For health, there's certainly more noise now: we can look at overall invocation error rates (a metric lambda gives us), but they're across several endpoints within the lambda function. This is still an open question for us, but solvable.
Having said that tooling is key and I find it somewhat lacking to turn on a switch and run my code as a lambda to say compare costs and such.
The serverless functions take up to 1s to boot. It is not fair to compare that to a VM that is on all the time. We should compare it to a container that is booted on demand. Right now only Heroku does that. At GitLab we're considering adding idling by sponging requests https://gitlab.com/gitlab-org/gitlab-ee/issues/3712
>You can’t run a fully simulated Lambda environment on your local machine.
... is now partially solved via SAM Local: http://docs.aws.amazon.com/lambda/latest/dg/test-sam-local.h...