That looks cool! Though it looks like it only works for Java-based Lambdas, unfortunately.
Does it only work with HTTP Lambdas / manual invocations, or does it also play well with others, like SQS subscriptions? I think that's the biggest pain point for local Lambda development with the AWS-provided tooling for me right now, as detailed in a blog post I wrote a while ago[0].
All the official tooling only lets you do HTTP calls and manual invocations for locally running Lambdas, which means you can't really test architectures comprised of multiple Lambdas, or where Lambdas are just a part of the whole setup.
Yes it is only for Java runtime but Node.js runtime is also in our near-term roadmap. If you are looking for Node.js runtime, could you please upvote issue here: https://github.com/thundra-io/merloc/issues/1 or create a new one if you have any other request.
It works for *all* Lambda invocation triggers: Manual, HTTP, SQS, SNS, Kinesis etc ...
This is actually a tool i’d need to use - thank you for creating it.
However i cant help but notice how large corps, such as amazon, build products on top of freely available software, intentionally leave gaps in their offering (aws local dev is horrible, amazon doesnt provide a local env so we wont build an alternative - but is happy to use ours) and then the community steps in and fills in the gaps. I yet have to understand how they get away with such things.
Hopefully amazon will have the decency to sponsor such projects.
The difference is that MerLoc runs functions on your local as a part of AWS cloud infa. This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services.
Doesn't something like localstack.cloud help with this need? I've just recently inherited some NodeJS lambdas and am thinking of looking into that to debug/develop locally.
LocalStack is great tool but the difference here is that MerLoc runs your function in local against real cloud services with the same credentials of your real AWS Lambda function. So there is no AWS service simulation, mocking or local implementation here.
Not exactly. The paid editions have free trials, but there's also a community edition that is free. However, it's pretty difficult to figure out what exactly is supported in the community edition. There's a whole list that mention various AWS services are only supported in "PRO", but I don't know if that means that free support might come later in community.
[Disclaimer] I am the co-founder and CTO at Thundra which is serverless monitoring and debugging solution.
We also have developed another debugging tool to debug AWS Lambda function on remote AWS Lambda environment from your local IDE: https://www.serverlessdebugger.com/
It doesn't support hot-reload but very useful for debugging functions.
Does the hotreload feature comes out of the box? Is there any limitation regarding what can we reload while debugging? Could you expand that feature a bit?
Yes, hot-reload features comes out of the box and you don't need install any other tool or service.
There is no limitation for debugging. When you changed any source code, it is compiled and then related functions are reloaded on your local automatically. So when you apply a change or fix for your function, you can check whether it works or not without deploy to AWS Lambda environment. So it increases serverless application development productivity by helping you developing and debugging faster.
This is awesome and something I definitely would use for my docker and node based lambdas. Looking forward to this project adding more labmda backends!
this looks like a great solution to problems in serverless application development. I would be interested in how this does or does not overlap with the AWS toolkit:
Hi,
As far as I see, yhe plugins you mentioned just runs the function as a standalone app.
But MerLoc runs it on your local as a part of AWS cloud infr This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services
Current version of MerLoc is similar to SST except some minor details.
The biggest difference so far is that you can use MerLoc with any deployment tool like AWS CDK, AWS CloudFormation, AWS SAM, Serverless Framework, SST, Terraform, etc ...
And we are also working on an improvement which will let you use the same network with the remote function on your local.
Are Lambda functions doing so much now that they they can’t just be run locally?
I get that if you’re trying to access some AWS service, this could be helpful but in that case I thought the general consensus is basically mock things out.
The lambda developer experience is pretty awful.
I generally put "main" methods in all my lambdas so I can test them locally, but otherwise hitting actual AWS APIs. I'm not a big fan of mocking. Doing any sort of edit/deploy cycle is way too slow.
I had great success with unit testing the lambda code and mocking external/aws services.
For serious software you want a proper local/unit test setup anyway. Using this in your dev cycles is just the natural next step. Instead of waiting for the cdk/cfn deploy to finish, which really is way too slow, you just execute 'jest' or whatever test framework you have.
Sure, figuring out how to mock everything takes some upfront time, but it pays off in the long run.
I always write a remote function as just "an async function that takes type A and returns type B" - then I can write tests for it very easily and run it locally, IF it doesn't use any AWS services like databases.
For deployment I have a generic "wrapper" that makes it lambda-compatible by converting a webEvent -> Json and my outputJSON -> web response event.
Some people cram their entire application into a single lambda. Others are a little better and have their application spread out amongst multiple lambdas.
It still amazes me that AWS doesn't provide good local debugging/development support out of the box. I feel like we're slowly going back in time. Iteration cycles are very high, debugging cloud dev is essentially placing a bunch of print statements and trying to trace the code, build times take forever...
This where Azure and the Microsoft ecosystem is pretty far ahead from a developer experience standpoint.
I’m not an MS fanboy by any stretch but VS Code, Azure Functions, and even local storage emulation with Azurite is pretty slick.
First time I played with it my background was very far from anything in this space (literally first time I used node) and I had a completely locally developed toy function with storage up and running in maybe 30 minutes. Then in a few clicks it was deployed and running in Azure.
All in less time than it took for me to wander through the AWS docs and console to even figure out which product to use.
But honestly, all a Lambda is a script with a function that takes two parameters - an event and a context. You can scaffold testing and development just like you would a controller action in an MVC framework - use a test harness that calls the method with the correct JSON payload and usually a null event context.
This looks awesome, congrats Serkan and team! Indeed, looks similar to SST, with the extensions you've mentioned.
I can see this becoming very useful if you want the local debuggability of Lambdas in your IDE, with code running remotely. (i.e., not leveraging locally emulated services)
Looking forward to giving it a whirl and see how it can integrate with (or even run against) LocalStack..! :) Keep up the great work!
You are missing the point. Lambda functions are made for tiny scripts or relaying events that barely need any debugging. Also, they are highly integrated with other cloud services so it doesn’t make much sense to bother about complex dev tools for them.
Does it only work with HTTP Lambdas / manual invocations, or does it also play well with others, like SQS subscriptions? I think that's the biggest pain point for local Lambda development with the AWS-provided tooling for me right now, as detailed in a blog post I wrote a while ago[0].
All the official tooling only lets you do HTTP calls and manual invocations for locally running Lambdas, which means you can't really test architectures comprised of multiple Lambdas, or where Lambdas are just a part of the whole setup.
[0]: https://spacelift.io/blog/aws-lambda-migration
Yes it is only for Java runtime but Node.js runtime is also in our near-term roadmap. If you are looking for Node.js runtime, could you please upvote issue here: https://github.com/thundra-io/merloc/issues/1 or create a new one if you have any other request.
It works for *all* Lambda invocation triggers: Manual, HTTP, SQS, SNS, Kinesis etc ...
However i cant help but notice how large corps, such as amazon, build products on top of freely available software, intentionally leave gaps in their offering (aws local dev is horrible, amazon doesnt provide a local env so we wont build an alternative - but is happy to use ours) and then the community steps in and fills in the gaps. I yet have to understand how they get away with such things.
Hopefully amazon will have the decency to sponsor such projects.
https://docs.aws.amazon.com/serverless-application-model/lat...
The difference is that MerLoc runs functions on your local as a part of AWS cloud infa. This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services.
LocalStack is great tool but the difference here is that MerLoc runs your function in local against real cloud services with the same credentials of your real AWS Lambda function. So there is no AWS service simulation, mocking or local implementation here.
Looks like they're starting a Pro version that requires signup/payment.
Or maybe I'm missing something?
We also have developed another debugging tool to debug AWS Lambda function on remote AWS Lambda environment from your local IDE: https://www.serverlessdebugger.com/
It doesn't support hot-reload but very useful for debugging functions.
There is no limitation for debugging. When you changed any source code, it is compiled and then related functions are reloaded on your local automatically. So when you apply a change or fix for your function, you can check whether it works or not without deploy to AWS Lambda environment. So it increases serverless application development productivity by helping you developing and debugging faster.
There any many people looking for Node support as it is the most common serverless runtime.
Could you please thumbs-up its feature request here: https://github.com/thundra-io/merloc/issues/1
https://aws.amazon.com/intellij/
https://aws.amazon.com/pycharm/
https://aws.amazon.com/visualstudiocode/
I have used the Pycharm in Python Lambda development and I was able to set a breakpoint in my IDE and hit it and see the status of all my variables.
Is MerLoc an alternative to that?
But MerLoc runs it on your local as a part of AWS cloud infr This means that function is triggered from real events, runs againts real AWS services and again response is returned to real AWS services
Current version of MerLoc is similar to SST except some minor details.
The biggest difference so far is that you can use MerLoc with any deployment tool like AWS CDK, AWS CloudFormation, AWS SAM, Serverless Framework, SST, Terraform, etc ...
And we are also working on an improvement which will let you use the same network with the remote function on your local.
I get that if you’re trying to access some AWS service, this could be helpful but in that case I thought the general consensus is basically mock things out.
For serious software you want a proper local/unit test setup anyway. Using this in your dev cycles is just the natural next step. Instead of waiting for the cdk/cfn deploy to finish, which really is way too slow, you just execute 'jest' or whatever test framework you have.
Sure, figuring out how to mock everything takes some upfront time, but it pays off in the long run.
For deployment I have a generic "wrapper" that makes it lambda-compatible by converting a webEvent -> Json and my outputJSON -> web response event.
Edit: Here's some more conversations I found on the topic of the terrible workflow for cloud dev if anyone is interested https://news.ycombinator.com/item?id=26855037
Unfortunately we have a full Javascript stack at my company, but I'll keep an eye on this in case anybody adds more dev environments!
I’m not an MS fanboy by any stretch but VS Code, Azure Functions, and even local storage emulation with Azurite is pretty slick.
First time I played with it my background was very far from anything in this space (literally first time I used node) and I had a completely locally developed toy function with storage up and running in maybe 30 minutes. Then in a few clicks it was deployed and running in Azure.
All in less time than it took for me to wander through the AWS docs and console to even figure out which product to use.
But honestly, all a Lambda is a script with a function that takes two parameters - an event and a context. You can scaffold testing and development just like you would a controller action in an MVC framework - use a test harness that calls the method with the correct JSON payload and usually a null event context.
This looks awesome, congrats Serkan and team! Indeed, looks similar to SST, with the extensions you've mentioned.
I can see this becoming very useful if you want the local debuggability of Lambdas in your IDE, with code running remotely. (i.e., not leveraging locally emulated services)
Looking forward to giving it a whirl and see how it can integrate with (or even run against) LocalStack..! :) Keep up the great work!