Thank you, so much to do in this space!
Bud looks great and I see we share many common goals. The dependency injection piece looks interesting. Did you build it custom or integrated an existing solution?
ha, add me to that list :) I think it's fine if CLIs are optional and I've tried to do that with Copper. Of course, your initial setup will be longer without scaffolding so it's not a great first time developer experience
Devs interested in this may also be interested in Pagoda [1], a rapid, easy full-stack web development starter kit in Go that I wrote. It leverages popular frameworks and modules that you can easily swap out, if desired. The readme contains full documentation and it's very much batteries-included.
just wanted to say i really like the approach of a "template/starter" that you took. My team isn't starting a new project anytime soon to really make full use of it, but we have pretty much adopted the pagoda service container approach in our home-grown "framework".
Cool project. I'm currently building something similar to hacker news using only the standard library. How could this project help me?
The way I use the standard library is like a super condensed version of React. Templates are my components. Since templates can be nested, templates can be used to build "components" and those components can be stored in separate files and compiled together at run time with a "model" being fed to the template(s) via a state object passed by the application.
So I may have a few dozen template files in a folder called /components, and another folder called /pages with a few templates that use these components. When a user visits a "page", the template file is "compiled" with the appropriate components.
Oh that's fun! In my demo video [1], I build a (minimal) HN clone so hopefully that answers your question in detail.
But the tldr is - you'll need a lot more than just templating for a production ready app. To name a few things - server, storage, migrations, logging, configs. IMO there's a huge benefit in having a batteries included toolkit that stays close to the stdlib - so you can totally keep your templates as is!
In what sense is this like React? Incremental re-rendering using html/template or text/template was virtually impossible last time I looked into it (for improving performance of some report generation), let alone getting any kind of DOM tree structure out.
It kinda does but there's a lot of boilerplate that goes in before we actually get to the app logic. And, understandably, there's no structure provided by the Go stdlib.
Once I made many projects with just using stdlib, I started taking common packages out. And Copper is essentially that. I can start writing my app code with minimal setup / boilerplate.
Go’s HTTP routing is very primitive. That’s one big area that I’ve always needed to use a 3rd party dependency. The main issue is that it doesn’t support placeholder values in route paths.
>The main issue is that it doesn’t support placeholder values
Can you elaborate? Like give me an example of a route path that uses placeholder values? I use Go to build web apps too and I really don't see a need for anything other than the standard library.
And what would that line of code be? As far as I am aware you have to parse the URL string, and it only gets more complicated when you add more URL parameters. Which definitely leads to boilerplate code shared across projects.
It's also not especially performant to do the naive (i.e. short) implementation like this, since you'll likely end up parsing/processing path elements left-to-right rather than in whatever order makes the most sense for your loading.
Reading through the comments, most of you are either new to Go or new to web dev in general.
Only that would explain how this SHITSHOW of a "framework" would get any praise at all.
What he copy/pasted already exists, only better.
The HN audience doesn't seem very sophisticaed. Hell even reddit's /r/golang is better informed than HN in this regard.
Just read through all the issues people have with gorm and if you've ever did any real Go development you would not pick gorm as the default database package in the first place.
Amateurs all of you, I'm seriously sick of your unprofessional lack of knowledge and experience.
Too bad it uses Wire. There’s absolutely no need for dependency injection frameworks in Go, you can just inject directly to receiver functions. It does the same thing without the crappy indirection added by this kind of frameworks.
So many frameworks in other languages are trying to get to the productivity of Rails but they just don't have the same spark imo.
There is no Rails equivalent in JS, theres lots of competitors that feel years away like SailsJS, the new Deno Fresh one etc, Adonisjs... Is NextJS/SvelteKit/RemixRun considered also? I don't even know if they have a standardised background job processor in JS land.
Java's solutions are dreadful imo for if you want to compare to Rails. Quarkus/SpringBoot/Micronaut are nowhere near productivity levels for a fullstack app. They lean heavily on the API only side of things. (I do like Java oddly enough)
PHP is the main competitor to Rails oddly enough, Laravel seems brilliant.
Go is just starting up in this space it looks like, Bud is another attempt at Rails in another language https://github.com/livebud/bud. However the Go ecosystem is heavily API only side of things instead of SSR. Go's templating libs suck imo.
Elixir of course has Phoneix which is apparently great, purely functional langs unfortunately dont fit my head and feel to abstract for myself (don't hate me)
Its no wonder we have the backend / frontend developer split nowadays.
Building a "Rails-like" framework in Go is honestly totally antithetical to the "Go way" of doing things.
Rails has a ton of magic, implicit behaviours, monkey patching, ERB, and so on.
Go is a touch below Java verbose, explicit, no magic, every function call can be very easily traced without having to do meta programming and code generation in your head to understand what's going on.
In my 8 years of writing Go, I would liken it the most to C, where you had to spell everything out, except without the manual memory management and the macro preprocessor.
Doing magic with Go via reflection or other implicit behaviours is generally annoying to deal with. One example is some libraries using struct tags, most of the time they work as expected, but sometimes you get some weird failure and these kinds of implicit behaviours are the culprit.
Overall, I don't rate these "all in one" frameworks highly in Go. The standard library is excellent for most applications, you only need to add some code to remove some boilerplate. For most apps that I have worked on in Go that involved web components, we maybe had to import e.g a websockets library or a more elegant routing library, but that's pretty much it.
Incidentally, I've found golang to be more verbose than Java.
Generics help a little bit, but the fact remains that error checking is pervasive and verbose, and you can't chain calls (e.g. slice.Map(func(a int) { return a * 2 }).Filter(func (a int) { return a % 2 == 0}).Reduce(0, func(a int, i int) { return a + i }) is not something that can be supported by golang when errors are involved, not to mention the extreme verbosity since it does not have a short way to pass functions).
Yes, writing function chains that compose from left to right, combined with the error checking convention, is a no-starter. So maybe you embed an error, and then check for error non-nil everywhere ? Or does KISS imply that you just panic and recover ? Some intensive googling finds that there's been many attempts to design an easy to use, easy to understand pipelining convention, but I have not found any one of them to be convincing.
SQL is an incredible language with decades of theory and implementation behind it. I have never found an ORM to be better or faster or more useful than hand-written SQL for anything but the simplest of queries, and even then the benefit is debatable.
99% of a CRUD app -- which is most of them -- is going to be simple CRUD stuff. Load a thing from database, let user edit it, save it. Doing that in 'plain' sql, even with templating, is repetitive and error prone.
An ORM that didn't let you escape to SQL when you wanted to do more complex things would be a total failure, of course -- but to suggest that they're not more convenient for the 80/20 or even 90/10 use case is just really hard to understand. Must be NIH syndrome.
If you're using postgres, I'm a major fan of the anti-ORM (ROM?) that is sqlc. Nothing I've used comes in Go close in productivity or safety (and ability to write proper queries but use them very simply, and stay up-to-date without too much extra toil). Last I checked they're also working on adding sqlite support.
As the post you're replying to describes, magic is antithetical to Go idioms. An ORM is basically database-as-magic, every ORM in Go is just a nightmare.
Something like sqlc[0] is leagues better than any ORM in terms of simplicity, complexity and maintainability.
ORM's help do simple stuff well enough, but they sure make hard tasks harder. Ever try to construct a 30 line report query using this years language flavor of an ORM?
https://sqlc.dev/ makes your SQL the focus, not your Go-specific query code.
https://sqlc.dev/ is not an ORM. It's not an abstraction over SQL. It is a brilliant way to make SQL the focus of your models by generating the code from your SQL, instead of the other way around.
Ent can replace an ORM in your architecture, and it's easiest to explain in two seconds as "an ORM for Go", but it's not really that. It's a way to describe a persistent object graph, without any reference to persistence or mapping details. When your chosen persistence layer is an RDBMS, as it often is, then it uses an ORM - but you rarely interact with it at that level even when specify your entity schemas. You can back it with an object DB or REST API instead and then it wouldn't need the RM part at all.
SQL is already a very high level API that compiles to low level algorithms? Why put another very high level API on top of that?
To top it off SQL is a leaky abstraction by nature. Two high level SQL queries with equivalent results can both compile into algorithms with COMPLETELY different performance profiles. You have to manipulate the SQL query to generate the correct performance profile. This means understanding things below the SQL abstraction.
You put a ORM or any new abstraction on top of that guess what? That abstraction must compile into hacked SQL. You have to be able to manipulate the ORM such that it generates the correct SQL such that the correct SQL generates the algorithm with the correct performance profile. The leak from the sql abstraction must propogate into the ORM layer which in itself must be a leaky abstraction. It's like dealing with a leaky pipe embedded within another leaky pipe.
Optimizing SQL is already a domain knowledge thing. Now you have to use new tricks to optimize the abstraction on top of it. Two Leaky abstractions on top of each other and both very high level is a bit of a head scratcher.
Why do people even make these abstractions that only make life harder? I think it's an illusion. It's to satisfy an OCD thing but people don't realize the OCD is an illusion. People want to deal with a single language, not have strings of another language living in the code. For example, SQL strings are seemingly kind of ugly in something like GO code.
Databases are the classic bottleneck of web development in terms of speed. Optimization is a very important part of writing SQL queries as a result. Having orms and other high level abstractions on top of this area is much much more harmful then it otherwise would be if databases did not exist in this bottle-necked area of web development.
In actuality it's also questionable whether or not a leaky abstraction in the first place was the right design decision. Is SQL the right abstraction for database queries? Or should we design another high level API that has a more 1 to 1 correspondence with optimization as in a High level API that's not leaky, like What Rust is to systems programming.
This is a good observation. One trend I'm noticing is that the "old way" (PHP, Rails, etc.) of doing things is making a comeback. Go is very well positioned for this but lacks the frameworks.
I'm hoping to add something like Phoenix to Copper. It should help with the "heavily API only side" problem. I've already added integrations for Tailwind and added some utilities on top of the templating (going to add more) to fix the lack of good templating.
Go and Java can never use the same framework patterns as Python/Php/Elixir/Ruby/JS because they are not dynamic enough. The Rails-style request mapped dispatching into active record ORM pattern requires a lot of flexibility on the host language side. For Go and Java, you basically end up with code generation or reflection, and the latter is a killer for performance. On the other hand, the performance of Go and Java is better by several orders of magnitude.
Models change pretty slowly, so codegen is a good solution! Ent uses codegen for its query API which looks pretty nice: https://entgo.io/docs/tutorial-todo-crud
I’m considering writing a Typescript clone of Ent with codegen powered by Typescript types. I like codegen over dynamic magic because the runtime behavior is often easier to understand. In Rails, I need to traverse a lot of space in Pry’s debugger mode to figure out WTF is happening. I would much rather have codegen.
Reflection is killer for performance because it brings Java/Go down to the level of a JIT-less language like Python (and ORMs aren't too kind to JITs in e.g. Ruby/JS either). The problem with reflection in Go/Java is mostly that it's hard to read, since the reflection-ful APIs are hosted rather than native syntax.
Agree reflection has (often big) trade offs but its not as bad as it was years ago, especially if its used during construct time rather than runtime, if you combine it with runtime caches you still get the benefits imo
Bud author here, thanks for including Bud on your list. That's a really good overview of the landscape!
My take is that Remix + Next.js + SvelteKit are going to continue to innovate fast in the Frontend and "Backend for Frontend" space. Rails and Laravel don't hold a candle to the experience you get in that ecosystem.
But the JS ecosystem is massive and, as a result, fragmented. As you mentioned, there's no consensus on ORMs, mailers, queues, etc.
I don't see those frameworks trying to push too far in that direction, they'll remain "UI focused". This is nice for their focus, but not great for someone who wants to launch a web app and doesn't want to figure out all the surrounding ecosystem tooling.
This is where Laravel, Rails (and soon Bud) (and I assume Copper) will shine. They provide more tools and interfaces out of the box for building full-featured backends. These frameworks definitely need to keep an eye on the best ideas coming out of the JS framework ecosystem though!
A rails-like framework will never happen in a language that doesn’t have the same meta programming capabilities as Ruby. Rails exists because Ruby exists not because DHH just happened to be a Ruby programmer. There is a reason people have tried to recreate it in other languages and it always feels jank - because Rails is designed specifically and enabled by the Ruby language.
Code generation provides the same kind of flexibility you get with meta programming, you just need to do more work to keep generated code in sync and out of the way.
> I'm doing something similar and love it. Do you embed the entire `public` directory and then traverse the embed.FS to access the files in memory?
I've done this in Rust before, I'm sharing it here as I'm assuming that Go has something similar.
I'm basically hard-coding the paths in the backend to also serve static assets, and embed the bytes of the asset at compile time, so when it runs, it's just serving it straight from memory. Here how it looks for style.css for example: https://codeberg.org/ditzes/ditzes/src/branch/master/src-tau...
It'd be trivial to move the structure to something like a map instead, where the URL is the key and another map where bytes, headers and such is stored. Mostly I didn't, because I'm just embedding few amount of files.
embed.FS is really cool. If you wanna see what it looks like, check out how I build ntfy [0]. It ships with the docs (built from mkdocs, see [1]) and the React web app [2] all in a single binary.
Here are the embed statements [3]. One thing to note is that embed.FS does not send 304 Not Modified status back, which is why I made a CachingEmbedFS [4].
That is how I am building mine and have thoroughly enjoyed it. Especially since 1.18 updated support to allow directories / files that start with an underscore.
I am using svelte for the frontend and that was biting me.
Please do! If nothing else, it's definitely a fun language to learn. We're also setting up a community of folks who are learning Go/Copper. Check the project readme link for the link.
So go is trying to be a server-side-render framework here, what about the SPA style in that go is a simple json-api server, and let SPA to do all the template and render, is gin the best framework for that? new to golang here.
I think that remains a very valid use case. In Copper, you can create one of those with the CLI. For example `copper create -frontend=vite:react github.com/nasa/starship` will create a react app with a JSON backed API ready to go
but it could be decent choice if u want to build integrated framework i can understand why people would choose it, another option is code_generation_meta_hell with sqlboiler. upper db [0] would have been perfect fit for this kind of project but is not that famous, its development is slow but stable.
ps: i like sqlboiler what i am saying is if u are building framework top on it then not that fun
Code generation is an awesome alternate, I'm more familiar with sqlc.dev. They're doing some really interesting work.
My goal with Copper is to provide out-of-the-box integrations with popular solutions to various problems. For now, I picked GORM but I definitely see adding support for other tools.
Will 2nd not being a fan of Gorm. In general not a fan of ORMs - implicit behaviours and magic query generation in a language like Go is completely antithetical to the explicit and verbose nature of the language.
[1] https://github.com/mikestefanello/pagoda
The way I use the standard library is like a super condensed version of React. Templates are my components. Since templates can be nested, templates can be used to build "components" and those components can be stored in separate files and compiled together at run time with a "model" being fed to the template(s) via a state object passed by the application.
So I may have a few dozen template files in a folder called /components, and another folder called /pages with a few templates that use these components. When a user visits a "page", the template file is "compiled" with the appropriate components.
A page might look like this:
And "nav", "thread", and "footer" are all components defined in another file. This allows for re-use across multiple pages.I want to do a write-up on it but I'm not sure if it's a novel idea.
But the tldr is - you'll need a lot more than just templating for a production ready app. To name a few things - server, storage, migrations, logging, configs. IMO there's a huge benefit in having a batteries included toolkit that stays close to the stdlib - so you can totally keep your templates as is!
[1] https://vimeo.com/723537998
Can you elaborate? Like give me an example of a route path that uses placeholder values? I use Go to build web apps too and I really don't see a need for anything other than the standard library.
That's one way to do it.
>more URL parameters
I thought thats what http.Request.ParseForm() was for (assuming correct formatting)
... is like, a core job of an HTTP router? You're not just begging the question, you're beating someone up and rifling through their pockets for it.
What he copy/pasted already exists, only better.
The HN audience doesn't seem very sophisticaed. Hell even reddit's /r/golang is better informed than HN in this regard. Just read through all the issues people have with gorm and if you've ever did any real Go development you would not pick gorm as the default database package in the first place.
Amateurs all of you, I'm seriously sick of your unprofessional lack of knowledge and experience.
HackerNews my ass, more like NoobNews. GTFO
There is no Rails equivalent in JS, theres lots of competitors that feel years away like SailsJS, the new Deno Fresh one etc, Adonisjs... Is NextJS/SvelteKit/RemixRun considered also? I don't even know if they have a standardised background job processor in JS land.
Java's solutions are dreadful imo for if you want to compare to Rails. Quarkus/SpringBoot/Micronaut are nowhere near productivity levels for a fullstack app. They lean heavily on the API only side of things. (I do like Java oddly enough)
PHP is the main competitor to Rails oddly enough, Laravel seems brilliant.
Go is just starting up in this space it looks like, Bud is another attempt at Rails in another language https://github.com/livebud/bud. However the Go ecosystem is heavily API only side of things instead of SSR. Go's templating libs suck imo.
Elixir of course has Phoneix which is apparently great, purely functional langs unfortunately dont fit my head and feel to abstract for myself (don't hate me)
Its no wonder we have the backend / frontend developer split nowadays.
Rails has a ton of magic, implicit behaviours, monkey patching, ERB, and so on.
Go is a touch below Java verbose, explicit, no magic, every function call can be very easily traced without having to do meta programming and code generation in your head to understand what's going on.
In my 8 years of writing Go, I would liken it the most to C, where you had to spell everything out, except without the manual memory management and the macro preprocessor.
Doing magic with Go via reflection or other implicit behaviours is generally annoying to deal with. One example is some libraries using struct tags, most of the time they work as expected, but sometimes you get some weird failure and these kinds of implicit behaviours are the culprit.
Overall, I don't rate these "all in one" frameworks highly in Go. The standard library is excellent for most applications, you only need to add some code to remove some boilerplate. For most apps that I have worked on in Go that involved web components, we maybe had to import e.g a websockets library or a more elegant routing library, but that's pretty much it.
Generics help a little bit, but the fact remains that error checking is pervasive and verbose, and you can't chain calls (e.g. slice.Map(func(a int) { return a * 2 }).Filter(func (a int) { return a % 2 == 0}).Reduce(0, func(a int, i int) { return a + i }) is not something that can be supported by golang when errors are involved, not to mention the extreme verbosity since it does not have a short way to pass functions).
An ORM that didn't let you escape to SQL when you wanted to do more complex things would be a total failure, of course -- but to suggest that they're not more convenient for the 80/20 or even 90/10 use case is just really hard to understand. Must be NIH syndrome.
I strongly disagree that not wanting to add yet another abstraction layer onto a proven technology would be NIH.
https://github.com/kyleconroy/sqlc/
Something like sqlc[0] is leagues better than any ORM in terms of simplicity, complexity and maintainability.
0: https://github.com/kyleconroy/sqlc
https://sqlc.dev/ makes your SQL the focus, not your Go-specific query code.
Why increase the complexity of something that's already a high level abstraction? Abstractions are about simplification. An orm and sqlc are not it.
https://entgo.io/
SQL is already a very high level API that compiles to low level algorithms? Why put another very high level API on top of that?
To top it off SQL is a leaky abstraction by nature. Two high level SQL queries with equivalent results can both compile into algorithms with COMPLETELY different performance profiles. You have to manipulate the SQL query to generate the correct performance profile. This means understanding things below the SQL abstraction.
You put a ORM or any new abstraction on top of that guess what? That abstraction must compile into hacked SQL. You have to be able to manipulate the ORM such that it generates the correct SQL such that the correct SQL generates the algorithm with the correct performance profile. The leak from the sql abstraction must propogate into the ORM layer which in itself must be a leaky abstraction. It's like dealing with a leaky pipe embedded within another leaky pipe.
Optimizing SQL is already a domain knowledge thing. Now you have to use new tricks to optimize the abstraction on top of it. Two Leaky abstractions on top of each other and both very high level is a bit of a head scratcher.
Why do people even make these abstractions that only make life harder? I think it's an illusion. It's to satisfy an OCD thing but people don't realize the OCD is an illusion. People want to deal with a single language, not have strings of another language living in the code. For example, SQL strings are seemingly kind of ugly in something like GO code.
Databases are the classic bottleneck of web development in terms of speed. Optimization is a very important part of writing SQL queries as a result. Having orms and other high level abstractions on top of this area is much much more harmful then it otherwise would be if databases did not exist in this bottle-necked area of web development.
In actuality it's also questionable whether or not a leaky abstraction in the first place was the right design decision. Is SQL the right abstraction for database queries? Or should we design another high level API that has a more 1 to 1 correspondence with optimization as in a High level API that's not leaky, like What Rust is to systems programming.
I'm hoping to add something like Phoenix to Copper. It should help with the "heavily API only side" problem. I've already added integrations for Tailwind and added some utilities on top of the templating (going to add more) to fix the lack of good templating.
I’m considering writing a Typescript clone of Ent with codegen powered by Typescript types. I like codegen over dynamic magic because the runtime behavior is often easier to understand. In Rails, I need to traverse a lot of space in Pry’s debugger mode to figure out WTF is happening. I would much rather have codegen.
My take is that Remix + Next.js + SvelteKit are going to continue to innovate fast in the Frontend and "Backend for Frontend" space. Rails and Laravel don't hold a candle to the experience you get in that ecosystem.
But the JS ecosystem is massive and, as a result, fragmented. As you mentioned, there's no consensus on ORMs, mailers, queues, etc.
I don't see those frameworks trying to push too far in that direction, they'll remain "UI focused". This is nice for their focus, but not great for someone who wants to launch a web app and doesn't want to figure out all the surrounding ecosystem tooling.
This is where Laravel, Rails (and soon Bud) (and I assume Copper) will shine. They provide more tools and interfaces out of the box for building full-featured backends. These frameworks definitely need to keep an eye on the best ideas coming out of the JS framework ecosystem though!
> One Binary
> Build frontend apps along with your backend and ship everything in a single binary.
I'm doing something similar and love it. Do you embed the entire `public` directory and then traverse the embed.FS to access the files in memory?
I've done this in Rust before, I'm sharing it here as I'm assuming that Go has something similar.
I'm basically hard-coding the paths in the backend to also serve static assets, and embed the bytes of the asset at compile time, so when it runs, it's just serving it straight from memory. Here how it looks for style.css for example: https://codeberg.org/ditzes/ditzes/src/branch/master/src-tau...
It'd be trivial to move the structure to something like a map instead, where the URL is the key and another map where bytes, headers and such is stored. Mostly I didn't, because I'm just embedding few amount of files.
[1]: https://pkg.go.dev/embed
Here are the embed statements [3]. One thing to note is that embed.FS does not send 304 Not Modified status back, which is why I made a CachingEmbedFS [4].
[0] https://github.com/binwiederhier/ntfy/blob/main/Makefile#L77
[1] https://ntfy.sh/docs/
[2] https://ntfy.sh/app
[3] https://github.com/binwiederhier/ntfy/blob/main/server/serve...
[4] https://github.com/binwiederhier/ntfy/blob/main/util/embedfs...
I am using svelte for the frontend and that was biting me.
Gotta put Go on my list of things to learn
As for database, jet is my favourite thus far.
but it could be decent choice if u want to build integrated framework i can understand why people would choose it, another option is code_generation_meta_hell with sqlboiler. upper db [0] would have been perfect fit for this kind of project but is not that famous, its development is slow but stable.
ps: i like sqlboiler what i am saying is if u are building framework top on it then not that fun
[0]: https://github.com/upper/db
edit: include link
My goal with Copper is to provide out-of-the-box integrations with popular solutions to various problems. For now, I picked GORM but I definitely see adding support for other tools.
[1] https://entgo.io/
https://news.ycombinator.com/showhn.html
https://news.ycombinator.com/newsguidelines.html
We ban accounts that break the rules like this. If you wouldn't mind reviewing them and sticking to them in the future, we'd appreciate it.
a.k.a. please don't feed the trolls
https://news.ycombinator.com/newsguidelines.html