Well not the Lisp userland many of us dreamed of but a very interesting project nonetheless.
The handbook[0] is a bit more clearer about the goals:
> Metropolis is a cluster operating system, meaning its goal is to run on a fleet of machines (be it physical or virtual) and pool their resources together into a unified API for operations and developer teams.
> A self-contained operating system: Metropolis is a full software stack, including the Linux kernel, userspace code, Kubernetes distribution and cluster management system. In contrast to traditional cluster administration, there are no puzzles to put together from a dozen vendors. The entire stack is tested as a single deployable unit.
> Eliminates state: Metropolis nodes don't have a traditional read-write filesystem, all of their state is contained on a separate partition with clear per-component ownership of data. All node configuration is managed declaratively on a per-node basis, and all cluster operations are all done by gRPC API.
Seems like they a specific use case in mind which will help keep the scope in check. Definitely looking forward to learn more about he project.
Also golang is a pretty good choice for such a project because it doesn't even have a libc dependency and calls the Kernel directly. So you can indeed have a very pure userland in a garbage collected language.
This looks interesting. However I can't find any documentation on how to program for it ? Where is their documentation about how we use this "unified API" which pools resources between nodes ?
IMVHO it's just another demonstration we need code and production together, we need end user programming and most try to deny that, while all tend to, a very small step at a time.
Python popularity, Notebook UIs, WebApps vs widget-based GUIs, this project... are all small examples of the same issue.
Is the end user you refer to a highly technical person? Because 1 in 5 Americans do not have the literacy skills to "complete tasks that require comparing and contrasting information, paraphrasing, or making low-level inferences"[0] so if you're describing end user software as something usable by everyone, expecting users to become programmers is simply an unrealistic expectation. The below average person is simply not smart enough to learn a DSL to get something done, and software should be accessible to everyone, even those who weren't lucky in the IQ lottery.
Well, "my" end-user is someone who have studied "computer science" a bit at school or to work, since he/she use computers every days, like he/she drive a car, so it's not acceptable he/she master computer usage less at level of mastery inferior to the one he/she master his/her car.
I know very well how illiterate so many people are, but that's not something to be justified but to be corrected. I imaging you do not want a bus driver who can barely make the bus move on the road piloting the bus you are on, why it's normal for you that let's say a tax administration employee is just able to click around while dealing with your taxes?
I've made a small experiment few years ago: with the help of some friend we have introduced for their very first time few kids to a desktop, a NixOS one running Emacs/EXWM. In 2 years at 6-8 years old, they was able to deal with emails, create some nicely formatted documents and doing basic math and so on with it. Two of them was presented a classic modern desktop: they became unable to do practically anything. It's a VERY small experiment of course, but to me it's enough to prove that people can learn if they are pushed toward a certain direction.
People can learn. I believe most programmers (who build side-projects) fall on the curious side. They explore stuff and thus discover software capabilities. Most people use software for a specific task, and learn only what is required. Making a software usable does not means making it a minimalist art project. It means making it consistent both in time (do not shuffle stuff around) and space (do not put things together randomly).
Learning how to type a letter in a word processor can be done in a day. But learning something like Microsoft Word should take a training. Just like you should buy a book about git or bash if you’re serious about learning them. Let the user figures it out is the wrong direction. As well as reducing the software capabilities (for potential power user) in order to reduce cognitive load for untrained users.
It's not that, it's another point: efficiency vs assembly line. Humans are not build to be robots, they are build to evolve.
If we learn a classic desktop paradigm computer system at school, while we learn all the basic and less basic cultural stuff we learn at school to became Citizens, we can profit from this knowledge for life. We can choose to dig deeper or not, but we have something useful for our entire life.
If we learn the modern desktop paradigm we never evolve. We are not Citizens, we are workers in an assembly line and upon any change from the factory owner our acquired knowledge goes to the bin.
I hope to have successfully described the point in my poor English. To give a simple example, I track my bills (well, like many, nothing special), I've crafted a bit of automation (org-mode notes + BeanCount + a bit of py automation), it took two/three days, MUCH more than most users do with modern tools, BUT thereafter anything goes nearly autonomously so I've spent 2/3 days + few seconds looking at my org-agenda regularly vs few minutes to start than keep spending few minutes all the time. In the short term the classic model is not good, in the middle term it's equivalent but demand more intellectual work, in the long term outshine the modern one so much that's like comparing a runner by feet against one on a jet in a speed race. Actually the effort spent in automating my bills notes it's also useful (at least in some parts) to automated other stuff, a bit at a time, an evolutionary step at a time I've built and keep up my PIM, again all the effort put pay back and I've gained valuable knowledge from doing that. The modern approach seems cheaper at first but it's much more expensive in the long run and gives little to no valuable knowledge at all.
The same model apply to any other aspects of our life, one to remain in economics the "ownership model" vs the "rent model", owning a home seems to be much, much more complex than rent one, it demand much more resources, much more computations and projection up front etc BUT it pay back much more thereafter. The rented home is just a regular expense that pile up year after year and at the end you have nothing. The owned home in most cases (essentially all, with insurances) have a final value, normally a big enough one to pay back the capex + opex of the time passed.
Since we are not made to live a single life but to evolve, doing better things a generation after another, passing knowledge and anything we can to newer generation, the modern paradigm is the slave paradigm, he/she produce a new generation that have no benefit from the old one, get nothing, leave nothing in the end. The classic is the human model, where we build families, passing what we have built and the accumulated knowledge, improving a generation after another.
I completely agree with you. I'm in the process moving from SaaS to a more personal computing space (selhosting, file-based app,…). I believe the dumbing down of user interfaces to sell an idea of "It just works" (magically) is a net negative. It also reduces the capabilities of the general population because there's no longer the idea of expertise (as expertise is not allowed). Maintaining a Arch Linux installation is not as easy as using MacOS but it removes the veneer of magic. And I think it's necessary knowledge if we need further improvement in the personal computing space. Just like C has its foot-guns, but learning it gives you a stabler foundation knowledge on how software works. It may not be for everyone, but removing the possibility is harmful for someone.
Allow me to suggest NixOS instead of Arch, for the same automation reason: NixOS meant a single (normally split in few) file(s) that describe your desired config, nixos-install/nixos-rebuild makes the system for you, so at any point in time if something break you just roll back to the previous build, not need to "emergency fix" something or have a test system aside. It's not a "distro war" but what I call a natural outcome of taking notes: in the same org-mode notes I keep my bills, my mails (ol-notmuch), my files (org-attach-ed and linked), a fully searchable "knowledge base" with just ripgrep, no recoll (xapian) or other indexers needed, but also my NixOS config, zsh config, ... and so on. Again the same automation (well, org-babel-tangle + little stuff) works for anything. The same documentation model works for anything. It's long to start, many things to learn, but it pay back much thereafter.
In my case while migrating toward a local approach I've tried some tools, like the idea of using Zotero for a bit anything, bookmarks, docs, ... it works, but it's not integrable with other tools like ALL modern apps, so I've put most in org-mode, bookmarks included, with Buku + ebuku to manage the SQLite DB from Emacs, org attaching stuff is slower than the Zotero connector but much more flexible, "Copy as Org-Mode" (FF extension) it's not as quick as Zotero notes, but the flexibility of one tool for anything is extreme, a thing we lost loosing the classic desktop model and today we can rediscover but still suffering the lack of development and the rest of the world heading in totally different directions.
Every piece of software doesn't need to be for everyone. Designing interfaces for the lowest common denominator throws away a lot of ideas that may make some people more productive to benefit people who won't be.
Hello, I'm one of the contributors to Monogon OS (codename Metropolis).
We didn't expect to go public with our project for a few more months, which is why the end-user documentation is still very bare, sorry! But fixing that is something we're prioritizing over the next few quarters.
How do you handle Ingress and network policies? How do you handle multiple clusters on a single physical machine (if I read the docs correctly) for ingress?
Bring your favourite Ingress controller. We support NodePort services, with plans for a LoadBalancer Service controller given the right network topology (a la MetalLB - either BGP to TOR switches to announce /32 / /128s, or maybe ARP/NDP-based annoucements for simpler deployments).
We don't support colocating multiple Kubernetes clusters on a single machine, or even within a single Monogon OS cluster. If you want multiple Kubernetes clusters, run multiple Monogon OS clusters.
We actually provide gVisor as the default runtime for Kubernetes workloads.
Monogon OS implements the layers underneath Kubernetes and gVisor - it's the operating system which runs on your nodes/machines and which comes together to form a cluster.
Thank you for sending this, Talos looks cool! I would not recommend actually running it in production though -- it does not seem possible to set up in a secure way. (unless you have an out-of-band VPN to the machine?)
The first time you send the machine config, you have to use the --insecure flag to avoid verifying its TLS cert. More concerning, there seems to be no way for you to authenticate yourself to the new machine. Anyone (most likely an automated scanner) could come in and make it theirs at this point.
Sure, there are solutions presented in the installation guide [1]. It usually involves using the cloud or virtualization platform's out of band channel, which Talos all supports, to securely provision a config on first boot.
You can also generate a custom installation medium or cloud image that pulls config from your trusted machines if you cannot use out-of-band provisioning.
You can also securely use the insecure maintenance mode when there is a firewall in front of the machine, which prevents access by non-administrator clients to the API ports on IP level.
I'm not a fan of Talos booting into insecure maintenance mode without config w/o prompting for at least a PIN displayed on-screen, but the problem you're describing in no way prevents production use.
We run Talos in production at Turnkey, including using it to schedule Nitro Enclaves so our most critical workloads can survive even if the whole cluster is compromised.
As others have mentioned, you do need a bit of DIY boootstrapping for PKI. Hopefully we can make our setup portable enough to open source soon.
This bears passing resemblance to Aurae [0]. Both projects are trying to reimagine the space between the Linux kernel and the cluster using modern languages for improved safety. Beyond that, I can’t confidently compare and contrast the projects at this time.
Zero abstractability, extremely imperative. It's very good at what it's for, but it's absolutely terrible for anyone who prefers a declarative or even functional style
The tradeoff is being able to decide how your project should be organized rather than someone else deciding for you. Sometimes you want that, sometimes you don't. The advantage of mixing paradigms in a single technology is the ability to switch / decide on paradigms without switching technologies.
Of course it has enumerations. It has had them from the get-go. After all, enums are a workaround to help deal with the shortcomings of a basic type system. You're bound to find them anywhere you find a basic type system, which certainly describes Go.
In fairness, there does seem to be some confusion floating around about the difference between enums and sum types, with some mistakenly calling sum types enums. Indeed, Go does not have sum types. This is what languages with more advanced type systems are bound to provide instead. Enums are pointless when you have a more expressive type system.
> Error handling requires lots of boilerplate.
What patterns are you seeing in other languages when it comes to error handling without the boilerplate?
Certainly there are other languages that provide shortcuts when not handling errors, but that's something else entirely.
The iota trick is not an actual enumeration as traditionally defined.
If err != nil after every fallible function call pales in comparison to being able to choose your own error handling such as automatic bubbling or wrapping that you can do in other languages. Go gives you two options: explicit handle or implicit ignore. There is no other option. The if statements clog up code unnecessarily. Rust does a great job of error handling, but I’d argue that almost every other language including Java does errors better than Go.
Enumeration is traditionally defined as establishing a number of something. iota accomplishes just that. That is what it is there for. Same as every other language that supports enums.
Maybe you’re thinking of the new definition that has recently emerged from confused Rust users that’s just another name for sum types? That’s actually sum types, not enumeration, though.
Letting an error bubble up isn’t handling it. Still interested in seeing what other languages are doing to eliminate the boilerplate when handling errors. For some reason you forgot to speak to it…
Explanation for enumeration: the thing about enumeration is you can ensure that you do an exhaustive handling of all cases in a switch statement. If an enum changes in a future library version you can get a compile error wherever the enum is enumerated in code which helps you handle new cases due to new features in a library. Go can’t do that.
I will agree on Go having bad error handling. That is the one pain point for me. As for complaints about lack of syntactic sugar and anti-expressivity: that's a feature, not a bug.
Remove the word "junior" and that's totally my opinion and why I absolutely adore it.
I can write a program while drunk or high and I know that I'll get 99% there, on the other hand python, js and C++ let me write basically whatever and it even let me run it.
Bazel is the externalized version of Google's internal build system. I think it has likely been overused by the overzealous in situations where its complexity is unwarranted and so a lot of people dislike it.
However it is absolutely fantastic at what it's intended for, providing a single build system across a large multilingual codebase, and building/running quickly, with extras for running tests, etc.
I for one, celebrate it, regardless of my dislike for Go's type system.
Every step helping taking C out of the picture is a welcome one.
At least it isn't like Microsoft where not matter what, .NET can't displace COM and C++ role on Windows, and isn't even used on new Azure Kubernetes samples.
I wouldn't mind Go if their executables didn't take hundreds of megabytes to do something a C/C++ executable could do in less than 2 megabytes.
There are a bunch of executable size issues languishing on their issue tracker and from what I understand it has to do with how some core features of the language are implemented that causes a combinatoric explosion of entries in a table in the executable. They also made an adjustment to stop compressing that table, which makes sense from a startup time perspective but there are also no levers I can pull to compress the table or disable the feature.
.NET executables have the same issue but I can always dynamically link them against a single runtime.
Funnily enough, .NET's NativeAOT took the opposite route and various tables have dehydrated form in the binary which will be hydrated at startup (with the size wins observed up to 30-35% depending on publish contents)
If you're interested, here's the PR history for the feature:
UPX doesn't always work in the cases where I've had this issue before. There was one application that was approaching 100 megabytes for what amounted to a really high-quality SSH bastion that I tried to use once. UPX got it down to about 30 megabytes, which was great, except that UPX needed to unpack the entire thing into memory to run. On the system I was trying to load this on I only had 128 megabytes of RAM so that size made it a non-starter for me. We do have a Go program to manage our CA certificates that uses UPX but we run it once during boot so it's much easier to manage.
I actually think being able to optionally dehydrate the tables and compute them at startup would be a big win for Go executable size to the degree that it might be a contender in the kinds of systems I work on.
Yes, this exact reason is why I'm not using UPX either. But it is useful when the conversations eventually devolve into binary size golfing ;) (ironic, I think NativeAOT does really well as of .NET 8 in this department).
I'm curious, what are the dependencies of that said Go program for it to take 100 MiB? Can't imagine a .NET (AOT or trimmed self-contained JIT) binary taking this much unless it is a GUI application.
The handbook[0] is a bit more clearer about the goals:
> Metropolis is a cluster operating system, meaning its goal is to run on a fleet of machines (be it physical or virtual) and pool their resources together into a unified API for operations and developer teams.
> A self-contained operating system: Metropolis is a full software stack, including the Linux kernel, userspace code, Kubernetes distribution and cluster management system. In contrast to traditional cluster administration, there are no puzzles to put together from a dozen vendors. The entire stack is tested as a single deployable unit.
> Eliminates state: Metropolis nodes don't have a traditional read-write filesystem, all of their state is contained on a separate partition with clear per-component ownership of data. All node configuration is managed declaratively on a per-node basis, and all cluster operations are all done by gRPC API.
Seems like they a specific use case in mind which will help keep the scope in check. Definitely looking forward to learn more about he project.
Also golang is a pretty good choice for such a project because it doesn't even have a libc dependency and calls the Kernel directly. So you can indeed have a very pure userland in a garbage collected language.
[0] https://docs.monogon.dev/metropolis-v0.1/handbook/index.html
Until I get to publishing it, the proto/gRPC definitions for node management are a good enough start: https://github.com/monogon-dev/monogon/blob/main/metropolis/...
And the top level API to actually deploy workloads is plain Kubernetes.
Python popularity, Notebook UIs, WebApps vs widget-based GUIs, this project... are all small examples of the same issue.
0: https://nces.ed.gov/pubs2019/2019179/index.asp
I know very well how illiterate so many people are, but that's not something to be justified but to be corrected. I imaging you do not want a bus driver who can barely make the bus move on the road piloting the bus you are on, why it's normal for you that let's say a tax administration employee is just able to click around while dealing with your taxes?
I've made a small experiment few years ago: with the help of some friend we have introduced for their very first time few kids to a desktop, a NixOS one running Emacs/EXWM. In 2 years at 6-8 years old, they was able to deal with emails, create some nicely formatted documents and doing basic math and so on with it. Two of them was presented a classic modern desktop: they became unable to do practically anything. It's a VERY small experiment of course, but to me it's enough to prove that people can learn if they are pushed toward a certain direction.
Learning how to type a letter in a word processor can be done in a day. But learning something like Microsoft Word should take a training. Just like you should buy a book about git or bash if you’re serious about learning them. Let the user figures it out is the wrong direction. As well as reducing the software capabilities (for potential power user) in order to reduce cognitive load for untrained users.
If we learn a classic desktop paradigm computer system at school, while we learn all the basic and less basic cultural stuff we learn at school to became Citizens, we can profit from this knowledge for life. We can choose to dig deeper or not, but we have something useful for our entire life.
If we learn the modern desktop paradigm we never evolve. We are not Citizens, we are workers in an assembly line and upon any change from the factory owner our acquired knowledge goes to the bin.
I hope to have successfully described the point in my poor English. To give a simple example, I track my bills (well, like many, nothing special), I've crafted a bit of automation (org-mode notes + BeanCount + a bit of py automation), it took two/three days, MUCH more than most users do with modern tools, BUT thereafter anything goes nearly autonomously so I've spent 2/3 days + few seconds looking at my org-agenda regularly vs few minutes to start than keep spending few minutes all the time. In the short term the classic model is not good, in the middle term it's equivalent but demand more intellectual work, in the long term outshine the modern one so much that's like comparing a runner by feet against one on a jet in a speed race. Actually the effort spent in automating my bills notes it's also useful (at least in some parts) to automated other stuff, a bit at a time, an evolutionary step at a time I've built and keep up my PIM, again all the effort put pay back and I've gained valuable knowledge from doing that. The modern approach seems cheaper at first but it's much more expensive in the long run and gives little to no valuable knowledge at all.
The same model apply to any other aspects of our life, one to remain in economics the "ownership model" vs the "rent model", owning a home seems to be much, much more complex than rent one, it demand much more resources, much more computations and projection up front etc BUT it pay back much more thereafter. The rented home is just a regular expense that pile up year after year and at the end you have nothing. The owned home in most cases (essentially all, with insurances) have a final value, normally a big enough one to pay back the capex + opex of the time passed.
Since we are not made to live a single life but to evolve, doing better things a generation after another, passing knowledge and anything we can to newer generation, the modern paradigm is the slave paradigm, he/she produce a new generation that have no benefit from the old one, get nothing, leave nothing in the end. The classic is the human model, where we build families, passing what we have built and the accumulated knowledge, improving a generation after another.
In my case while migrating toward a local approach I've tried some tools, like the idea of using Zotero for a bit anything, bookmarks, docs, ... it works, but it's not integrable with other tools like ALL modern apps, so I've put most in org-mode, bookmarks included, with Buku + ebuku to manage the SQLite DB from Emacs, org attaching stuff is slower than the Zotero connector but much more flexible, "Copy as Org-Mode" (FF extension) it's not as quick as Zotero notes, but the flexibility of one tool for anything is extreme, a thing we lost loosing the classic desktop model and today we can rediscover but still suffering the lack of development and the rest of the world heading in totally different directions.
We didn't expect to go public with our project for a few more months, which is why the end-user documentation is still very bare, sorry! But fixing that is something we're prioritizing over the next few quarters.
But I guess the cat's out of the bag, so I might as well stay around and answer any questions that pop up. We've also quickly set up a public Matrix room, which was long overdue: https://app.element.io/#/room/#monogon-os-community:matrix.o...
How do you handle Ingress and network policies? How do you handle multiple clusters on a single physical machine (if I read the docs correctly) for ingress?
Bring your favourite Ingress controller. We support NodePort services, with plans for a LoadBalancer Service controller given the right network topology (a la MetalLB - either BGP to TOR switches to announce /32 / /128s, or maybe ARP/NDP-based annoucements for simpler deployments).
We don't support colocating multiple Kubernetes clusters on a single machine, or even within a single Monogon OS cluster. If you want multiple Kubernetes clusters, run multiple Monogon OS clusters.
Monogon OS implements the layers underneath Kubernetes and gVisor - it's the operating system which runs on your nodes/machines and which comes together to form a cluster.
[0] https://www.talos.dev/
See this: https://www.talos.dev/v1.6/introduction/getting-started/#mod...
The first time you send the machine config, you have to use the --insecure flag to avoid verifying its TLS cert. More concerning, there seems to be no way for you to authenticate yourself to the new machine. Anyone (most likely an automated scanner) could come in and make it theirs at this point.
Is there a solution for that?
You can also generate a custom installation medium or cloud image that pulls config from your trusted machines if you cannot use out-of-band provisioning.
You can also securely use the insecure maintenance mode when there is a firewall in front of the machine, which prevents access by non-administrator clients to the API ports on IP level.
I'm not a fan of Talos booting into insecure maintenance mode without config w/o prompting for at least a PIN displayed on-screen, but the problem you're describing in no way prevents production use.
[1] https://www.talos.dev/v1.6/talos-guides/install/
As others have mentioned, you do need a bit of DIY boootstrapping for PKI. Hopefully we can make our setup portable enough to open source soon.
[^1]: https://gokrazy.org/
[0] https://aurae.io/
Of course it has enumerations. It has had them from the get-go. After all, enums are a workaround to help deal with the shortcomings of a basic type system. You're bound to find them anywhere you find a basic type system, which certainly describes Go.
In fairness, there does seem to be some confusion floating around about the difference between enums and sum types, with some mistakenly calling sum types enums. Indeed, Go does not have sum types. This is what languages with more advanced type systems are bound to provide instead. Enums are pointless when you have a more expressive type system.
> Error handling requires lots of boilerplate.
What patterns are you seeing in other languages when it comes to error handling without the boilerplate?
Certainly there are other languages that provide shortcuts when not handling errors, but that's something else entirely.
If err != nil after every fallible function call pales in comparison to being able to choose your own error handling such as automatic bubbling or wrapping that you can do in other languages. Go gives you two options: explicit handle or implicit ignore. There is no other option. The if statements clog up code unnecessarily. Rust does a great job of error handling, but I’d argue that almost every other language including Java does errors better than Go.
Maybe you’re thinking of the new definition that has recently emerged from confused Rust users that’s just another name for sum types? That’s actually sum types, not enumeration, though.
Letting an error bubble up isn’t handling it. Still interested in seeing what other languages are doing to eliminate the boilerplate when handling errors. For some reason you forgot to speak to it…
You’re thinking of sum types, or at least something like what Pascal has which is a variant on sum types only with support for integer tags.
That’s not a property of enumeration. Enumeration is simply about numbering. Literally the dictionary definition.
It's fine if you want a dull language that your junior devs will find hard to shoot themselves in the foot with. It's like a safer C.
I can write a program while drunk or high and I know that I'll get 99% there, on the other hand python, js and C++ let me write basically whatever and it even let me run it.
However it is absolutely fantastic at what it's intended for, providing a single build system across a large multilingual codebase, and building/running quickly, with extras for running tests, etc.
Every step helping taking C out of the picture is a welcome one.
At least it isn't like Microsoft where not matter what, .NET can't displace COM and C++ role on Windows, and isn't even used on new Azure Kubernetes samples.
https://learn.microsoft.com/en-us/samples/azure-samples/aks-...
There are a bunch of executable size issues languishing on their issue tracker and from what I understand it has to do with how some core features of the language are implemented that causes a combinatoric explosion of entries in a table in the executable. They also made an adjustment to stop compressing that table, which makes sense from a startup time perspective but there are also no levers I can pull to compress the table or disable the feature.
.NET executables have the same issue but I can always dynamically link them against a single runtime.
If you're interested, here's the PR history for the feature:
- https://github.com/dotnet/runtime/pull/77884
- https://github.com/dotnet/runtime/pull/79209
- https://github.com/dotnet/runtime/pull/78545
- https://github.com/dotnet/runtime/pull/79732
- https://github.com/dotnet/runtime/pull/78546
- https://github.com/dotnet/runtime/pull/78688
- https://github.com/dotnet/runtime/pull/78748
And if anything, you can always UPX them except on macOS where it is broken for some reason. There also exists a package that adds it as a build step: https://www.nuget.org/packages/PublishAotCompressed
I actually think being able to optionally dehydrate the tables and compute them at startup would be a big win for Go executable size to the degree that it might be a contender in the kinds of systems I work on.
I'm curious, what are the dependencies of that said Go program for it to take 100 MiB? Can't imagine a .NET (AOT or trimmed self-contained JIT) binary taking this much unless it is a GUI application.
What am I looking at?
A sqlc compiler? Why not use sqlite, which is proven beyond doubt?
A POSIX.2 userland in Go would not be unwelcomed in any way.