If you need an automated tool to set something up, that's a problem in my book.
From someone who understands k8s, but doesn't have much practical knowledge of it, the amount of "getting started" and "quick start" articles and tools is a major turnoff for me. If I can't easily read and understand the system and deploy it, I'm using blackbox system that I need to manage but can't understand.
I need an automated tool to deploy this thing, another tool to manage it, a tool to manage my applications, another tool to do specialized CI/CD, and so on and so on. Just seems insane to me.
Managing self-installed k8s is a pain in the ass right now, but not much more than trying to manually install Linux without using e.g. the Ubuntu installer. Would that put you off using Linux?
Speaking as a daily Linux user since 1996, there are so many parts of the system I still don't understand, but it's not necessary for me to e.g. understand exactly how the font catalog and text rendering works to write a document. The same is true here.
There are quite a few companies in this space trying to improve the self-hosted experience, just give it some time
As for all the tools you supposedly need, the only one you actually need is kubectl, and it's more than enough to accomplish everything. New ecosystems always endure this abundance of "helpful tools" before consolidating on a few important choices.
You're talking about Linux desktop, no? It's apples and oranges because I assume we're talking about maintaining production systems here.
k8s is a mess IMO. It's a giant abstraction and you're forced to interact with the whole thing all at once. If you've done any kind of sysadmin/ops work on production systems then k8s makes you feel like you're re-learning everything all over again. That's a huge red flag to me. When a technology tries to reinvent every layer of the stack instead of gracefully integrating with existing tools then I immediately question the motives and/or competence of the creators.
I'm not sure how to express this next point without a touch of vitriol so I'm just going to say it: the way containerization is being promoted for general purpose use is ridiculous. We're living in a world where a 8 core 16G server is dirt cheap. The amount of WORK an application server can do with those resources is insane (if built well). The idea of slicing/dicing that mono application into 8 tiny pods with 3.33333 allotted millicores and a bunch networked cross traffic -- I, I... just don't get it. Why?
The vast vast majority of business critical applications can serve all of their traffic with a single server running on a modest (by today's standards) VPS. Add a little redundancy and auto-scaling and you're good to go.
When I see small startups trying to maintain the k8s black box I just facepalm myself into unconsciousness.
I don't understand all this angst. In a well run infrastructure, Kubernetes basically just boils down to a new config file to describe some proxy rewrite rules, and how much RAM your thing needs.
If you want to run VPS-sized pods full of untracked mutable state with their own ghetto logging / monitoring / deployment / configuration management baked in, nothing is stopping you, but by the time we're "maintaining production systems", and we discover that crumby little $5 VPS has been touched by 7 people over 2 years, nobody knows what is on it any more, turns out the default logrotate configuration was shredding audit logs, and nobody has a clue how to reinstall it without literally starting over, things start to look a little different.
- "I wish I had a list of all the jobs that were supposed to be on this VPS"
- "I wish I didn't have to setup random log archiving cronjobs for all these apps"
- "I wish I knew what directories I need to back up"
- "I wish I could split this one job off the VPS without having to rewrite and retest its config"
- "I wish I could run a separate deployment environment on the VPS without having to set up a whole new VPS"
- "I wish I could rotate this API key but I haven't a clue where it is stored or what is referencing it"
- "I wish I could give the intern access to deploy the web site without letting him also take a copy of the HR database"
- "I wish I could deploy this new job version and roll it back if I fuck things up"
- "I wish I could run this ancient proprietary tool that needs 32bit libs from Debian 0.4 without installing them globally and potentially bricking the whole VPS"
- "I wish I didn't have to go install and configure a bunch of monitoring plugins every time I deploy a new app"
You can set all that up by hand of course, after all it does represent years of fruitful low-intellect busywork for idle hands, or you can come up with your own self-assembled collection of third party tools to do it for you, investing the research, installation and configuration time that entails, and accepting the cost of having created an ad-hoc system only you understand, or you could simply pick one big framework that does it all at once and try to make it work.
The nice thing with the one-big-framework approach is that you also get a common language and operational model understood by thousands of people you can hire on the open market. They understand how your configs are laid out, they know how to discover mutable state and query the health of all your existing jobs. And by the time they leave the company, you can understand all that about the work they've done.
This is only addressing Kubernetes as an operational methodology. Containerization as an architectural style is an entirely different topic.
I would never run VPS-sized pods because that negates most of the advantages k8s provides (as an infra abstraction). I would not run k8s in the first place.
> I wish...
This right here is a big part of the problem. "I wish". "What if" is another one.
In the real world you don't have all of these problems initially and often times never at all. They trickle in as you grow. And in our business an absolutely essential property of good tooling is that it grows with your needs.
This talk of "years of fruitful low-intellect busywork for idle hands" for "self-assembled collection" of tools is the antithesis of the Unix philosophy. I like the Unix philosophy. It's worked out very well for us so far.
> a common language and operational model understood by thousands of people you can hire on the open market
We've been doing just fine with sysadmins? The community standardizes on a set of tools and people learn them. Do you really believe that pre-k8s it was some kinda combinatorial wild west without any shared knowledge? Come on. Come. On.
k8s environments aren't all that standardized either btw. There's just as much duct tape and glue as everything else. CI/CD configuration, service meshes, multi-cloud provider configurations, etc. All of these interfaces bring their own quirks and pitfalls. And your k8s nodes still need to be optimized just like before. The control plane is only one part of the picture.
All I'm saying is this... an experienced sysadmin with a mature config management system and AWS/GCP/Azure will get you a perfectly maintainable infrastructure for most use cases.
There are big organizations where k8s makes sense b/c you'd end up building a worse home grown version of basically the same thing. I'll give you that. But I'm seeing k8s pitched as the default infra solution for SMBs and the like. That's where the angst is coming from.
If GNU/Linux was unusable without various hacked-on helper frontends, none of which was comprehensive and none of which even tried to standardize, then yes that would be a red flag. In practice, you have things like Ubuntu putting a nice frontend on, but you also have things like the Arch folks, who explicitly refuse to provide an installer because the "manual" process is totally reasonable. So as an outsider, my understanding is that there are plenty of different k8s distros, and a lot of helper frontends (and no clear winners among them); are there people who find vanilla/handcrafted k8s okay? Is there an "Arch K8s" distro? Or even, is there an Ubuntu of k8s (successfully hides away the details and produces a clean working system that never requires a user to leave the comfort of the GUI)? Unfortunately, I don't have the domain knowledge to tell whether any given helper scripts are going to bite me down the road, so this is a genuine question from someone who'd like to get into k8s but can't see which option is sane.
> are there people who find vanilla/handcrafted k8s okay?
Yes, we do.
> Is there an "Arch K8s" distro?
Kubernetes the hard way is pretty much Arch
> is there an Ubuntu of k8s (successfully hides away the details and produces a clean working system that never requires a user to leave the comfort of the GUI)?
Kubespray etc.
> so this is a genuine question from someone who'd like to get into k8s but can't see which option is sane.
It completely depends on how much money your company is willing to spend on it. The larger the organization, the more it makes sense to go for "the Arch way". If you are a tiny start-up with single digit Devops folks, stick to managed offerings like GKE.
Basically most of what runs on k8s, including most of the k8s control plane, lives in a container. So for the most part, the only material concern about the host machine and OS is what kernel it is running, because that may limit what kind of jobs you can run in the containers.
Then you probably haven't tried to schedule and manage thousands of containers in production. The reason people use Kubernetes is the problems it solves are much more significant than the complexity of bootstrapping it.
Where I work, we run thousands of JVM instances on bare VMs without issues. We've been doing it for years.
As we're moving these VM based deployments to k8s, the overhead is going up as well as issues. The old system was super easy to understand. "this program downloads the new build file, it extracts it to this directory, starts it up, checks if it's healthy, spins down the old instance, an then updates the load balancer with the new port"
Most of the kubernetes clusters are deployed with EKS, GKE or digital ocean or other managed providers, none of them are bootstraped, you dont have to know how to install kubernetes to use them, and you are not supposed to be even concerned about that. So having a quick way to setup one locally for testing or learning makes sense.
You dont have to know how to build a car, to learn how to drive.
This is getting ridiculous. Ketchup is only 3 letters longer and where the hell does the s come from? If you expect my to pronounce it ketchup then spell it correctly. Otherwise I'm calling it k 3 sup.
I don’t necessarily disagree, but it’s probably a lot easier to come up with unique project names. So many names are taken in package registries or the community in general.
Also in this project, if they didn’t say it was called “ketchup”, then k3sup makes some sense. If we accept that k3s exists, k3sup sounds reasonable.
You mean c2sup? Either way I create unnecessary confusion to save typing one character? Not worth it. I'm not a fan of i18n but at least it has a purpose.
It looks like this is more like "k3s up" to bootstrap k3s, which I just learned is purportedly a lightweight Kubernetes [1]. They named it K3s because:
> We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
This program is a kubernetes boostrap, AKA k8s bootstrap, AKA k8s "up", something that _actually_ sounds like ketchup instead of kethreesup. The author not seeing that makes me both sad and angry.
"light-weight utility to get from zero to KUBECONFIG" -- what is kubeconfig?
Here I am a total n00b in Kubernetes, hoping to get a clear path for learning it as an app dev. I heard K3S as "easier" K8S, but still confused how to get started and be productive with it (as app dev).
DISCLAIMER: I work for Red Hat Consulting as an OpenShift/Kubernetes consultant.
You might want to try minikube [0], minishift [1], CDK [2] or CRC [3]. You'll want to have a machine with a bit of resources available. 2 cores+, 8GB+ RAM, 20GB+ disk space for minikube, minishift, CDK. CRC is happier with 4 cores+, 16GB+ RAM, 40GB+ disk space.
kubeconfig (`~/.kube/config`) [4] is a little like your `~/.ssh` folder where your Kubernetes clusters sessions, credentials, metadata are stored & maintained.
I would recommend Kind instead of any of these tools. It's the best local k8s evaluation tool I have found yet. We use it in our projects for local development too.
From someone who understands k8s, but doesn't have much practical knowledge of it, the amount of "getting started" and "quick start" articles and tools is a major turnoff for me. If I can't easily read and understand the system and deploy it, I'm using blackbox system that I need to manage but can't understand.
I need an automated tool to deploy this thing, another tool to manage it, a tool to manage my applications, another tool to do specialized CI/CD, and so on and so on. Just seems insane to me.
Speaking as a daily Linux user since 1996, there are so many parts of the system I still don't understand, but it's not necessary for me to e.g. understand exactly how the font catalog and text rendering works to write a document. The same is true here.
There are quite a few companies in this space trying to improve the self-hosted experience, just give it some time
As for all the tools you supposedly need, the only one you actually need is kubectl, and it's more than enough to accomplish everything. New ecosystems always endure this abundance of "helpful tools" before consolidating on a few important choices.
k8s is a mess IMO. It's a giant abstraction and you're forced to interact with the whole thing all at once. If you've done any kind of sysadmin/ops work on production systems then k8s makes you feel like you're re-learning everything all over again. That's a huge red flag to me. When a technology tries to reinvent every layer of the stack instead of gracefully integrating with existing tools then I immediately question the motives and/or competence of the creators.
I'm not sure how to express this next point without a touch of vitriol so I'm just going to say it: the way containerization is being promoted for general purpose use is ridiculous. We're living in a world where a 8 core 16G server is dirt cheap. The amount of WORK an application server can do with those resources is insane (if built well). The idea of slicing/dicing that mono application into 8 tiny pods with 3.33333 allotted millicores and a bunch networked cross traffic -- I, I... just don't get it. Why?
The vast vast majority of business critical applications can serve all of their traffic with a single server running on a modest (by today's standards) VPS. Add a little redundancy and auto-scaling and you're good to go.
When I see small startups trying to maintain the k8s black box I just facepalm myself into unconsciousness.
/rant, sorry
If you want to run VPS-sized pods full of untracked mutable state with their own ghetto logging / monitoring / deployment / configuration management baked in, nothing is stopping you, but by the time we're "maintaining production systems", and we discover that crumby little $5 VPS has been touched by 7 people over 2 years, nobody knows what is on it any more, turns out the default logrotate configuration was shredding audit logs, and nobody has a clue how to reinstall it without literally starting over, things start to look a little different.
- "I wish I had a list of all the jobs that were supposed to be on this VPS"
- "I wish I didn't have to setup random log archiving cronjobs for all these apps"
- "I wish I knew what directories I need to back up"
- "I wish I could split this one job off the VPS without having to rewrite and retest its config"
- "I wish I could run a separate deployment environment on the VPS without having to set up a whole new VPS"
- "I wish I could rotate this API key but I haven't a clue where it is stored or what is referencing it"
- "I wish I could give the intern access to deploy the web site without letting him also take a copy of the HR database"
- "I wish I could deploy this new job version and roll it back if I fuck things up"
- "I wish I could run this ancient proprietary tool that needs 32bit libs from Debian 0.4 without installing them globally and potentially bricking the whole VPS"
- "I wish I didn't have to go install and configure a bunch of monitoring plugins every time I deploy a new app"
You can set all that up by hand of course, after all it does represent years of fruitful low-intellect busywork for idle hands, or you can come up with your own self-assembled collection of third party tools to do it for you, investing the research, installation and configuration time that entails, and accepting the cost of having created an ad-hoc system only you understand, or you could simply pick one big framework that does it all at once and try to make it work.
The nice thing with the one-big-framework approach is that you also get a common language and operational model understood by thousands of people you can hire on the open market. They understand how your configs are laid out, they know how to discover mutable state and query the health of all your existing jobs. And by the time they leave the company, you can understand all that about the work they've done.
This is only addressing Kubernetes as an operational methodology. Containerization as an architectural style is an entirely different topic.
> I wish...
This right here is a big part of the problem. "I wish". "What if" is another one.
In the real world you don't have all of these problems initially and often times never at all. They trickle in as you grow. And in our business an absolutely essential property of good tooling is that it grows with your needs.
This talk of "years of fruitful low-intellect busywork for idle hands" for "self-assembled collection" of tools is the antithesis of the Unix philosophy. I like the Unix philosophy. It's worked out very well for us so far.
> a common language and operational model understood by thousands of people you can hire on the open market
We've been doing just fine with sysadmins? The community standardizes on a set of tools and people learn them. Do you really believe that pre-k8s it was some kinda combinatorial wild west without any shared knowledge? Come on. Come. On.
k8s environments aren't all that standardized either btw. There's just as much duct tape and glue as everything else. CI/CD configuration, service meshes, multi-cloud provider configurations, etc. All of these interfaces bring their own quirks and pitfalls. And your k8s nodes still need to be optimized just like before. The control plane is only one part of the picture.
All I'm saying is this... an experienced sysadmin with a mature config management system and AWS/GCP/Azure will get you a perfectly maintainable infrastructure for most use cases.
There are big organizations where k8s makes sense b/c you'd end up building a worse home grown version of basically the same thing. I'll give you that. But I'm seeing k8s pitched as the default infra solution for SMBs and the like. That's where the angst is coming from.
Yes, we do.
> Is there an "Arch K8s" distro?
Kubernetes the hard way is pretty much Arch
> is there an Ubuntu of k8s (successfully hides away the details and produces a clean working system that never requires a user to leave the comfort of the GUI)?
Kubespray etc.
> so this is a genuine question from someone who'd like to get into k8s but can't see which option is sane.
It completely depends on how much money your company is willing to spend on it. The larger the organization, the more it makes sense to go for "the Arch way". If you are a tiny start-up with single digit Devops folks, stick to managed offerings like GKE.
As we're moving these VM based deployments to k8s, the overhead is going up as well as issues. The old system was super easy to understand. "this program downloads the new build file, it extracts it to this directory, starts it up, checks if it's healthy, spins down the old instance, an then updates the load balancer with the new port"
Also in this project, if they didn’t say it was called “ketchup”, then k3sup makes some sense. If we accept that k3s exists, k3sup sounds reasonable.
> We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
[1]: https://rancher.com/docs/k3s/latest/en/
This program is a kubernetes boostrap, AKA k8s bootstrap, AKA k8s "up", something that _actually_ sounds like ketchup instead of kethreesup. The author not seeing that makes me both sad and angry.
Here I am a total n00b in Kubernetes, hoping to get a clear path for learning it as an app dev. I heard K3S as "easier" K8S, but still confused how to get started and be productive with it (as app dev).
You might want to try minikube [0], minishift [1], CDK [2] or CRC [3]. You'll want to have a machine with a bit of resources available. 2 cores+, 8GB+ RAM, 20GB+ disk space for minikube, minishift, CDK. CRC is happier with 4 cores+, 16GB+ RAM, 40GB+ disk space.
kubeconfig (`~/.kube/config`) [4] is a little like your `~/.ssh` folder where your Kubernetes clusters sessions, credentials, metadata are stored & maintained.
[0] https://github.com/kubernetes/minikube
[1] https://www.okd.io/minishift/
[2] https://developers.redhat.com/products/cdk/overview
[3] https://developers.redhat.com/products/codeready-containers/...
[4] https://kubernetes.io/docs/concepts/configuration/organize-c...