https://raymii.org/s/tags/kubernetes.xml RSS feed for tag kubernetes on Raymii.org https://raymii.org/s/tutorials/High_Available_Mosquitto_MQTT_Broker_on_Kubernetes.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/High_Available_Mosquitto_MQTT_Broker_on_Kubernetes.html In this post, we'll walk through a fully declarative, Kubernetes-native setup for running a highly available MQTT broker using Eclipse Mosquitto. This configuration leverages core Kubernetes primitives (`Deployments`, `Services`, `ConfigMaps`, and `RBAC`), alongside Traefik `IngressRouteTCP` to expose MQTT traffic externally. It introduces a lightweight, self-healing failover mechanism that automatically reroutes traffic to a secondary broker if the primary becomes unhealthy. The setup also demonstrates internal MQTT bridging, allowing seamless message propagation between brokers. The big advantage over a single Pod deployment (which, in case of node failure, k8s will restart after 5 minutes) is that this setup has a downtime of only 5 seconds and shared state, so all messages will be available on a failover. Wed, 14 May 2025 22:11:00 GMT Wed, 14 May 2025 22:11:00 GMT https://raymii.org/s/tutorials/Safely_expose_the_Kubernetes_Dashboard_in_Traefik_k3s_via_a_ServersTransport.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Safely_expose_the_Kubernetes_Dashboard_in_Traefik_k3s_via_a_ServersTransport.html I'm using the Headlamp dashboard for my [high-available local kubernetes cluster](/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html) because I find that to be faster, more clear and useful than the full blown Kubernetes Dashboard. In [my first article](/s/tutorials/My_First_Kubernetes_k3s_cluster_on_3_Orange_Pi_Zero_3s_including_k8s_dashboard_hello-node_and_failover.html#toc_4) I accessed the dashboard via a local port forward. This article documents how to expose the dashboard via an `Ingress` and some `Traefik` specific `annotations`. The dashboard helm chart sets up HTTPS internally, `Traefik` does not like that by default. Most of the time, all internal cluster communication is insecure (I'm not sure why, seems to be a bad idea). A few of the guides online suggest disabling HTTPS for the dashboard internally or, for the k3s specific case, disabling HTTPS validation entirely. Both of those are too broad for my use case, so I decided to figure out how to make `Traefik` talk to the `kubernetes-dashboard-kong-proxy` via `https`, without disabling certificate validation. Tue, 11 Mar 2025 22:11:00 GMT Tue, 11 Mar 2025 22:11:00 GMT https://raymii.org/s/tutorials/Create_Kubernetes_user_restricted_to_one_namespace_with_resource_limits.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Create_Kubernetes_user_restricted_to_one_namespace_with_resource_limits.html This guide shows you how to use Role-based access control (RBAC) to create a user account that only has rights for one specific namespace. I'll also show you how to limit the resource usage of that `Namespace`. Last but not least, I'll also show you how to create a `kubeconfig` file for that specific user. Mon, 29 Jul 2024 04:39:00 GMT Mon, 29 Jul 2024 04:39:00 GMT https://raymii.org/s/tutorials/nameConstraints_on_your_Self_Signed_Root_CA_in_Kubernetes_with_cert_manager.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/nameConstraints_on_your_Self_Signed_Root_CA_in_Kubernetes_with_cert_manager.html If you have [set up a Self Signed Root CA](/s/tutorials/Self_signed_Root_CA_in_Kubernetes_with_k3s_cert-manager_and_traefik.html) for your local Kubernetes Cluster and have trusted the Root Certificate, you are at risk if the key is compromised. If the key is stolen, it can be used to create trusted certificates for everything. Luckily there is something we can do, using `nameConstraints` to limit the scope of the Root Certificate to, in our case, a single domain (`k3s.homelab.mydomain.org`). This means that if your key would be compromised, it would only be able to issue certificates for anything under that domain, not your bank for example. Wed, 17 Jul 2024 23:22:00 GMT Wed, 17 Jul 2024 23:22:00 GMT https://raymii.org/s/tutorials/Self_signed_Root_CA_in_Kubernetes_with_k3s_cert-manager_and_traefik.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Self_signed_Root_CA_in_Kubernetes_with_k3s_cert-manager_and_traefik.html Now that I'm learning Kubernetes for a few weeks, I'm finally at the point where I was 20 years ago with regular boring old tech, being able to [host multiple domains](/s/tutorials/Kubernetes_k3s_Ingress_for_different_domains_like_virtual_hosts.html), [password protection](/s/tutorials/Password_protect_web_services_in_Kubernetes_k3s_traefik_with_basic_auth.html) and [high available clusters](/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html). It seems we have to re-invent the wheel every time but in the end, it's just resume-driven development, the underlying stack costs more, is way more complex but for the user, nothing changes, they see the same website as always. [Not all change is progress](https://luddites.latenightlinux.com/). Enough of being a curmudgeon, time to continue with Kubernetes. In this episode of 'Remy discovers Kubernetes', I'm setting up `cert-manager`, **not with Lets Encrypt**, but with a self-signed certificate authority. I'll also show you how to set up a regular certificate, one you've for example bought somewhere. I'll also cover `nameConstraints` to make the risk of compromise of your trusted root ca lower. Wed, 17 Jul 2024 04:22:00 GMT Wed, 17 Jul 2024 04:22:00 GMT https://raymii.org/s/tutorials/Password_protect_web_services_in_Kubernetes_k3s_traefik_with_basic_auth.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Password_protect_web_services_in_Kubernetes_k3s_traefik_with_basic_auth.html Now that I have a [high-available local kubernetes cluster](/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html) and am [experimenting with deploying apps](/s/snippets/Using_nodeSelector_to_deploy_a_Kubernetes_Helm_chart_only_on_x86_or_amd64_nodes_not_arm64.html), it's also time to look into securing those apps using certificates and passwords. In this case I'm going to set up password authentication, like a `.htaccess` file in `Apache2`, to protect the `Longhorn` dashboad, which by default requires no authentication. This means deploying an `Ingress`, a `Middleware` and a `Secret`. Mon, 15 Jul 2024 20:11:00 GMT Mon, 15 Jul 2024 20:11:00 GMT https://raymii.org/s/snippets/Using_nodeSelector_to_deploy_a_Kubernetes_Helm_chart_only_on_x86_or_amd64_nodes_not_arm64.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/snippets/Using_nodeSelector_to_deploy_a_Kubernetes_Helm_chart_only_on_x86_or_amd64_nodes_not_arm64.html My [k3s cluster](/s/tutorials/My_First_Kubernetes_k3s_cluster_on_3_Orange_Pi_Zero_3s_including_k8s_dashboard_hello-node_and_failover.html) runs on Orange Pi Zero 3 small board computers, with a 1.5 GHz Allwinner H618 Quad-Core Cortex-A53 ARM64 CPU. Nowadays most popular software has support for `aarch64` due to the popularity of boards like the Raspberry Pi and the Apple M1 series processors, but smaller projects or niche software often can only run on x86/amd64. If you write your own yaml files for deployment you can use a `nodeSelector` combined with the `kubernetes.io/arch=amd64`, but with a Helm Chart this is not that obvious. This small snippets shows you the correct syntax to force deploy a Helm Chart to only amd64 nodes in your cluster. This assumes you have a mixed cluster, I added a small virtual machine to it for testing. Thu, 11 Jul 2024 23:39:00 GMT Thu, 11 Jul 2024 23:39:00 GMT https://raymii.org/s/tutorials/Kubernetes_k3s_Ingress_for_different_domains_like_virtual_hosts.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Kubernetes_k3s_Ingress_for_different_domains_like_virtual_hosts.html Now that I have a [high-available local kubernetes cluster](/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html) it's time to learn not just managing the cluster but actually deploying some services on there. Most examples online use a `NodePort` or a `LoadBalancer` to expose a service on a port, but I want to have domains, like, `grafana.homelab.mydomain.org` instead of `192.0.2.50:3000`. Back in the old days this was called [Virtual Host](https://web.archive.org/web/20240515131604/https://httpd.apache.org/docs/2.4/vhosts/), using 1 IP for multiple domains. My k3s cluster uses `traefik` for its incoming traffic and by defining an `Ingress` we can route a domain to a service (like a `ClusterIP`). This page will show you how. Wed, 10 Jul 2024 20:39:00 GMT Wed, 10 Jul 2024 20:39:00 GMT https://raymii.org/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/High_Available_k3s_kubernetes_cluster_with_keepalived_galera_and_longhorn.html After my [first adventure with Kubernetes](/s/tutorials/My_First_Kubernetes_k3s_cluster_on_3_Orange_Pi_Zero_3s_including_k8s_dashboard_hello-node_and_failover.html), getting started with k3s on my small 3 node ARM cluster that [boots via PXE / NFS](/s/tutorials/Netboot_PXE_Armbian_on_an_Orange_Pi_Zero_3_from_SPI_with_NFS_root_filesystem.html), I noticed that there is only one k3s node that has the `control-plane,master` role. If that node fails you can no longer manager the cluster. Other nodes can fail and then the workloads (pods) will be restarted eventually after 5 minutes, but this node is special. Time to change that and make it a high available cluster. K3s [supports](https://web.archive.org/web/20240703112841/https://docs.k3s.io/datastore/ha) high-availability with embedded `etcd` and with external databases like `MySQL` and `postgres`. `etcd` will thrash your storage (SD cards) so I decided to go with a `MySQL` cluster using `Galera` for the database and `keepalived` for the High Available Cluster IP. This guide will show you how to configure the HA database and HA-IP and I'll also setup [longhorn](https://web.archive.org/web/20240707025724/https://longhorn.io/) for high-available block storage inside kubernetes. The end result is that I can pull the power from any two of the three nodes without the k3s cluster or workloads going down. Tue, 09 Jul 2024 22:30:00 GMT Tue, 09 Jul 2024 22:30:00 GMT https://raymii.org/s/tutorials/My_First_Kubernetes_k3s_cluster_on_3_Orange_Pi_Zero_3s_including_k8s_dashboard_hello-node_and_failover.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/My_First_Kubernetes_k3s_cluster_on_3_Orange_Pi_Zero_3s_including_k8s_dashboard_hello-node_and_failover.html I've been working as an embedded C++ developer for over 5 years now so my sysadmin / devops skills are becoming a bit rusty. The odd bit of Ansible here and there but no new stuff. I figured it was time to expore Kubernetes, as it is what all the cool kids do these days. So I [got myself 3 new SBC's] (/s/tutorials/Netboot_PXE_Armbian_on_an_Orange_Pi_Zero_3_from_SPI_with_NFS_root_filesystem.html), the [Orange Pi Zero 3] (https://web.archive.org/web/20240623200133/http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-Zero-3.html). I'll be using these to install and setup a basic Kubernetes cluster, getting the Dashboard working, installing a Hello World app and testing how the failover works. Fri, 28 Jun 2024 22:30:00 GMT Fri, 28 Jun 2024 22:30:00 GMT https://raymii.org/s/tutorials/Netboot_PXE_Armbian_on_an_Orange_Pi_Zero_3_from_SPI_with_NFS_root_filesystem.html?utm_medium=rss&utm_source=raymii&utm_campaign=tagrss https://raymii.org/s/tutorials/Netboot_PXE_Armbian_on_an_Orange_Pi_Zero_3_from_SPI_with_NFS_root_filesystem.html Because I wanted to experiment with Kubernetes I bought a few cheap SBC's and a Power over Ethernet switch to run `k3s`. Since Kubernetes is very resource intensive I wanted to try to boot the boards via the network without causing wear on the Micro SD card. The boards have built-in SPI flash from which it can boot `u-boot` and Armbian works quite well with a root filesystem over NFS. This guide will help you with netbooting an Orange Pi Zero 3 running Armbian. Tue, 25 Jun 2024 22:30:00 GMT Tue, 25 Jun 2024 22:30:00 GMT