Raymii.org

Table of Contents

• DNS hostname
• Routers, Middlewares and Services in traefik
• Adding basic auth password protection

Now that I have a high-available local kubernetes cluster and am experimenting with deploying apps, it's also time to look into securing those apps using certificates and passwords. In this case I'm going to set up password authentication, like a .htaccess file in Apache2, to protect the Longhorn dashboad, which by default requires no authentication. This means deploying an Ingress, a Middleware and a Secret.

Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:

I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!

Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.

You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $200 credit for 60 days. Spend $25 after your credit expires and I'll get $25!

Longhorn is a distributed replicated storage solution for Kubernetes and in my high-available k3s cluster I use it to make sure PersistentVolumes are replicated among nodes, thus making sure that when a Node fails, any deployment on there with a PersistentVolume can start up on another node without issues. By default k3s creates these volumes as local folders on a node, so when a node fails, the volume is also gone (until that node is back up). Longhorn resolves this by replicating the volumes among nodes and making them available to Kubernetes.

In my previous guide you can read on how to install and configure Longhorn. In that guide we Exposed the dashboard with the following command:

kubectl expose service longhorn-frontend --type=LoadBalancer --port=8877 --target-port 8000  --name=longhorn-frontend-ext --namespace longhorn-system

If you open your browser and go to your http://HA-IP:8877 you should be greeted by the dashboard and you can do anything without ever logging in or creating a token. This is not secure so lets fix it by adding a password (and in an upcoming article, https and certificates).

Here you can see a screenshot of my Longhorn dashboard while volumes are being rebuilt after a Node failure:

A bit of set up is required, so lets get started!

I'm using k3s/kubernetes version v1.30.2+k3s1.

DNS hostname

I recently wrote a small guide on how to expose a Service on a hostname (domain name) instead of an ip:port combo. In this guide I'll assume that you also have set up such a domain for your k3s cluster.

I'll be using the following domain in this guide:

longhorn.k3s.homelab.mydomain.org

Routers, Middlewares and Services in traefik

I'm not that familiar with traefik but after diving into their documentation, for traefik version 2, I think I have a better understanding of the terms used.

• A Router is comparable to a frontend (like in haproxy)
• A Service is comparable to a backend
• A Middleware sits in between the two and can modify the request, headers, do redirects and such stuff. You can have multiple Middlewares.

image source

One of the functions that a Middleware can provide is basic-auth, in our case the password protection.

Adding basic auth password protection

We must create a Middleware resource for traefik which handles the password authentication. My kubernetes distribution,k3s, comes with traefik, if you use nginx, this guide won't work for you.

Middleware is a Custom Resource Definition of the traefik middlewares.

The username and password itself are generated like you would do for a .htpasswd file in Apache2. On your local admin workstation, install the required tools to generate such passwords:

apt install apache2-utils

Navigate to your kubernetes folder, then to the longhorn folder. In my high available cluster guide I created such a folder structure for deployment and it should contain a longhorn.yaml file.

Generate the file longhorn-auth-file with a password in it for user admin:

htpasswd -c longhorn-auth-file admin

If you ever want to add a user, omit the -c option and rerun the command with a different username. You must update the kubernetes Secret as well with the new content.

In a Kubernetes Secret the string (in our case generated by htpasswd) must be base64-encoded. Do that for the file we just generated:

base64 longhorn-auth-file

Output:

dXNlcjok[...]Cgo=

If you think you are never going to need more than 1 user, you could pipe the username/password:

htpasswd -nb admin password | openssl base64

That will result in the same string, but is less flexible because you cannot add users later on.

Create a file for your Ingress for the dashboard:

vim longhorn-ingress.yaml

The content consist out of multiple pieces of yaml, separated by three dashes (---). I'll be covering them piece by piece, but they all go into one file.

First is the Secret:

apiVersion: v1
kind: Secret
metadata:
  name: longhorn-basic-auth-secret
  namespace: longhorn-system
data:
  users: |2
    dXNlcjok[...]Cgo=
---

The |2 is yaml syntax, this stackoverflow post explains what happens (Block Scalar Header and newline trimming).

You must paste the base64 string below the |2 line, indented with 2 spaces below users:.

The Secret is named longhorn-basic-auth-secret and that name will be used for the next part, the Middleware:

apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
  name: longhorn-basic-auth-middleware
spec:
  basicAuth:
  secret: longhorn-basic-auth-secret
  realm: "Longhorn Dashboard"
---

This basically wraps the traefik middleware configuration. It contains the users and a realm, which is what will show up in the basic auth prompt. Note that as I said earlier, this will only work for the traefik service.

Last part of the file is the Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: longhorn-ui-ingress
  namespace: longhorn-system
  annotations:
    spec.ingressClassName: traefik
    traefik.ingress.kubernetes.io/router.middlewares: longhorn-system-longhorn-basic-auth-middleware@kubernetescrd
spec:
  rules:
  - host: longhorn.k3s.homelab.mydomain.org
    http:
    paths:
      - path: "/"
      pathType: Prefix
      backend:
        service:
        name: longhorn-frontend
        port:
          number: 80

This is a fairly standard Ingress, except for the following annotations:

spec.ingressClassName: traefik
traefik.ingress.kubernetes.io/router.middlewares: longhorn-system-longhorn-basic-auth-middleware@kubernetescrd

This last line must have the following format:

<namespace>-<middleware-name>@kubernetescrd

The character @ is not allowed in the Middleware name. If you want multiple Middlewares, you must separate them with a comma.

Apply the file:

kubectl apply -n longhorn-system -f longhorn-ingress.yaml

Output:

secret/longhorn-basic-auth-secret created
middleware.traefik.io/longhorn-basic-auth created
ingress.networking.k8s.io/longhorn-ui-ingress created

Open your browser and navigate to the domain you set up in the Ingress file and you should be prompted by a login prompt:

You might notice my URL starting with https, but that is part of an article that is still in the works.

Tags: apache , apache2 , armbian , cloud , helm , htpasswd , k3s , k8s , kubernetes , linux , orange-pi , raspberry-pi , security , tutorials