These are the steps I use to set up k3s lightweight kubernetes for local development with Arch Linux.

This guide results in a deployment using LetsEncrypt via Traefik, HTTPS support, and a vanilla nginx web server.

Note that, with these instructions, LetsEncrypt will only generate a valid HTTPS certificate if the computer where k3s is being installed can be reached via HTTP on port 80 over the public Internet using the domain name we specify below.

Routing and Firewall

Enable IP forwarding.

sudo sysctl -w net.ipv4.ip_forward=1

Note that nftables is not currently a supported firewall backend for k8s/k3s.

Here is an exaple of how to switch from running nftables to a firewall management application, firewalld, using the iptables backend. 

sudo pacman -Rs nftables \
  && sudo pacman -Sy firewalld \
  && sudo sed -i 's/FirewallBackend=.*/FirewallBackend=iptables/g' /etc/firewalld/firewalld.conf \
  && sudo systemctl restart firewalld \
  && sudo systemctl enable firewalld

Depending on your needs the necessary ports for HTTP and HTTPS traffic should be opened and port forwarding should have already been established upstream so the Internet can reach this machine. 

sudo firewall-cmd --add-port=443/tcp --permanent \
  && sudo firewall-cmd --add-port=80/tcp --permanent \
  && sudo firewall-cmd --reload

Reboot (I find I have to do this) to ensure all firewall and routing settings have taken effect. Otherwise, I encounter errors with the k3s installation. 

sudo reboot

Install k3s

yay -Sy --needed k3s-1.22-bin kubectl \
  && sudo systemctl enable k3s \
  && sudo systemctl start k3s \
  && mkdir -p "${HOME}/.kube" \
  && sudo cp /etc/rancher/k3s/k3s.yaml "${HOME}/.kube/config" \
  && sudo chown $(id -u):$(id -g) "${HOME}/.kube/config"

Wait until the system looks healthy like so.

archlinux% kubectl -n kube-system get pods
NAME                                     READY   STATUS      RESTARTS   AGE
coredns-85cb69466-5xqkh                  1/1     Running     0          64s
metrics-server-9cf544f65-4hb9j           1/1     Running     0          64s
local-path-provisioner-64ffb68fd-z4v7z   1/1     Running     0          64s
helm-install-traefik-crd--1-nq2vn        0/1     Completed   0          64s
helm-install-traefik--1-hpf5w            0/1     Completed   1          64s
svclb-traefik-hbz7l                      2/2     Running     0          27s
traefik-74dd4975f9-prjqb                 1/1     Running     0          28s

Configure Traefik and LetsEncrypt Support

Update the Traefik LetsEncrypt resolver configuration. Swap out the email below with your valid email address. 

cat << 'EOF'| sudo tee /var/lib/rancher/k3s/server/manifests/traefik-config.yaml
kind: HelmChartConfig
apiVersion: helm.cattle.io/v1
metadata:
  name: traefik
  namespace: kube-system

spec:
  valuesContent: |-
    additionalArguments:
      - --certificatesresolvers.traefikresolver.acme.email=me@myfakedomain.com
      - --certificatesresolvers.traefikresolver.acme.storage=/data/acme.json
      - --certificatesresolvers.traefikresolver.acme.httpchallenge.entrypoint=web
    persistence:
      enabled: true
      path: /data
EOF

Create and Apply Nginx Config

Create a file at nginx.yaml with these contents.

Replace wwww.myfakedomain.com below with your domain. 

kind: Namespace
apiVersion: v1
metadata:
  name: nginx-namespace
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  namespace: nginx-namespace
  labels:
    app: traefiklabs
    name: nginx

spec:
  replicas: 2 # tells deployment to run 2 pods matching the template
  selector:
    matchLabels:
      app: traefiklabs
      task: nginx-task
  template:
    metadata:
      labels:
        app: traefiklabs
        task: nginx-task
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
  name: nginx-service
  namespace: nginx-namespace

spec:
  ports:
    - name: http
      port: 80
  selector:
    app: traefiklabs
    task: nginx-task
---
kind: Middleware
apiVersion: traefik.containo.us/v1alpha1
metadata:
  name: nginx-middleware
  namespace: nginx-namespace

spec:
  redirectScheme:
    scheme: https
    permanent: true
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: nginx-ingress
  namespace: nginx-namespace
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web, websecure
    traefik.ingress.kubernetes.io/router.middlewares: nginx-namespace-nginx-middleware@kubernetescrd
    traefik.ingress.kubernetes.io/router.tls.certresolver: traefikresolver

spec:
  rules:
    # Set an /etc/hosts entry to test this
    - host: www.myfakedomain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-service
                port:
                  number: 80

Apply the configuration.

kubectl apply -f ./nginx.yaml

Wait until the nginx deployment looks healthy like so.

archlinux% kubectl -n nginx-namespace get pods

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-75bd49fbcd-6f7bj   1/1     Running   0          18s
nginx-deployment-75bd49fbcd-lznb6   1/1     Running   0          18s

Test

Create an entry at /etc/hosts to allow testing the host routing for the nginx deployment. 

...

127.0.0.1 www.myfakedomain.com

Use -k with bash to ignore SSL errors if LetsEncrypt needs time or the server is not actually exposed to the Internet. 

curl -k https://www.myfakedomain.com

Bonus: authenticated images from Google Artifact Registry

Assuming an image in us-central1 then a config file at /etc/rancher/k3s/registries.yaml could be used to authenticate with a service key. 

configs:
  us-central1-docker.pkg.dev:
    auth:
      username: _json_key
      password: '{"type":"service_account","project_id":"MY PROJECT", ... "}'

Bonus: Traefik Dashboard

The Traefik dashboard can be temporarily exposed for debugging like so. Use ssh or another tunneling/port-forwarding tool as needed to access the dashboard on your network. 

ssh -L 127.0.0.1:9000:192.168.0.123:9000 admin@192.168.0.123
kubectl --namespace kube-system port-forward --address 0.0.0.0 deployment/traefik 9000:9000

Then visit http://127.0.0.1:9000/dashboard/#/http/routers

Notes