Agent on Kubernetes

Layout

kubernetes/apps/network/towonel-agent/
├── ks.yaml
└── app/
    ├── kustomization.yaml
    ├── ocirepository.yaml
    ├── helmrelease.yaml
    ├── rbac.yaml
    ├── externalsecret.yaml
    └── dnsendpoint.yaml

HelmRelease

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: towonel-agent
spec:
  chartRef:
    kind: OCIRepository
    name: towonel-agent
  interval: 1h
  values:
    defaultPodOptions:
      automountServiceAccountToken: true
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        fsGroup: 10001
        seccompProfile: { type: RuntimeDefault }
    controllers:
      towonel-agent:
        type: deployment
        # NodePort UDP load-balances per packet; iroh state is per-pod.
        replicas: 1
        annotations:
          reloader.stakater.com/auto: "true"
        serviceAccount:
          name: towonel-agent
        containers:
          app:
            image:
              repository: codeberg.org/towonel/towonel-agent
              tag: latest
            env:
              RUST_LOG: info
              TOWONEL_AGENT_HEALTH_LISTEN_ADDR: 0.0.0.0:9090
              TOWONEL_AGENT_IROH_PORT: "51820"
              TOWONEL_AGENT_K8S_AUTODISCOVER: "true"
              TOWONEL_AGENT_K8S_SERVICE: towonel-agent
              TOWONEL_AGENT_SERVICES: |
                [{"hostname":"*.example.dev","origin":"envoy.network.svc:443"}]
              TOWONEL_AGENT_TCP_SERVICES: |
                [{"name":"ssh","origin":"forgejo-ssh.network.svc:2222","listen_port":2222}]
              TOWONEL_AGENT_UDP_SERVICES: |
                [{"name":"turn","origin":"coturn.network.svc:3478","listen_port":3478}]
              NODE_NAME:
                valueFrom: { fieldRef: { fieldPath: spec.nodeName } }
              POD_NAMESPACE:
                valueFrom: { fieldRef: { fieldPath: metadata.namespace } }
              TOWONEL_INVITE_TOKEN:
                valueFrom:
                  secretKeyRef:
                    name: towonel-agent-secret
                    key: TOWONEL_INVITE_TOKEN
            probes:
              liveness:  { enabled: true, custom: true, spec: { httpGet: { path: /healthz, port: 9090 } } }
              readiness: { enabled: true, custom: true, spec: { httpGet: { path: /healthz, port: 9090 } } }
            securityContext:
              allowPrivilegeEscalation: false
              readOnlyRootFilesystem: true
              capabilities: { drop: ["ALL"] }
            resources:
              requests: { cpu: 50m, memory: 64Mi }
              limits:   { cpu: 500m, memory: 64Mi }
    service:
      app:
        ports:
          http: { port: 9090 }
      iroh:
        controller: towonel-agent
        type: NodePort
        ipFamilyPolicy: RequireDualStack
        ipFamilies: [IPv4, IPv6]
        ports:
          iroh:
            port: 51820
            targetPort: 51820
            nodePort: 31820
            protocol: UDP
    serviceMonitor:
      app:
        endpoints:
          - { port: http, interval: 1m }

TOWONEL_AGENT_SERVICES is SNI-routed TLS passthrough. _TCP_SERVICES and _UDP_SERVICES are opaque; listen_port is the public edge port.

Reachability

The edge reverse-dials the agent over iroh QUIC. Pod-local IPs are not reachable, so advertise something that is — a NodePort UDP service, an external relay, or both.

• TOWONEL_AGENT_IROH_PORT pins the iroh UDP bind to the Service’s targetPort (required for NodePort).
• TOWONEL_AGENT_K8S_AUTODISCOVER=true queries the API for the node’s ExternalIP/InternalIP and the Service’s UDP nodePort, then advertises them as iroh external addresses. Needs NODE_NAME and POD_NAMESPACE from the downward API.
• TOWONEL_AGENT_K8S_SERVICE (default towonel-agent) names the NodePort Service to read.
• TOWONEL_AGENT_RELAY_URL overrides the hub-served relay (set TOWONEL_HUB_RELAY_URL on the hub for the default).
• TOWONEL_AGENT_EXTRA_LOCAL_ADDRS is a CSV of externally-reachable host:port (IPv6 in brackets) when autodiscovery doesn’t apply.

RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: towonel-agent
  namespace: network
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: towonel-agent-discover-nodes
rules:
  - { apiGroups: [""], resources: [nodes], verbs: [get] }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: towonel-agent-discover-nodes
roleRef: { apiGroup: rbac.authorization.k8s.io, kind: ClusterRole, name: towonel-agent-discover-nodes }
subjects:
  - { kind: ServiceAccount, name: towonel-agent, namespace: network }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: towonel-agent-discover-services
  namespace: network
rules:
  - { apiGroups: [""], resources: [services], verbs: [get] }
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: towonel-agent-discover-services
  namespace: network
roleRef: { apiGroup: rbac.authorization.k8s.io, kind: Role, name: towonel-agent-discover-services }
subjects:
  - { kind: ServiceAccount, name: towonel-agent, namespace: network }

Nodes are cluster-scoped; services are namespaced.

Secret

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: towonel-agent-secret
spec:
  secretStoreRef:
    kind: ClusterSecretStore
    name: onepassword
  target:
    name: towonel-agent-secret
    creationPolicy: Owner
  data:
    - secretKey: TOWONEL_INVITE_TOKEN
      remoteRef:
        key: towonel-tunnel
        property: invite_token

Reloader rolls the pods on secret change.

DNS

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: towonel-agent
spec:
  endpoints:
    - dnsName: app.example.dev
      recordType: A
      targets: ["203.0.113.10"]