Agent on Kubernetes

The towonel-agent chart renders the Deployment, ServiceAccount and metrics Service. The exposed services are structured values (agent.services, agent.tcpServices, agent.udpServices) that the chart renders into the JSON environment variables the agent expects. It’s published to oci://codeberg.org/towonel/charts/towonel-agent, ships a values.schema.json, and pins its image by digest per release.

Layout

kubernetes/apps/network/towonel-agent/
├── ks.yaml
└── app/
    ├── kustomization.yaml
    ├── ocirepository.yaml
    ├── helmrelease.yaml
    ├── externalsecret.yaml
    └── dnsendpoint.yaml

OCIRepository

Pin ref.tag to a published chart release.

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: towonel-agent
  namespace: network
spec:
  interval: 15m
  layerSelector:
    mediaType: application/vnd.cncf.helm.chart.content.v1.tar+gzip
    operation: copy
  ref:
    tag: <chart-version>
  url: oci://codeberg.org/towonel/charts/towonel-agent

HelmRelease

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: towonel-agent
spec:
  chartRef:
    kind: OCIRepository
    name: towonel-agent
  interval: 1h
  values:
    controllers:
      main:
        replicas: 2
        annotations:
          reloader.stakater.com/auto: "true"
    agent:
      services:
        - hostname: "*.example.dev"
          origin: envoy.network.svc:443
      tcpServices:
        - name: ssh
          origin: forgejo-ssh.network.svc:2222
          listen_port: 2222
      udpServices:
        - name: turn
          origin: coturn.network.svc:3478
          listen_port: 3478
      inviteTokenSecret:
        name: towonel-agent-secret
    serviceMonitor:
      main:
        enabled: true

agent.services is SNI-routed TLS passthrough. tcpServices and udpServices are opaque; listen_port is the public edge port. The invite token is wired from agent.inviteTokenSecret; metrics serve on agent.healthPort (default 9090) with /healthz probes. The agent is stateless; each replica derives the same tenant identity from the invite seed but gets its own transport identity and registers as an independent edge session, so two replicas give redundancy on a rolling update.

Reachability

The edge reverse-dials the agent over iroh QUIC. Pod-local IPs are not reachable, so advertise something that is — an external relay (the default, hub-served) or a NodePort UDP service.

With the relay alone, no extra config is needed: the hub serves the relay URL (set TOWONEL_HUB_RELAY_URL on the hub) and the agent dials out. Override per-agent with relayUrl.

For direct paths without a relay round-trip, set directConnect.enabled. The chart then binds a fixed iroh UDP port, exposes it as a NodePort, grants the Node/Service read RBAC, mounts the API token, and turns on node autodiscovery:

directConnect:
  enabled: true
  irohPort: 51820
  nodePort: 31820 # omit to let Kubernetes assign one

• irohPort pins the iroh UDP bind to the NodePort Service’s targetPort.
• Autodiscovery queries the API for the node’s ExternalIP/InternalIP and the Service’s UDP nodePort, then advertises them as iroh external addresses. The chart wires NODE_NAME/POD_NAMESPACE from the downward API and the ClusterRole (nodes) + Role (services) it needs.
• extraLocalAddrs is a CSV of externally-reachable host:port (IPv6 in brackets) for addresses autodiscovery can’t derive.

directConnect adds a second (NodePort) Service, so auto-detection for the metrics ServiceMonitor no longer works — pin it with serviceMonitor.main.serviceName: main.

The NodePort Service selects every replica, so an edge dialing one node’s port can land on a different replica — and each replica has its own transport identity. Direct NodePort paths are most reliable with a single replica; keep multiple replicas on the relay path (the default).

If pod datapaths drop QUIC’s offloaded UDP sends, set disableUdpGso: true, or hostNetwork: true to bind the host’s UDP namespace directly.

Secret

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: towonel-agent-secret
spec:
  secretStoreRef:
    kind: ClusterSecretStore
    name: onepassword
  target:
    name: towonel-agent-secret
    creationPolicy: Owner
  data:
    - secretKey: TOWONEL_INVITE_TOKEN
      remoteRef:
        key: towonel-tunnel
        property: invite_token

Reloader rolls the pods on secret change.

DNS

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: towonel-agent
spec:
  endpoints:
    - dnsName: app.example.dev
      recordType: A
      targets: ["203.0.113.10"]