AlmaForge Network Setup

Use this page when opening ports, placing the proxy behind a load balancer, configuring TLS, or reviewing outbound firewall rules. For the conceptual explanation of how the proxy and TLS routing work, see How it works. For resource schemas, see Declarative Configuration.

For a single-host deployment, start with one public DNS name pointing at the Proxy, inbound TCP/443, and built-in Let's Encrypt. Keep diagnostics and metrics private. Allow outbound access to your IdP and any configured external storage. The Standalone guide puts these defaults together. For multiple Proxy replicas, use TCP passthrough and centrally managed certificate files.

How AlmaForge listens

AlmaForge has one primary listener: the shared TLS listener on ALMA_LISTEN (default 0.0.0.0:443). It carries the web UI, REST API, SSH, agent reverse tunnels, Auth Service gRPC, Kubernetes, and database protocols through TLS ALPN routing. This listener is bound on any host running the Proxy Service or the Auth Service. Proxy peering is the exception: it has its own listener and does not use the shared TLS listener.

Set ALMA_<X>_LISTEN when a protocol needs its own TCP socket for network segmentation, audit boundaries, or a separate load-balancer rule. Most dedicated ports add an entry point while keeping the protocol available on the shared TLS listener. MySQL and Postgres move to the dedicated port instead. See How dedicated ports change access.

There is no plaintext mode. Client traffic is TLS, apart from the two protocols that negotiate their own encryption after a plaintext opening (SSH and Postgres), and a plain HTTP request to the shared TLS listener is closed rather than redirected. Any load balancer in front of it must pass TCP through unchanged. See Load balancer configuration.

Listening ports

Shared TLS listener

Bound whenever the Proxy Service or Auth Service is enabled. Auth-only hosts expose the Auth API on this listener, not the public web services.

Port	Env var	Purpose
443	ALMA_LISTEN	Single ingress for operator and agent traffic. Client protocols share it through TLS ALPN routing. The Kubernetes SNI route is on by default.

Optional dedicated proxy listeners

Set the matching env var or CLI flag only when a protocol needs its own TCP socket. If the value omits a port, almad fills in the protocol's default port.

Env var	Default port if host-only	Purpose	Effect on :443
ALMA_SSH_LISTEN	3023	Dedicated proxy SSH listener.	Added alongside
ALMA_KUBE_LISTEN	3026	Dedicated Kubernetes proxy listener.	Added alongside
ALMA_TUNNEL_LISTEN	3024	Dedicated agent reverse tunnel listener.	Added alongside
ALMA_MYSQL_LISTEN	3036	Native MySQL wire protocol.	Replaces the :443 route
ALMA_POSTGRES_LISTEN	5432	Native Postgres wire protocol.	Replaces the :443 route
ALMA_MONGO_LISTEN	27017	Native Mongo wire protocol.	Added alongside
ALMA_PEER_LISTEN	3021	Proxy-to-proxy peering for HA proxy clusters.	Never on :443

Redis, Snowflake, Cassandra, Elasticsearch, OpenSearch, DynamoDB, and ClickHouse do not have dedicated listener ports. They use the shared TLS listener through ALPN-aware AlmaForge clients or the alma proxy db local wrapper.

How dedicated ports change access

Setting a dedicated port either adds another way to reach a protocol or moves that protocol off the shared TLS listener. The effect depends on the protocol.

ALMA_SSH_LISTEN, ALMA_KUBE_LISTEN and ALMA_TUNNEL_LISTEN add entry points. The shared TLS listener keeps serving those protocols, so both entry points stay live and either one works. Clients prefer the dedicated address once it is advertised.

The three sockets are not alike, which matters when you place a load balancer in front of one. ALMA_TUNNEL_LISTEN runs its own ALPN router and accepts alma-tunnel, the join service, and HTTP, so it needs the same L4 passthrough as the shared TLS listener. Terminating TLS in front of it stops agents registering. ALMA_SSH_LISTEN reads the first byte and splits raw SSH from TLS, and the Kubernetes port is served directly, so neither depends on ALPN surviving.

All three database ports let psql, mysql and mongo connect directly, because the dedicated socket speaks that database's own wire protocol. Once one is advertised, alma no longer needs a local proxy purely to reach the shared TLS listener, and the native client goes to the dedicated port instead. A local proxy is still started for its own reasons, per-session MFA among them.

ALMA_MYSQL_LISTEN and ALMA_POSTGRES_LISTEN move their protocols to the dedicated ports: the matching :443 route is not registered while the dedicated address is set, so :443 stops answering for those two. Publish the dedicated port to every client that needs the database.

ALMA_MONGO_LISTEN adds an entry point. Mongo uses TLS natively, so the shared TLS listener keeps serving it through the shared native-TLS database route whether or not a dedicated port is bound. Clients prefer the dedicated address, but :443 still answers.

A dedicated port is a network-segmentation tool, not an access control. Restricting who can reach it is a firewall rule, not a almad setting.

Proxy SSH direct listener

This listener binds a dedicated TCP socket on the proxy host for a protocol that may need a separate network path.

Env var	Opened when...	Purpose
ALMA_SSH_LISTEN	Proxy Service role is enabled and the env is set	Dedicated proxy SSH ingress (alma ssh raw-SSH path). Configures the Proxy Service, not the SSH Node service.

If the env var is unset, no separate listener binds. Proxy SSH remains reachable through the shared TLS listener, which is also where the Auth Service is reached.

Operator-only listeners

Off by default. Bind explicitly with the env var.

Env var	Purpose
ALMA_DIAG_LISTEN	/healthz and /readyz, plus /debug/pprof/* (Go pprof) under ALMA_DEBUG, and /metrics while ALMA_METRICS_LISTEN is unset.
ALMA_METRICS_LISTEN	Prometheus /metrics.

The diagnostic listener is plain HTTP. So is the metrics listener, until ALMA_METRICS_TLS_CERT, ALMA_METRICS_TLS_KEY, and ALMA_METRICS_CLIENT_CA turn it into mutual TLS that refuses a scraper without a client certificate. Bind both to a private interface or loopback either way. A typical metrics address is 127.0.0.1:3081.

Binding port 443 requires root or the CAP_NET_BIND_SERVICE capability on Linux. If almad runs unprivileged without that capability, startup fails with permission denied. To run unprivileged:

• Set ALMA_LISTEN=0.0.0.0:8443 (or any port ≥ 1024) and front 443 with a separate process (nginx stream, HAProxy, iptables REDIRECT).
• Grant the capability: setcap cap_net_bind_service=+ep /usr/local/bin/almad.
• Run as root.

Advertised addresses

ALMA_ADVERTISE is the canonical client-facing endpoint. It appears in SSO redirect URLs, ACME challenges, web UI command snippets, agent reverse-tunnel reconnects, and SSH host certificate SANs. Each protocol has its own advertise override: ALMA_KUBE_ADVERTISE, ALMA_TUNNEL_ADVERTISE, ALMA_SSH_ADVERTISE, ALMA_MYSQL_ADVERTISE, ALMA_POSTGRES_ADVERTISE, ALMA_MONGO_ADVERTISE, ALMA_PEER_ADVERTISE. Set a per-protocol advertise value only when that protocol has a different public hostname or port. Otherwise, protocols use ALMA_ADVERTISE.

For the full env-var and CLI surface, see Environment Configuration: Listen and Advertise Addresses and CLI Reference: Networking.

Firewall rules

Inbound to the proxy host:

Source	Port	Why
Operator workstations	443/TCP	Web UI, alma CLI, kubectl, database clients.
Agent hosts (SSH, DB, App, Kube)	443/TCP	Outbound reverse tunnel registration and session traffic.
Let's Encrypt validation servers	443/TCP	ACME challenge (if ACME is enabled).
operator-defined	varies	Each ALMA_<X>_LISTEN you bind opens its own inbound port.

Inbound to the auth host (only when auth runs on separate hosts):

Source	Port	Why
Proxy Service hosts	443/TCP	Auth gRPC API on the shared TLS listener, mTLS-protected.

Outbound from agent hosts:

Destination	Port	Why
Proxy ALMA_ADVERTISE	443/TCP	Reverse tunnel registration, heartbeats, session traffic.

Agents (Database Service, Application Service, Kubernetes Service) open no inbound listeners. Nothing else needs to reach them, including the Auth Service and the storage backend.

Constraints

• Use L4 TCP passthrough in front of the shared TLS listener. Anything that terminates TLS, including an L4 load balancer configured with a TLS listener, strips the ClientHello ALPN value and breaks direct ALPN routing. Compatible clients have an HTTP connection-upgrade path, with additional requirements. See load balancer configuration.
• DNS must resolve ALMA_ADVERTISE to the proxy host (or to the L4 load balancer in front of it).
• Application DNS must resolve each application's published hostname to the Proxy. A wildcard record (*.$DOMAIN) covers application subdomains, alongside the base $DOMAIN record.

Load balancer configuration

Direct ALPN routing has one hard requirement: the load balancer must not terminate TLS. The TLS ClientHello, including the ALPN list, must reach AlmaForge intact. If the load balancer decrypts and re-encrypts the connection, the original ALPN value is lost. The proxy sees only h2 or http/1.1 instead of the client's requested protocol. Keep TCP passthrough as the default. See HTTP connection upgrade for the separate client-assisted path through a compatible HTTP proxy.

L4 is necessary but not sufficient

"L4 load balancer" is not the same thing as "TCP passthrough". An AWS NLB with a TLS listener, an HAProxy frontend with ssl on the bind directive, or any other listener that decrypts and re-encrypts traffic still strips the ClientHello. The load-balancer listener must be TCP, not TLS. If the configuration mentions a certificate or an ACM ARN, the load balancer is terminating TLS.

Direct exposure

The simplest deployment has no load balancer. A single proxy host with a public IP and an OS-level firewall (firewalld, nftables, or a cloud security group) is enough for a small or mid-sized cluster. Open 443/TCP to the world. Leave metrics and diagnostics closed unless you bind them on a private interface.

This is the recommended starting point. Add a load balancer only when you need horizontal scale-out, multi-AZ redundancy, or a dedicated DoS mitigation layer.

AWS Network Load Balancer

• Listener: TCP, port 443 (not TLS).
• Target group: TCP, port 443, target type instance or ip.
• Health check: HTTP GET /readyz on the private diagnostics port configured by ALMA_DIAG_LISTEN. A TCP check on 443 checks only that the listener accepts connections, not service readiness.
• TLS termination: off. Do not attach a certificate.
• Cross-zone load balancing: on.
• Preserve client IP: on (the default for NLB instance targets), or enable PROXY protocol v2 on the target group. See PROXY protocol.

HAProxy

HAProxy

frontend almaforge_in
    bind *:443
    mode tcp
    option tcplog
    default_backend almaforge_proxies

backend almaforge_proxies
    mode tcp
    balance roundrobin
    option tcp-check
    server proxy1 10.0.1.10:443 check send-proxy-v2
    server proxy2 10.0.1.11:443 check send-proxy-v2

Required: mode tcp and no ssl keyword on the server lines. send-proxy-v2 is optional but recommended. Pair it with ALMA_REAL_IP_FROM naming the HAProxy hosts on the AlmaForge side, otherwise the headers it sends are ignored.

Kubernetes Service

For a managed Kubernetes cluster, use a Service of type LoadBalancer with a cloud annotation that requests an L4 LB (NLB on AWS, TCP LB on GCP, Standard LB on Azure):

load-balancer-service.yaml:

examples/networking/load-balancer-service.yaml

# Kubernetes LoadBalancer Service for the shared AlmaForge ingress port.
apiVersion: v1
kind: Service
metadata:
  name: almaforge-proxy
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: 'nlb'
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: ingress
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app: almaforge-proxy

externalTrafficPolicy: Local preserves the client IP. For self-hosted Kubernetes, use MetalLB or kube-vip in L2 or BGP mode.

nginx (stream module)

Only the stream module works. http terminates TLS.

nginx

stream {
    upstream almaforge_proxies {
        server 10.0.1.10:443;
        server 10.0.1.11:443;
    }

    server {
        listen 443;
        proxy_pass almaforge_proxies;
        proxy_protocol on;
    }
}

Do not configure proxy_ssl_* directives or a certificate on the nginx side.

What does not work

• AWS Application Load Balancer (ALB). It terminates TLS and replaces the client's ALPN with h2 / http/1.1, preventing direct protocol routing.
• GCP HTTPS Load Balancer. Same problem.
• Any nginx, Traefik, Envoy, Apache, or Caddy deployment with TLS termination enabled. Web UI traffic can work while direct ALPN connections for SSH, databases, Auth gRPC, and agent tunnels fail.
• CDNs and L7 HTTPS proxies. See Cloudflare and other CDNs below.

These constraints apply to direct ALPN connections. An HTTP proxy must also support the connection-upgrade path before AlmaForge-aware clients can tunnel those connections through it.

Cloudflare and other CDNs

Cloudflare's default "orange cloud" mode terminates TLS at the edge, applies WAF and caching rules, then opens a fresh TLS connection to the origin with standard h2 / http/1.1 ALPN values. SSH, databases, Auth gRPC, agent reverse tunnels, and Kubernetes fail because their custom ALPN values never reach the proxy. The web UI keeps working, which makes this failure easy to misread.

Two supported ways to use Cloudflare with AlmaForge:

1. Disable the Cloudflare HTTPS proxy for the AlmaForge hostname. Set the DNS record to "DNS only" (grey cloud). DNS still resolves through Cloudflare, but client connections go straight to the origin and the original ALPN survives. This is the recommended setup and is what Requirements tells new users to do.
2. Use Cloudflare Spectrum in TCP mode. Spectrum is a true L4 product that forwards TCP bytes without touching TLS. ALPN survives, and every protocol works. Spectrum is a paid Cloudflare feature and is the only Cloudflare product compatible with multi-protocol ALPN routing.

Other CDNs (Fastly, Akamai, Amazon CloudFront, Google Cloud CDN, Azure Front Door) have the same constraint. If the product description mentions "TLS termination at the edge", "HTTP/2 to origin", or "WAF" without a true TCP passthrough option, it will break everything except the web UI. Use a non-CDN L4 load balancer such as AWS NLB, GCP Network LB, or Azure Standard LB. A dedicated TCP-passthrough product also works.

HTTP connection-upgrade fallback

If L4 passthrough is not available, AlmaForge-aware clients can tunnel ALPN traffic through an HTTP/1.1 connection upgrade: GET /api/v1/connectionupgrade with Upgrade: alpn or Upgrade: alpn-ping. This lets the cluster sit behind an L7 proxy that only forwards HTTP.

Caveats:

• It is a fallback, not the recommended path. L4 TCP passthrough is still what production clusters should use.
• Only AlmaForge-aware clients (alma, agents) know how to open the upgrade. Native clients (psql, mysql, kubectl, stock ssh) still need an alma proxy <...> local proxy to bridge the gap.
• The front end must forward Upgrade headers without rewriting and hold long-lived connections open. Cloudflare's standard proxy is unreliable for this.
• Each connection pays an extra HTTP round trip.

Verify the path with:

Terminal

curl -v https://proxy.example.com/api/v1/connectionupgrade \    -H "Connection: Upgrade" \    -H "Upgrade: alpn-ping" \    --no-alpn

A working path returns HTTP/1.1 101 Switching Protocols and emits binary keepalive frames for ~30 seconds.

PROXY protocol

When AlmaForge sits behind an L4 load balancer, the proxy sees the load balancer's IP as the connection source instead of the real client. The PROXY protocol (HAProxy's wire format) prepends the original client IP and port so AlmaForge can attribute audit events, enforce IP pinning, and apply rate limits.

almad accepts PROXYv1 and PROXYv2, over IPv4 and IPv6.

A PROXY header carries no proof of who wrote it. Anyone who can open a TCP connection can prepend one and name any source address they like, so two things have to be true before AlmaForge believes one.

Who may speak for another address. ALMA_REAL_IP_FROM lists the networks, as CIDRs or bare addresses in either family:

env

ALMA_REAL_IP_FROM=10.0.1.0/24,2001:db8:1::/64

Set it to the addresses your load balancer connects from, which is its private-subnet address rather than the public address clients reach. Until you set it, no client-supplied address is believed anywhere.

Which channel they may speak through. ALMA_REAL_IP_HEADER names one:

Value	Address comes from
proxy_protocol	Default. The PROXY protocol header.
X-Forwarded-For	The X-Forwarded-For header. Web listener only.
X-Real-IP	The X-Real-IP header. Web listener only.
off	Nothing, while leaving the network list configured.

Exactly one is consulted. This is the part that matters: an L4 load balancer cannot strip a client's HTTP headers, so if a header were read as well as the PROXY line, a client behind your balancer could set its own address through the channel you did not choose. Naming one source closes that off. It is the same model as nginx's set_real_ip_from and real_ip_header.

A line from an unlisted network, or through a channel you did not name, is read off the wire, logged, and discarded. The connection is served on the address it really came from.

Restrict the Proxy listener to the load balancer's source networks when all clients use that entry point. The same load balancer can forward SSH, tunnel, and database traffic through the shared listener. If you also allow direct connections, clients outside ALMA_REAL_IP_FROM cannot use forwarded-address headers to override their observed source address.

Whether a header is required at all is a third, separate question, answered by ALMA_PROXY_PROTOCOL:

Value	Behavior
unset	Default. A PROXY header is allowed but not required.
on	Every connection must carry exactly one PROXY header, or it is rejected.
off	Any connection carrying a PROXY header is rejected.

Rules of thumb:

• Behind an L4 LB that sends PROXY headers: set ALMA_REAL_IP_FROM to the balancer's network. The default source already reads the PROXY line, so that is the only setting you need. Add ALMA_PROXY_PROTOCOL=on if every connection reaches you through the balancer and you want a missing header treated as an error.
• Behind an L7 LB that terminates HTTPS: set ALMA_REAL_IP_FROM and ALMA_REAL_IP_HEADER=X-Forwarded-For.
• Directly exposed on a public IP, no balancer: leave all three unset. ALMA_PROXY_PROTOCOL=off additionally rejects the connection outright instead of ignoring the header, which is useful if you would rather see the attempt fail loudly.

Enable PROXY protocol on the load balancer side to match:

• AWS NLB: target group setting Proxy protocol v2: enabled.
• HAProxy: send-proxy-v2 on each server line in the backend (shown in the HAProxy snippet).
• nginx (stream): proxy_protocol on; inside the server block (shown in the nginx snippet).

Match the header policy to the load balancer. ALMA_PROXY_PROTOCOL=on rejects a connection without a header, and off rejects one with a header. With the default permissive policy, an untrusted header is ignored rather than used as the client address.

X-Forwarded-For

An L7 load balancer that terminates the web listener writes the client's address into a header instead. Name the network it connects from and the header it writes:

env

ALMA_REAL_IP_FROM=10.0.1.0/24
ALMA_REAL_IP_HEADER=X-Forwarded-For

Use X-Real-IP instead if that is what your balancer sets. The header is read on requests arriving from those networks and ignored on every other request. Naming a header also stops the PROXY protocol from setting the address, so the two can never disagree.

X-Forwarded-For carries a comma-separated chain that each hop appends to, and the last entry is the address the hop that spoke to us observed. That is the one taken, which means an appending balancer such as an AWS ALB works without further configuration: whatever a client puts in the header ends up to the left of the address the balancer saw, and is ignored.

This applies only to the web listener. The shared TLS listener and non-web protocols use the TCP source address, or the PROXY header when one is believed, and never read HTTP headers.

Web Application Firewalls

A Web Application Firewall inspects HTTP/1.1 and HTTP/2 request streams. That works for a plain HTTPS application. It does not work for a multi-protocol access plane.

Why WAFs do not work on the shared TLS listener

The shared TLS listener on ALMA_LISTEN carries SSH, databases, Auth gRPC, Kubernetes, and agent reverse tunnels through TLS ALPN on port 443. A WAF that tries to parse those flows as HTTP breaks in two ways:

• TLS termination strips ALPN. A WAF has to terminate the TLS handshake to inspect the plaintext payload. The moment it does, the custom alma-* ALPN values are lost and direct protocol routing fails. This is the same failure mode described in What does not work and Cloudflare and other CDNs.
• Non-HTTP payloads confuse HTTP rules. Even if a WAF could be coaxed into forwarding the bytes untouched, the binary wire format of Postgres, MySQL, MongoDB, or SSH is not HTTP. WAF rulesets that block "suspicious payloads" will flag legitimate database traffic as an attack and drop it.

The shared TLS listener already has mutual TLS, short-lived certificates, protocol-aware session recording, and RBAC in the proxy. A WAF in front of it adds little and breaks the routing model.

Network segmentation with dedicated listeners

If a protocol needs separate network controls, use its _LISTEN/_ADVERTISE pair to put it on a dedicated TCP listener. You can then apply L4 policy to that port independently of the shared TLS listener: firewall rules, dedicated load balancers, and allowlisted source ranges.

env

# Shared TLS listener on :443 carries everything by default
ALMA_LISTEN=0.0.0.0:443
ALMA_ADVERTISE=access.example.com:443

# Add a Kubernetes entry point for separate network policy
ALMA_KUBE_LISTEN=0.0.0.0:3026
ALMA_KUBE_ADVERTISE=kube.example.com:3026

The SSH, Kubernetes and reverse-tunnel ports add entry points, so the shared TLS listener still serves those protocols through ALPN/SNI for clients that can only reach ALMA_ADVERTISE. Clients prefer the dedicated address when one is advertised. Only the MySQL and Postgres ports move their protocol off :443 entirely. See How dedicated ports change access.

A WAF is only useful in front of HTTP-speaking traffic. Dedicated SSH, database, reverse-tunnel, and peer ports carry non-HTTP protocols. An HTTP WAF in their path will reject or mangle the traffic. Treat dedicated listeners as a network-segmentation tool, not as HTTP WAF coverage for non-HTTP protocols.

TLS certificates

TLS is mandatory. Configure ACME or supply certificate files for the Proxy. If neither is configured, it generates a self-signed certificate that clients must trust explicitly. The environment variables shown below are documented in Environment Configuration: TLS and Certificates.

ACME (Let's Encrypt)

Use ACME for clusters with a public DNS name. Enable it and AlmaForge handles issuance and renewal through the TLS-ALPN-01 challenge on port 443.

Let's Encrypt validates the challenge by connecting to your domain on port 443. The name must resolve publicly, and its validation servers must be able to reach the port. This does not work for a cluster on a private network, behind a CDN that terminates TLS, or on a name that does not resolve from the internet. Use operator-supplied certificates in those environments and manage renewal on your own schedule.

The DNS-01 challenge uses a TXT record instead of an inbound connection. It is not supported today.

Environment variables (recommended):

env

ALMA_ACME_ENABLED=true
[email protected]
ALMA_ADVERTISE=proxy.example.com:443

CLI flags:

Terminal

almad start --acme [email protected] \        --advertise=proxy.example.com:443

systemd unit (EnvironmentFile):

systemd

[Service]
EnvironmentFile=/etc/almaforge/almad.env
ExecStart=/usr/local/bin/almad

with /etc/almaforge/almad.env containing the three variables above.

Prerequisites:

• A DNS A or AAAA record points at the proxy's public IP (or at an L4 load balancer that forwards to the proxy), resolvable from the public internet.
• Port 443/TCP is reachable from Let's Encrypt's public validation servers, so they can complete the inbound challenge.
• Outbound HTTPS to acme-v02.api.letsencrypt.org is allowed from the proxy host, which is how the certificate is ordered and collected once the challenge passes.
• ALMA_ADVERTISE matches the DNS record so the issued certificate's SAN covers it.
• For a wildcard-style deployment (per-resource hostnames), both $DOMAIN and *.$DOMAIN resolve to the proxy.

Renewals happen automatically before expiry. If a renewal fails, AlmaForge logs the error and keeps serving the old certificate until it expires. Common causes are public port 443 becoming unreachable, DNS moving without a matching ALMA_ADVERTISE update, or a CDN terminating TLS in front of the proxy.

Operator-supplied certificates

For private, air-gapped, or policy-restricted deployments, supply a certificate and key pair directly.

Environment variables (recommended):

env

ALMA_TLS_CERT=/etc/almaforge/tls/fullchain.pem
ALMA_TLS_KEY=/etc/almaforge/tls/privkey.pem
ALMA_ADVERTISE=proxy.example.com:443

CLI flags:

Terminal

almad start --tls-cert=/etc/almaforge/tls/fullchain.pem \        --tls-key=/etc/almaforge/tls/privkey.pem \        --advertise=proxy.example.com:443

By default, almad loads these files at startup. Replace the certificate and matching key, then roll the Proxy hosts to use them. To reload without restarting, set ALMA_TLS_RELOAD_INTERVAL to a positive duration such as 1m before renewal. See the TLS environment reference. The certificate must cover ALMA_ADVERTISE. For wildcard deployments it should also cover every published database, application, and Kubernetes hostname.

Self-signed certificates work for lab testing, but every client must trust them out of band. Avoid them when testing network behavior because trust errors can hide real configuration problems.

Egress traffic

These flows are outbound-only from the proxy or auth hosts. None of them require inbound access.

Do not mistake the browser redirect for the entire SSO flow. The Auth Service also connects directly to the provider. It reads the discovery document and signing keys from the issuer URL, then exchanges the authorization code for tokens on every sign-in. Login fails if an Auth host cannot reach the issuer, even when the browser redirect succeeds.

Destination	Port	Reason
acme-v02.api.letsencrypt.org	443/TCP	ACME order creation and certificate retrieval (if ACME enabled).
Cluster state backend	varies	State backend reads and writes. See Storage Configuration.
Audit backend	varies	Audit event writes. See Storage Configuration.
Session recording backend	varies	Recording uploads. See Storage Configuration.
OIDC identity provider issuer URL	443/TCP	Discovery, signing key fetch, and code-to-token exchange on every sign-in. Required once an SSO connector exists.
cloudidentity.googleapis.com	443/TCP	Google Workspace group lookup (only when a Google connector is configured with a service account).
Slack webhook / API	443/TCP	Access-request notifications (if Slack plugin enabled).
Jira REST API	443/TCP	Access-request notifications (if Jira plugin enabled).
PagerDuty API	443/TCP	Access-request notifications (if PagerDuty plugin enabled).
Telegram Bot API	443/TCP	Access-request notifications (if Telegram plugin enabled).

Agent hosts need outbound access to the proxy address they are configured to use. In the common case, this is ALMA_ADVERTISE on 443. If you publish a dedicated tunnel address, allow that address instead.

Egress through an HTTP proxy

Sites that force outbound traffic through a corporate HTTP proxy can route AlmaForge through it too. almad and alma honor the standard environment variables:

Variable	Purpose
HTTPS_PROXY	Egress proxy for HTTPS connections.
HTTP_PROXY	Egress proxy for plaintext HTTP (rare).
NO_PROXY	Comma-separated list of hosts and CIDRs to bypass the proxy.

The HTTP proxy must be reachable from the process using it. A loopback address refers to that process's own host or container network, not to an administrator's workstation. When almad runs as a systemd unit, set the variables in the unit file:

systemd

[Service]
Environment="HTTPS_PROXY=http://egress.corp.example:8080/"
Environment="NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"

Observability

ALMA_DIAG_LISTEN and ALMA_METRICS_LISTEN are off by default. When set, they bind to the configured address. A common choice is 127.0.0.1:3081 for metrics and a separate loopback port for diagnostics. Diagnostics is plain HTTP, and so is metrics until the three metrics TLS variables turn on mutual TLS, so bind both to a private interface.

For endpoint details, TLS / mTLS configuration, and the full metric reference, see Metrics.

Troubleshooting

Symptom → cause

Symptom	Most likely cause
Web UI works, but alma ssh, alma db, or kubectl fail immediately	An L7 proxy in front of almad is stripping custom ALPN. Use L4 TCP passthrough or Cloudflare Spectrum.
alma db connect fails with "TLS handshake failure"	An L7 proxy is rewriting ALPN. The web UI works because it uses standard h2 / http/1.1.
Agent reverse tunnels never come up	Outbound 443 from the agent host to the proxy is blocked, or ALMA_ADVERTISE does not resolve from the agent.
ACME issuance fails on first start	Public DNS wrong, port 443 unreachable from Let's Encrypt, or a CDN between Let's Encrypt and almad is terminating TLS.
Audit events show the load balancer's IP, not the real client	ALMA_REAL_IP_FROM does not cover the address the balancer connects from, ALMA_REAL_IP_HEADER names a channel the balancer does not use, or the balancer is not sending the header at all.
Logs report a PROXY line was ignored	The connection did not arrive from a network in ALMA_REAL_IP_FROM, or ALMA_REAL_IP_HEADER names a header rather than proxy_protocol.
IP pinning passes for everyone behind an LB	Certificates are pinned to the address observed at login. If that is the balancer, every certificate pins to the balancer and the check compares one balancer address against another. Set ALMA_REAL_IP_FROM and ALMA_REAL_IP_HEADER so the real client address is the one observed.
kubectl returns x509 errors against the cluster	An L7 proxy is rewriting SNI. Same fix as the ALPN cases.
almad refuses to start with "permission denied: bind 443"	Running unprivileged without CAP_NET_BIND_SERVICE. setcap, run as root, or use 8443 with a front-end on 443.

Quick checks

Check the public HTTPS path first:

Terminal

curl -s https://proxy.example.com/api/v1/ping | jq

If this fails, the problem is DNS, firewall, or TLS, not application configuration inside almad.

For ALPN-specific issues, such as a database client failing while the web UI works, capture the handshake:

Terminal

openssl s_client -connect proxy.example.com:443 \    -alpn alma-postgres -servername proxy.example.com < /dev/null

If the handshake fails with alert: no application protocol, an L7 proxy in the path is stripping the ALPN value.

For a listener inventory on the proxy host:

Terminal

sudo ss -ltnp | grep "almad"LISTEN 0  4096  *:443           *:*   users:(("almad",pid=1234,fd=8))LISTEN 0  4096  127.0.0.1:3081  *:*   users:(("almad",pid=1234,fd=9))

Expect the shared TLS listener on 443, one socket per configured ALMA_<X>_LISTEN, and optional ALMA_DIAG_LISTEN / ALMA_METRICS_LISTEN sockets.

See also

• Requirements: prerequisite checklist for a fresh install.
• Environment Configuration: full env-var reference, including listen and advertise addresses.
• CLI Reference: full CLI flag reference.
• Declarative Configuration: resource schema reference.
• Storage Configuration: backend-specific outbound network requirements.
• Metrics: full reference for the 3081 endpoint.