Credential Export
Short-lived certificates expire by design, but many database clients, automation tools, and services cannot request replacements. They only know how to read credential files from stable paths. Without a renewal process, those clients stop connecting when the certificate they loaded expires.
alma export handles the file renewal. It renders credentials at
startup, writes them to a directory or Kubernetes Secret, and repeats the
render on a configured interval. A client that reads the files for each new
connection sees the current credentials. A long-running client that keeps an
in-memory copy must reload the files or restart before its copy expires.
The Export file describing what to write stays on the client and is never
stored in the cluster. Only the rendered credentials reach the directory or
Kubernetes Secret you configure.
Credential Export is experimental. Its configuration fields, the file names it writes, and the commands it renders may change in a future release.
Before you begin
- Install
almaon the host or in the pod where the exporter will remain running. - Select an authenticated identity. A person can use
alma login. A machine authenticates through workload identity instead. - Grant that identity access to every resource named by an output. If an
output sets
roles, the identity also needs role impersonation permission for every named role. - Give the process write access to each destination. A Kubernetes Secret has additional namespace and authentication requirements described under Destinations.
Export an identity to a directory
Save this Export file as export.yaml. The example writes an identity bundle
to /tmp/almaforge-export. Use a workload-owned absolute path for a long-lived
deployment.
# Client-local export configuration consumed by `alma export -c`.
# This resource is not stored in the cluster backend.
apiVersion: almaforge.com/v1
kind: Export
metadata:
name: local-credentials
spec:
outputs:
- type: identity
destination:
type: directory
path: /tmp/almaforge-export
Confirm that alma has selected the intended identity and cluster:
Start the export process:
The command stays in the foreground after the first render. In another terminal, confirm that the files exist:
Use the generated SSH configuration to test the identity. The
generated ssh_config matches Host *.<cluster-name>, so address the
target as <node>.<cluster-name> and substitute a node you can reach:
The first render has succeeded when the files appear and the SSH command reaches the node.
Authenticate as a machine
The exporter uses the identity selected in the local profile. A person can run
alma login before starting it. An unattended exporter cannot
complete an interactive login, so it must use an identity its platform already
provides.
Set the workload identity variables in the process environment and start the exporter. It joins on startup, so there is no interactive login step and no long-lived secret on disk:
ALMA_CLIENT_ID names a Token in the cluster, and ALMA_JOIN_METHOD picks
what the machine proves: an AWS caller identity, a GCP or Azure instance
document, a Kubernetes ServiceAccount JWT, or a CI provider's OIDC token.
The join methods section covers the
allow rules for each one.
On the admin side, create the roles the machine will use, apply the Token
that pins its platform identity, then bind a bot to that token:
Every output rendered by a machine must set roles. A bot's own role only
allows it to assume the roles assigned when the bot was created. Leaving
roles empty therefore renders a certificate that does not grant access to a
resource.
alma keeps its profile under $XDG_CONFIG_HOME/almaforge, falling
back to $HOME/.config/almaforge. A service account or container user without
a home directory must set XDG_CONFIG_HOME to a writable path, or the first
render fails before it reaches the cluster.
Export database credentials for a scheduled job
A scheduled psql job illustrates the fixed-path problem. The exporter keeps
the mTLS files current at a path the job already knows, and each new psql
process reads them when it connects. No alma process sits in the
database connection path.
# Keeps a live PostgreSQL client certificate on disk for a scheduled
# job. `format: tls` writes tls.crt, tls.key and tls.cas, which is the
# layout psql and most database drivers already expect.
apiVersion: almaforge.com/v1
kind: Export
metadata:
name: orders-postgres
spec:
# Renewal defaults to a third of this, so the files are rewritten
# every 10 minutes.
certificateTTL: 30m
outputs:
- type: database
# A machine identity carries no access of its own, only the right
# to assume the roles it was created with. Leave this empty and
# the rendered certificate opens nothing.
roles: [db-readonly]
# The DatabaseTarget name, the database inside it, and the
# database account to connect as.
service: orders-postgres
database: orders
username: reporting
format: tls
destination:
type: directory
path: /var/lib/almaforge-export/orders-postgres
format: tls adds tls.crt, tls.key and tls.cas to the destination
alongside the identity bundle. Point psql at all three:
The port is 443 unless the Proxy has a dedicated PostgreSQL listener. See
PostgreSQL access for how the Proxy
routes a native client, and for the interactive equivalent of this flow.
Other engines use the same output with a different format. MongoDB reads
mongo.crt and mongo.cas from format: mongo, and CockroachDB reads the
cockroach/ subdirectory from format: cockroach. Clients that consume a
generic identity bundle need no format at all.
The setup is working when the query succeeds before and after at least one
certificateTTL period.
Run the exporter as a service
The exporter is a foreground process that must remain active while credentials are needed. Run it under a supervisor that restarts it, and use a destination the consuming workload can read.
systemd
Save the Export file as /etc/almaforge/export.yaml and the unit as
/etc/systemd/system/alma-export.service:
[Unit]
Description=AlmaForge credential export
After=network-online.target
[Service]
ExecStart=/usr/local/bin/alma export -c /etc/almaforge/export.yaml
Environment=XDG_CONFIG_HOME=/var/lib/almaforge-export
EnvironmentFile=-/etc/almaforge/export.env
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target
Packages install alma to /usr/bin and the tarball installer uses
/usr/local/bin, so point ExecStart at whichever
command -v alma reports.
For a machine identity, put the workload identity variables in the referenced environment file:
ALMA_PROXY=almaforge.example.com:443
ALMA_CLIENT_ID=orders-report
ALMA_JOIN_METHOD=aws
Start the service and follow the journal until the first render completes:
A successful start logs one Rendered output line for each output. Request an
immediate render before the next interval with
sudo systemctl kill -s HUP alma-export.
Kubernetes
In a pod, run the exporter as a sidecar. It writes into an emptyDir that the
application container mounts read-only. The sidecar updates the files, while
the application remains responsible for rereading them after a renewal.
Apply the Token and create the bot first:
# Lets the orders-report pod join as the orders-report bot. The pod
# proves identity with the ServiceAccount JWT Kubernetes already mounts,
# so the Deployment holds no secret of its own.
apiVersion: almaforge.com/v1
kind: Token
metadata:
name: orders-report # ALMA_CLIENT_ID value in the pod.
spec:
roles:
- system:bot
# Required whenever roles include system:bot. Create the bot with
# `alma create bot orders-report --roles=db-readonly
# --token-name=orders-report`.
botName: orders-report
kubernetes:
type: in_cluster
allow:
- serviceAccount: almaforge-export:orders-report # namespace:name.
Then apply the workload:
# Runs `alma export` beside an application container. The two share an
# emptyDir, so the application reads certificates from a fixed path
# while the sidecar rewrites them on each renewal interval.
#
# Apply bot-token.yaml to the cluster first, and create the bot it
# names. The pod then joins with its ServiceAccount JWT and needs no
# secret of its own.
apiVersion: v1
kind: Namespace
metadata:
name: almaforge-export
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: orders-report
namespace: almaforge-export
---
apiVersion: v1
kind: ConfigMap
metadata:
name: orders-report-export
namespace: almaforge-export
data:
export.yaml: |
apiVersion: almaforge.com/v1
kind: Export
metadata:
name: orders-report
spec:
certificateTTL: 30m
outputs:
- type: database
roles: [db-readonly]
service: orders-postgres
database: orders
username: reporting
format: tls
destination:
type: directory
path: /var/run/almaforge-export
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: orders-report
namespace: almaforge-export
spec:
# Each pod gets its own emptyDir, so replicas do not share files. They
# do each issue a full set of certificates on every renewal interval,
# which is why this stays at one.
replicas: 1
selector:
matchLabels:
app.kubernetes.io/instance: orders-report
template:
metadata:
labels:
app.kubernetes.io/name: orders-report
app.kubernetes.io/instance: orders-report
spec:
serviceAccountName: orders-report
automountServiceAccountToken: true
containers:
- name: export
image: quay.io/almaforge/almaforge:latest
command: ['alma', 'export', '-c', '/etc/almaforge/export.yaml']
env:
- name: ALMA_PROXY
value: almaforge.example.com:443
- name: ALMA_CLIENT_ID
value: orders-report
- name: ALMA_JOIN_METHOD
value: kubernetes
# alma keeps its own profile under $XDG_CONFIG_HOME/almaforge.
# Point it at a writable volume so the render loop does not
# depend on a home directory existing in the image.
- name: XDG_CONFIG_HOME
value: /var/run/almaforge-profile
volumeMounts:
- name: credentials
mountPath: /var/run/almaforge-export
- name: profile
mountPath: /var/run/almaforge-profile
- name: config
mountPath: /etc/almaforge
readOnly: true
resources:
requests:
cpu: '50m'
memory: '64Mi'
limits:
memory: '128Mi'
# Stand-in for the workload that consumes the credentials. Any
# image that reads certificates from a fixed path works.
- name: report
image: postgres:16
command: ['sleep', 'infinity']
volumeMounts:
- name: credentials
mountPath: /var/run/almaforge-export
readOnly: true
volumes:
- name: credentials
emptyDir: {}
- name: profile
emptyDir: {}
- name: config
configMap:
name: orders-report-export
The sidecar is working when its log shows a completed render and the application container can read the files:
The Export file
An Export is a client-local resource. alma export -c reads it from
a file, or from stdin when the path is -. It is never applied to the cluster
and never appears in alma get.
Unknown fields are rejected, so a misspelled key fails the command instead of being silently ignored.
| Field | Type | Default | Notes |
|---|---|---|---|
apiVersion | string | Required | Must be almaforge.com/v1 |
kind | string | Required | Must be Export |
metadata.name | string | Required | A DNS-1123 label: lower-case letters, digits and - |
spec.certificateTTL | duration | 1h | Lifetime of every issued certificate. Must be positive |
spec.renewalInterval | duration | One third of certificateTTL | Time between renders. Must be shorter than certificateTTL |
spec.outputs | list | Required | At least one entry |
The exporter renders every output once at startup and starts another render on each renewal interval. A failure during the first render exits the command. A later failure stops that render cycle, is logged, and is retried at the next interval.
Outputs
Each entry under spec.outputs renders one credential bundle.
type | Use | Required fields |
|---|---|---|
identity | SSH and API client credentials | None |
database | Credentials routed to one database service | service |
kubernetes | A kubeconfig for one Kubernetes cluster | kubernetesCluster |
application | Credentials routed to one application | appName |
sshHost | An OpenSSH host key and host certificate | principals |
Every output accepts roles. Leave it empty to use the selected identity's
role set. Naming roles issues credentials with exactly those roles, requires
permission to impersonate them, and prevents the rendered identity from
issuing another certificate.
sshHost is the exception. Its host certificate always carries the node
system role. For this output type, roles controls which identity may request
the certificate but does not change what the certificate grants.
Output fields
| Field | Applies to | Notes |
|---|---|---|
type | Every output | Required. One of the five types above |
destination | Every output | Required. See Destinations |
roles | Every output | Roles to assume. Empty means the caller's own role set |
service | database | Required. The DatabaseTarget name |
database | database | Database inside that target to route to |
username | database | Database account to connect as |
format | database | One of tls, mongo, cockroach. Omit for a generic bundle |
kubernetesCluster | kubernetes | Required. The KubernetesTarget name |
appName | application | Required. The AppTarget name |
principals | sshHost | Required. At least one SSH host principal |
The output object is shared across all five types. A known field that does not
apply to the selected type is accepted but ignored. Keep each output limited
to the fields listed for its type.
Files written
The identity, database, kubernetes, and application outputs all write
these files:
key,key.pub,key-cert.pub, andtlscertidentityalma-host-ca.crt,alma-user-ca.crt, andalma-database-ca.crt
They also leave an empty .write-test alongside the credentials. It holds
nothing and is safe to ignore.
Output-specific files and exceptions are listed below:
type | Additional files |
|---|---|
identity | known_hosts, plus ssh_config for a directory destination |
database | Depends on format, as listed below |
kubernetes | kubeconfig.yaml |
application | None |
sshHost | Writes only ssh_host, ssh_host-cert.pub, and ssh_host-user-ca.pub |
A database output supports these formats:
format | Additional files |
|---|---|
| Omitted | No format-specific files |
tls | tls.key, tls.crt, and tls.cas |
mongo | mongo.crt and mongo.cas |
cockroach | cockroach/node.key, cockroach/node.crt, and cockroach/ca.crt |
Destinations
Every output needs its own destination. Two outputs in the same Export file
cannot write to the same directory or Kubernetes Secret.
type | directory | kubernetesSecret |
|---|---|---|
identity | Supported | Supported without ssh_config |
database | Supported | Supported except when format is cockroach |
kubernetes | Supported | Not supported |
application | Supported | Supported |
sshHost | Supported | Supported |
Directory
A directory destination requires an absolute path. The exporter requests
mode 0700 for a directory it creates and mode 0600 for new credential
files.
| Field | Default | Notes |
|---|---|---|
path | Required | Absolute path to the destination directory |
symlinks | trySecure on Linux, insecure elsewhere | secure fails the write when the kernel cannot guarantee a symlink-safe open, trySecure warns and continues, insecure skips the check |
acls | try on Linux, off elsewhere | Accepted values are try, off and required. required is refused on a host without POSIX ACL support |
Generated ssh_config and kubeconfig.yaml files call the current
alma executable and refer to credentials by their absolute paths.
The consuming process must see the same executable and destination paths.
Kubernetes Secret
A kubernetesSecret destination requires a Secret name and the
POD_NAMESPACE environment variable. The exporter creates an opaque Secret if
it does not exist. Its Kubernetes identity must be able to read and apply the
named Secret in that namespace.
| Field | Default | Notes |
|---|---|---|
name | Required | Name of the Secret in the POD_NAMESPACE namespace |
Set KUBECONFIG to use a specific kubeconfig. Without it, the exporter uses
in-cluster credentials. It does not load the default kubeconfig, so a process
running outside a pod must set KUBECONFIG or the render fails.
Operations
Revocation. Lock the identity when access must end before its certificates
expire. The next render fails, and services reject the certificates already on
disk after they receive the lock. Deleting the Token only prevents the
machine from re-attesting after its join credential ages out. It does not
revoke certificates already issued. See
Revoking access for propagation behavior.
One writer per destination. Concurrent export processes pointed at the same directory or Secret overwrite each other's credentials. Neither destination takes a lock that would stop it.
Reloading the consumer. The exporter replaces files in place and does not restart anything. A process that reads its certificate once at boot keeps using the copy it loaded, so it needs its own reload after a renewal, or a restart before the old certificate expires.
Credential handling. The rendered certificates and keys are credential
material, identity included. Keep a directory destination on a dedicated,
unshared volume. A directory created by the exporter requests mode 0700.
Restrict access to a Kubernetes Secret destination with namespace-scoped RBAC
and volume mounts.
Signals. SIGHUP and SIGUSR1 request an immediate render without
restarting the process. SIGINT and SIGTERM stop it, which is what
systemctl stop and a pod eviction send.
Troubleshooting
The exporter reports that it is not logged in
The command found no usable identity and prints the alma login line
for the cluster. For a person, run that command and start the exporter again
from the same shell. For a machine, confirm that ALMA_PROXY,
ALMA_CLIENT_ID, and ALMA_JOIN_METHOD all reach the process. An incomplete
workload identity configuration falls through to the interactive path.
Check what the process actually sees:
The credentials render but open nothing
The rendered certificate carries a role set that grants no access. A bot's own role only allows it to assume other roles, so machine outputs must name the roles they need.
Set roles on the output to the roles the bot was created with, and compare
them against what the target requires:
The renewal interval is rejected
spec.renewalInterval must be strictly shorter than spec.certificateTTL, and
the message names both values. Shortening certificateTTL without touching
renewalInterval is the common way to hit this. Remove renewalInterval to
take the default of one third of the TTL.
The destination path is rejected
A directory destination requires an absolute path. Relative paths are
rejected so the destination does not depend on the exporter's working
directory.
Two outputs collide on one destination
Each output owns its destination. The error names both output indexes and the directory or Secret they share. Give each one its own path.
A kubeconfig cannot be written to a Kubernetes Secret
A kubernetes output needs a directory destination. The kubeconfig it renders
addresses the identity file by absolute path, which a Secret cannot provide, so
the render fails with a message that the destination must be a directory. A
database output with format: cockroach is refused earlier and for its own
reason: it writes into a subdirectory, and a Secret holds a flat set of keys.
Write those outputs to a directory. In a pod, an emptyDir shared with the
consuming container is the usual answer.
A Kubernetes Secret destination cannot find its namespace
The exporter reads POD_NAMESPACE and fails when it is unset. Kubernetes does
not inject it by default. Add it to the exporter container as a downward-API
environment variable sourced from metadata.namespace, or set it explicitly
when the process runs outside a pod.
Next steps
- Workload Identity: authenticate a non-human caller without a long-lived secret.
- PostgreSQL access: the interactive paths to the same databases.
- CLI reference: review the full command surface.