Skip to main content
Install Tolmo Satellite in your cluster with Helm. It is one static binary that relays network connections to the targets that you allow. All testing runs at Tolmo, not in your cluster.
This feature is in early access. Tolmo enables it for each organization on request. See Availability.

Prerequisites

  • A Kubernetes cluster where you can run helm install.
  • Helm 3.8 or later.
  • A Tolmo API key. Create one in Settings → API keys and select Never expires. When the key expires, the satellite cannot connect and Tolmo cannot reach the cluster until you install a new key.
  • Outbound TCP 443 from the cluster to api.tolmo.com and relay.tolmo.com. See Network requirements.

Setup

1

Store your API key in a Kubernetes Secret

2

Create values.yaml

values.yaml
  • Replace <CLUSTER_NAME> with a name for this cluster, such as prod-eu. Use lowercase letters, digits, and hyphens. Use one name and one API key for each cluster.
  • By default, the allowed range is every namespace and every port. Tolmo tests only the targets that you enable. To narrow the range, add allowedRange. Tolmo cannot reach a target outside this range:
3

Install the chart

Helm installs the latest release. To pin a release, add --version <CHART_VERSION>.
4

Validate the installation

Check that the satellite is running:
The pod shows READY 1/1 when the satellite is connected to Tolmo.If the pod does not become ready, read the logs of the satellite:
The satellite is now connected. No target is reachable until it is enabled for testing. See Enable targets for testing.

You control what Tolmo can test

The satellite opens a connection only when both conditions are true:
  1. The target is inside the allowed range in your values file. By default, the range is every namespace and every port.
  2. The target is enabled for testing in Tolmo, on an enabled port.
Nothing is enabled by default. When a target is enabled or disabled, the satellite applies the change in less than one minute. The satellite always refuses the Kubernetes API, cloud metadata endpoints, loopback addresses, and its own addresses.

Enable targets for testing

The satellite reports the targets that it finds in your cluster. Tolmo tests none of them until you enable them. To enable targets, open Domains in Tolmo and select the Private tab. An organization admin enables each target by hand, and can disable it at any time.

Reference

The satellite needs two outbound destinations, both on TCP 443:The satellite also needs DNS, a route from its pod to the targets in the allowed range, and the Kubernetes API, to find the targets that you can enable (see Permissions and RBAC).The NetworkPolicy of the chart opens the Kubernetes API twice: to the addresses of the API server, and to the ClusterIP of the kubernetes Service, which the satellite calls. Some CNIs, such as the AWS VPC CNI on EKS, match egress before the ClusterIP is translated, so the chart opens both. helm install and helm upgrade read these addresses from the cluster. If you render the chart without access to the cluster, such as with helm template or Argo CD, set them in the values file. Otherwise the satellite cannot read your cluster, and Tolmo finds no new target.
  • discovery.apiServer.cidrs: the addresses that kubectl get endpoints kubernetes -n default shows.
  • discovery.apiServer.serviceIPs: the ClusterIP that kubectl get svc kubernetes -n default -o jsonpath='{.spec.clusterIP}' shows.
values.yaml
On EKS, the addresses of the API server can change: use the subnets of the control plane in discovery.apiServer.cidrs. The ClusterIP does not change.A NetworkPolicy applies once your CNI has set it up for the pod. On EKS with the VPC CNI, this takes a few seconds after a new pod starts, and the pod’s traffic is allowed until then, unless the CNI runs in strict mode. Keep your own egress controls for the namespace of the satellite.If your egress is allowed by IP address, limit the Tolmo rule of the chart to these five addresses:
values.yaml
The namespaces that you test must also admit the satellite. If a target namespace accepts only its own traffic, the connections of the satellite time out and Tolmo reports dial_failed. Add a NetworkPolicy to that namespace that admits the satellite pods:
  • Keep namespaceSelector and podSelector in the same from entry (one -). Two entries admit every pod in the namespace of the satellite, and every pod with the satellite label in the target namespace.
  • Match on the namespace and the app.kubernetes.io/name label. The other labels change with the release name.
  • When you move the satellite to another namespace, update this rule.
  • On Cilium or Calico, a policy of their own (CiliumNetworkPolicy, Calico NetworkPolicy) needs the same rule.
If a target is in a mesh with strict mutual TLS (a PeerAuthentication in STRICT mode), it accepts only connections from the mesh. A satellite outside the mesh is refused: the connection is reset, or Tolmo reports upstream_failed. The satellite does not log a refusal: it logs the connection as "result":"allowed" with "bytes_down":0. Put the satellite in the mesh:
  1. Label the namespace of the satellite for sidecar injection, then restart the satellite:
    The pod shows READY 2/2 with its istio-proxy sidecar. If it still shows 1/1, the sidecar was not injected. See the note on private control planes below.
  2. Keep the connections to Tolmo out of the sidecar. With direct egress, exclude the API and relay addresses:
    values.yaml
    Behind an egress proxy, exclude the proxy port instead, such as traffic.sidecar.istio.io/excludeOutboundPorts: "3128". Never exclude a port that your targets use, such as 443: those connections then leave without mutual TLS and are refused again.
  3. With the NetworkPolicy of the chart, let the sidecar reach istiod:
    values.yaml
  4. If the targets have an AuthorizationPolicy, allow the identity of the satellite. Otherwise the sidecar answers 403 RBAC: access denied.
    The service account is the release name, tolmo-satellite here. serviceAccount.name overrides it.
EKS, GKE, and other private control planes. The Kubernetes control plane calls istiod on TCP 15017 to inject the sidecar. On EKS, allow TCP 15017 from the cluster security group to the nodes. On a GKE private cluster, add a firewall rule for this port from the control plane range. Without it, the restarted pod is not created and the old pod stays 1/1, or the pod starts without a sidecar.Other meshes work the same way. For example, with Linkerd, add linkerd.io/inject: enabled to the namespace of the satellite, and keep the relay connection out of the mesh.
The satellite logs one JSON line for each connection. A failed connection shows "result":"denied","reason":"dial_failed" and the destination. Tolmo reports the same reason. It means that the target is in the allowed range and its name resolves, but no TCP connection opened within 10 seconds.To test the target side without Tolmo, run a pod with the namespace and label of the satellite, and call the Service:
  • A response here and dial_failed from Tolmo: check the NetworkPolicy of the chart.
  • A timeout here: check the ingress policies of the target namespace, or another egress policy in the namespace of the satellite.
In an injected namespace, this pod gets a sidecar too. Retry if the first call fails while the sidecar starts. The pod runs as the default service account, so an AuthorizationPolicy that names the satellite refuses it.The chart README has more details. See Custom configuration (values.yaml) to download it.
When the satellite cannot read what the discovery values ask for from the Kubernetes API, Tolmo shows its discovery as error and finds no new target. The logs of the satellite give the reason in a cannot list line:
If your cluster can leave only through a forward proxy, add the proxy to the values file. The standard HTTPS_PROXY and NO_PROXY variables work.
The proxy must allow CONNECT to api.tolmo.com and relay.tolmo.com on port 443. If it also inspects TLS, see TLS inspection.
Tolmo presents publicly trusted certificates on api.tolmo.com and relay.tolmo.com. The satellite verifies them with the public certificate authorities in its image. You have no Tolmo certificate to install.A proxy or firewall that inspects TLS answers the satellite with a certificate from your own certificate authority. The satellite refuses it and logs certificate signed by unknown authority. You have two options.Exempt api.tolmo.com and relay.tolmo.com from inspection. We recommend this option.Or make the satellite trust your certificate authority. Add it to the values file:
The satellite then trusts it in addition to the public authorities. You can add several certificates one after another.
With this option, your proxy can read the API key of the satellite and the traffic that Tolmo sends to your targets. A test of an HTTPS target stays encrypted between Tolmo and the target.
The connection to relay.tolmo.com is a long-lived tunnel, not a web request. A proxy that decrypts and forwards it works. A proxy that accepts only HTTP inside TLS closes it, and the pod never becomes ready: exempt relay.tolmo.com from inspection.
To let you select targets from a list, the satellite reads resources from the Kubernetes API and sends what it finds to Tolmo: names, namespaces, and TCP ports. It also sends its version and the allowed range.The satellite has read-only access (get, list, watch). It does not read Secrets, ConfigMaps, Pods, or workloads.The chart grants a right for each kind that is on in the values, and for nothing else. Only discovery.services is on by default: the satellite reads Services only.discovery.scope sets where these rights apply:A Role cannot grant what only a ClusterRole can. With discovery.scope: namespaced, the chart refuses to install when:
  • allowedRange.namespaces is empty or contains *. List the namespaces.
  • A cluster-scoped kind is on. Set it to false, or use discovery.scope: cluster.
Each namespace must exist before the install or upgrade that adds it. Otherwise Helm fails with namespaces "<NAMESPACE>" not found. You also need the right to create Roles and RoleBindings in them.For the minimal RBAC, a Role that reads Services in the namespaces that you list, use these values:
To see the RBAC before you install, render the chart:
When all switches are false, the chart creates no Role or ClusterRole, and the pod gets no service-account token. Tolmo then sees no new target, so no more targets can be enabled.
Tolmo signs the satellite image and the Helm chart with this public key, also served at https://tolmo.com/satellite/cosign.pub. Check that both match before you trust it:
cosign.pub
Verify the chart before you install it, and the image, with cosign:
To run only a signed image in your cluster, add an admission policy. With Kyverno 1.17 or later (Kyverno 1.20 no longer accepts the older kyverno.io/v1 ClusterPolicy):
mutateDigest replaces the image tag of the pod with the digest that was verified, so the node runs exactly the signed image.The chart README has the same policy for the Sigstore policy controller.If you copy the image to your own registry, copy its signature with it, and use your registry in the policy:
Common values:The chart package includes a README with all values, and a changelog:
Pass your values file again, or use --reset-then-reuse-values to keep the values of the installed release. Do not use --reuse-values: it ignores the defaults of new values in the chart, so the upgrade can fail with an error that names a value got "<nil>".An upgrade that keeps the same API key rolls the pods with no interruption.
You can stop all access from your cluster at any time.Stop the satellite and keep the configuration:
Uninstall the satellite:
You can also revoke the API key in Settings → API keys. The satellite then refuses all connections.

Availability

This feature is in early access. To request it, contact support@tolmo.com.