{"token_count": 6046}

# Register a Kubernetes Cluster using IAM Joining

In this guide, you will register a Kubernetes cluster with Teleport by using a Teleport Agent's IAM identity to automatically join the Teleport cluster.

## How it works

You can register multiple Kubernetes clusters with Teleport by deploying the Teleport Kubernetes Service on each cluster you want to register without having to distribute a joining secret to the Kubernetes cluster.

Once the Kubernetes cluster is registered for the first time, the agent will store its Teleport identity in a Kubernetes secret. The agent will use this identity to automatically join the cluster on subsequent restarts.

## Prerequisites

- A running Teleport cluster accessible at a hostname with a valid TLS certificate. If you want to get started with Teleport, [sign up](https://goteleport.com/signup) for a free trial or [set up a demo environment](https://goteleport.com/docs/get-started/deploy-community.md).

- The `tctl` and `tsh` clients.

  Installing `tctl` and `tsh` clients

  1. Determine the version of your Teleport cluster. The `tctl` and `tsh` clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at `/v1/webapi/find` and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service:

     **Mac/Linux**

     ```
     $ TELEPORT_DOMAIN=teleport.example.com:443
     $ TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')"
     ```

     **Windows - Powershell**

     ```
     $ $TELEPORT_DOMAIN = "teleport.example.com:443"
     $ $TELEPORT_VERSION = (Invoke-RestMethod -Uri "https://${TELEPORT_DOMAIN}/v1/webapi/find").server_version
     ```

  2. Follow the instructions for your platform to install `tctl` and `tsh` clients:

     **Mac**

     Download the signed macOS .pkg installer for Teleport, which includes the `tctl` and `tsh` clients:

     ```
     $ curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg
     ```

     In Finder double-click the `pkg` file to begin installation.

     ---

     DANGER

     Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security.

     ---

     **Windows - Powershell**

     ```
     $ curl.exe -O https://cdn.teleport.dev/teleport-v$TELEPORT_VERSION-windows-amd64-bin.zip
     Unzip the archive and move the `tctl` and `tsh` clients to your %PATH%
     NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP.
     Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\<username>) instead.
     ```

     **Linux**

     All of the Teleport binaries in Linux installations include the `tctl` and `tsh` clients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our [installation page](https://goteleport.com/docs/installation/single-machine.md).

     ```
     $ curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     $ tar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz
     $ cd teleport
     $ sudo ./install
     Teleport binaries have been copied to /usr/local/bin
     ```

  Connecting with TLS routing disabled

  This guide's commands assume your Teleport cluster uses TLS routing (`proxy_listener_mode: multiplex`), where the `tctl` and `tsh` clients reach every Teleport service through the Proxy Service's web address on port `443`. If you're not sure whether this applies to your cluster, check with whoever manages it.

  If your cluster uses separate listener ports instead, adjust ports as follows:

  - **`tsh` commands** (e.g., `tsh login --proxy=...`): continue using the Proxy Service web address on port `3080` (or `443` if behind a load balancer). Do not change these to port `3025`.

  - **Direct `tctl` or Auth Service API commands**: use port `3025` for the Auth Service gRPC listener:

    ```
    $ tctl status --auth-server=teleport.example.com:3025
    ```

* A Kubernetes cluster version >= v1.17.0
* An existing [IAM OpenID Connect (OIDC)](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html) provider for your cluster
* Helm >= 3.4.2
* AWS CLI >= `2.10.3` or `1.27.81`

Verify that Helm and Kubernetes are installed and up to date.

```
$ helm version
version.BuildInfo{Version:"v3.4.2"}

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17+"}
Server Version: version.Info{Major:"1", Minor:"17+"}
```

- Check that you can connect to your Teleport cluster and verify that you can run `tctl` and `tsh` commands using your current credentials.

  1. Assign teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email\@example.com to your Teleport username.

  2. Authenticate to your Teleport cluster. This depends on whether your shell is interactive or not.

     **In an interactive shell:** Run the following command. By default, this triggers a multi-factor authentication prompt:

     ```
     $ tsh login --proxy=teleport.example.com --user=email@example.com
     $ tctl status
     Cluster  teleport.example.com
     Version  18.10.0
     CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
     ```

     **On non-interactive environments:** If you are running `tsh` and `tctl` as an AI agent, in a CI/CD environment, or similar, make sure the `TELEPORT_IDENTITY_FILE` environment variable is assigned to a valid file path with credentials for your cluster. `tsh` and `tctl` read the file path from the environment variable and do not require a separate authentication step. If there is no identity file available, we recommend that you [set up Machine ID](https://goteleport.com/docs/machine-workload-identity/getting-started.md) to provision one automatically.

     When executing `tctl` commands with an identity file, you must pass the `--auth-server` flag to provide the Teleport Auth Service address, which is not included in the identity file. If you provide the Proxy Service address, `tctl` connects to the Proxy Service, which forwards traffic to and from the Teleport Auth Service. Update 443 to `3025` if you are contacting the Auth Service directly with `tctl`:

     ```
     $ tctl status --auth-server=teleport.example.com:443
     ```

     For `tsh` commands that read an identity file, you must pass the `--proxy` flag, which points `tsh` to the address of the Teleport Proxy Service:

     ```
     $ tsh status --proxy=teleport.example.com
     ```

     Ensure client commands can access your identity file. Replace path/to/identity/file with the path to your identity file:

     ```
     $ export TELEPORT_IDENTITY_FILE="${TELEPORT_IDENTITY_FILE:-path/to/identity/file}"
     ```

     Add the `--auth-server` or `--proxy` flags to all subsequent `tctl` and `tsh` commands.

  If you can connect to the cluster and run the `tctl status` command, you can use your current credentials to run subsequent `tctl` commands from your workstation. If you host your own Teleport cluster, you can also run `tctl` commands on the computer that hosts the Teleport Auth Service for full permissions.

## Step 1/3. Create a Kubernetes service account with an IAM identity

Teleport supports a mode where agents running in AWS can join the cluster using the IAM identity they are running as. It allows you to register Kubernetes clusters running in AWS without having to distribute a joining secret to the Kubernetes cluster.

To securely join the cluster without relying on the EKS node's Identity, a Teleport Agent must run as a separate Kubernetes service account with an attached IAM role. Relying on the node's identity is not recommended as it can be easily compromised since every pod running on the node has access to the node's identity if IAM Roles for Service Accounts (IRSA) is not configured.

For IRSA to work correctly, it requires the Kubernetes cluster to have an IAM OpenID Connect that maps IAM roles to Kubernetes service accounts.

The Kubernetes service account must have access to the `sts:GetCallerIdentity` API but does not require any other permissions.

To create the IAM policy, run the following command:

```
$ cat >iam-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:GetCallerIdentity",
            "Resource": "*"
        }
    ]
}
EOF
```

Then create the IAM policy:

```
$ aws iam create-policy --policy-name kube-iam-policy --policy-document file://iam-policy.json
{
    "Policy": {
        "PolicyName": "kube-iam-policy",
        "PolicyId": "ANPAW2Y2Q2Y2Y2Y2Y2Y2Y",
        "Arn": "arn:aws:iam::123456789012:policy/kube-iam-policy",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "Description": "",
        "CreateDate": "2021-03-18T15:12:00+00:00",
        "UpdateDate": "2021-03-18T15:12:00+00:00"
    }
}
```

Now we need to create the Kubernetes service account and map it to the IAM role. There are two ways of doing this. You can use `eksctl` if your cluster was provisioned using it or you can use the AWS CLI method.

**Using eksctl**

`eksctl` supports automatic creation of new IAM roles and mapping it into the Kubernetes Service Account in the target namespace. Make sure to assign aws-account-id to your AWS account ID:

```
$ eksctl create iamserviceaccount \
    --name teleport-kube-agent-sa \
    --namespace teleport-agent \
    --cluster kube-cluster \
    --region aws-region \
    --attach-policy-arn arn:aws:iam::aws-account-id:policy/kube-iam-policy \
    --role-name kube-iam-role \
    --approve
```

The referenced parameters are:

- teleport-kube-agent-sa is the name of the Kubernetes service account.
- teleport-agent is the namespace where the Teleport Kubernetes Service is running.
- aws-region is the AWS region where the cluster is running.
- kube-iam-policy is the name of the IAM policy created in the previous step.
- kube-cluster is the name of the Kubernetes cluster.
- kube-iam-role is the name of the IAM role to create.

Once the command completes, you should see a new IAM role created in your AWS account and a new Kubernetes service account created in the target namespace.

**Using AWS CLI**

Creating a new IAM role and mapping it into the Kubernetes service account in the target namespace using the AWS CLI requires some additional steps.

First, we need to create the target namespace in the Kubernetes cluster and the Kubernetes service account.

```
$ kubectl create ns teleport-agent
namespace/teleport-agent created

$ kubectl create sa teleport-kube-agent-sa -n  teleport-agent
serviceaccount/teleport-kube-agent-sa created
```

Then we need to create the IAM role and trust relationship. For that, we need to get the AWS account ID and the OIDC provider URL. If your cluster doesn't have one configured check the following guide: [IAM OpenID Connect (OIDC)](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html).

To extract the AWS account ID you can use the following command:

```
$ account_id=$(aws sts get-caller-identity --query "Account" --output text)
```

The OIDC provider URL can be extracted from the cluster configuration:

```
$ oidc_provider=$(aws eks describe-cluster --name kube-cluster --region aws-region --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
$ echo $oidc_provider
oidc.eks.eu-west-1.amazonaws.com/id/[...]
```

If the output of the command is empty, you need to configure the OIDC provider as mentioned above.

To create the IAM role and trust relationship, run the following command:

```
$ cat >trust-relationship.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::$account_id:oidc-provider/$oidc_provider"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "$oidc_provider:aud": "sts.amazonaws.com",
          "$oidc_provider:sub": "system:serviceaccount:teleport-agent:teleport-kube-agent-sa"
        }
      }
    }
  ]
}
EOF
```

To create the IAM role, run the following command:

```
$ aws iam create-role --role-name kube-iam-role --assume-role-policy-document file://trust-relationship.json --description "my-role-description"
```

Then attach the service account with the IAM role annotation:

```
$ kubectl annotate serviceaccount -n teleport-agent teleport-kube-agent-sa eks.amazonaws.com/role-arn=arn:aws:iam::$account_id:role/kube-iam-role
```

At this point, the IAM role is ready to be used by the Teleport Kubernetes Service's service account.

## Step 2/3. Create the AWS joining token

Create a dynamic token which will allow agents from your AWS account to join your Teleport cluster using the roles defined.

Under the hood, Kubernetes Service instances will prove that they are running in your AWS account by sending a signed Identity Document which matches an allow rule configured in your AWS joining token.

Create the following `token.yaml` with an `allow` rule specifying your AWS account and the AWS ARN the agents will be running as.

```
$ cat >token.yaml <<EOF
kind: token
version: v2
metadata:
  # the token name is not a secret because instances must prove that they are
  # running in your AWS account to use this token
  name: kube-iam-token
spec:
  # use the minimal set of roles required
  roles: [Kube]
  # set the join method allowed for this token
  join_method: iam
  allow:
  # aws_arn is optional and allows you to restrict the IAM role of joining Agents
  # to a specific IAM role
  - aws_account: "$account_id"
    aws_arn: "arn:aws:sts::$account_id:assumed-role/kube-iam-role/*"
EOF
```

Run `tctl create token.yaml` to create the token.

## Step 3/3. Deploy the Teleport Kubernetes Service

Configure Helm to fetch Teleport charts from the Teleport Helm repository:

```
$ helm repo add teleport https://charts.releases.teleport.dev
```

Refresh the local Helm cache by fetching the latest charts:

```
$ helm repo update
```

Switch `kubectl` to the Kubernetes cluster and run:

```
Deploy a Kubernetes agent. It dials back to the Teleport cluster tele.example.com.
$ CLUSTER=iam-cluster
$ PROXY=tele.example.com:443
Install the Teleport Kubernetes agent. Does not create a service account and uses the existing
service account. See serviceAccount.create and serviceAccount.name parameters.
$ helm install teleport-agent teleport/teleport-kube-agent \
  --set kubeClusterName=${CLUSTER?} \
  --set roles="kube" \
  --set proxyAddr=${PROXY?} \
  --set joinParams.method=iam \
  --set joinParams.tokenName=kube-iam-token \
  --set serviceAccount.create=false \
  --set serviceAccount.name=teleport-kube-agent-sa \
  --create-namespace \
  --namespace=teleport-agent \
  --version 18.10.0
```

Make sure that the Teleport Agent pod is running. You should see one Teleport agent pod with a single ready container:

```
$ kubectl -n teleport-agent get pods
NAME               READY   STATUS    RESTARTS   AGE
teleport-agent-0   1/1     Running   0          32s
```

List connected clusters using `tsh kube ls` and switch between them using `tsh kube login`:

```
$ tsh kube ls

Kube Cluster Name Selected
----------------- --------
iam-cluster

kubeconfig now points to the iam-cluster cluster
$ tsh kube login iam-cluster
Logged into Kubernetes cluster "iam-cluster". Try 'kubectl version' to test the connection.

kubectl command executed on `iam-cluster` but is routed through the `tele.example.com` cluster.
$ kubectl get pods
```

If the agent pod is healthy and ready but you cannot see your Kubernetes cluster, it is likely related to RBAC permissions associated with your roles. On the other hand, if you can see your Kubernetes cluster but unable to see any pods, it's likely that your Teleport role does not allow access to pods in the Kubernetes cluster. For both cases, please refer to the section below.

Not seeing Kubernetes clusters?

To authenticate to a Kubernetes cluster via Teleport, your Teleport user's roles must allow access as at least one Kubernetes user or group.

1. Retrieve a list of your current user's Teleport roles. The example below requires the `jq` utility for parsing JSON:

   ```
   $ CURRENT_ROLES=$(tsh status -f json | jq -r '.active.roles | join ("\n")')
   ```

2. Retrieve the Kubernetes groups your roles allow you to access:

   ```
   $ echo "$CURRENT_ROLES" | xargs -I{} tctl get roles/{} --format json | \
     jq '.[0].spec.allow.kubernetes_groups[]?'
   ```

3. Retrieve the Kubernetes users your roles allow you to access:

   ```
   $ echo "$CURRENT_ROLES" | xargs -I{} tctl get roles/{} --format json | \
     jq '.[0].spec.allow.kubernetes_users[]?'
   ```

4. If the output of one of the previous two commands is non-empty, your user can access at least one Kubernetes user or group, so you can proceed to the next step.

5. If both lists are empty, create a Teleport role for the purpose of this guide that can view Kubernetes resources in your cluster.

   Create a file called `kube-access.yaml` with the following content:

   ```
   kind: role
   metadata:
     name: kube-access
   version: v7
   spec:
     allow:
       kubernetes_labels:
         '*': '*'
       kubernetes_resources:
         - kind: '*'
           namespace: '*'
           name: '*'
           verbs: ['*']
       kubernetes_groups:
       - viewers
     deny: {}

   ```

6. Apply your changes:

   ```
   $ tctl create -f kube-access.yaml
   ```

   ---

   TIP

   You can also create and edit roles using the Web UI. Go to **Access -> Roles** and click **Create New Role** or pick an existing role to edit.

   ---

7. Assign the `kube-access` role to your Teleport user. The commands to run depend on how you authenticate to Teleport, that is, whether you have a local Teleport user, a single sign-on authentication connector, or a non-interactive user:

   **Local User**

   1. Retrieve your local user's roles as a comma-separated list:

      ```
      $ ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")')
      ```

   2. Edit your local user to add the new role:

      ```
      $ tctl users update $(tsh status -f json | jq -r '.active.username') \
        --set-roles "${ROLES?},kube-access"
      ```

   3. Sign out of the Teleport cluster and sign in again to assume the new role.

   **GitHub**

   1. Open your `github` authentication connector in a text editor:

      ```
      $ tctl edit github/github
      ```

   2. Edit the `github` connector, adding `kube-access` to the `teams_to_roles` section.

      The team you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the team must include your user account and should be the smallest team possible within your organization.

      Here is an example:

      ```
        teams_to_roles:
          - organization: octocats
            team: admins
            roles:
              - access
      +       - kube-access

      ```

   3. Apply your changes by saving and closing the file in your editor.

   4. Sign out of the Teleport cluster and sign in again to assume the new role.

   **SAML**

   1. Retrieve your `saml` configuration resource:

      ```
      $ tctl get --with-secrets saml/mysaml > saml.yaml
      ```

      Note that the `--with-secrets` flag adds the value of `spec.signing_key_pair.private_key` to the `saml.yaml` file. Because this key contains a sensitive value, you should remove the saml.yaml file immediately after updating the resource.

   2. Edit `saml.yaml`, adding `kube-access` to the `attributes_to_roles` section.

      The attribute you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.

      Here is an example:

      ```
        attributes_to_roles:
          - name: "groups"
            value: "my-group"
            roles:
              - access
      +       - kube-access

      ```

   3. Apply your changes:

      ```
      $ tctl create -f saml.yaml
      ```

   4. Sign out of the Teleport cluster and sign in again to assume the new role.

   **OIDC**

   1. Retrieve your `oidc` configuration resource:

      ```
      $ tctl get oidc/myoidc --with-secrets > oidc.yaml
      ```

      Note that the `--with-secrets` flag adds the value of `spec.signing_key_pair.private_key` to the `oidc.yaml` file. Because this key contains a sensitive value, you should remove the oidc.yaml file immediately after updating the resource.

   2. Edit `oidc.yaml`, adding `kube-access` to the `claims_to_roles` section.

      The claim you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.

      Here is an example:

      ```
        claims_to_roles:
          - name: "groups"
            value: "my-group"
            roles:
              - access
      +       - kube-access

      ```

   3. Apply your changes:

      ```
      $ tctl create -f oidc.yaml
      ```

   4. Sign out of the Teleport cluster and sign in again to assume the new role.

   **Non-Interactive**

   For non-interactive users such as AI agents and CI/CD runners, follow these steps.

   If you are using Machine ID to issue identity files to your user:

   1. Add the new role to your bot. Assign mybot to the name of the Machine ID bot that issues your certificate:

      ```
      $ tctl bots update mybot --add-roles kube-access
      ```

   2. Force `tbot` to regenerate the certificate:

      ```
      $ pkill -HUP tbot
      ```

      The bot will also generate a certificate with the new role on its next renewal.

   Otherwise, update your user and regenerate your identity file:

   1. Retrieve your local user's roles as a comma-separated list. Make sure username is the Teleport user you want to assign the new role to.

      ```
      $ ROLES=$(tctl get user/username --format=json | jq -r '.[].spec.roles | join(",")')
      ```

   2. Edit your local user to add the new role:

      ```
      $ tctl users update username \
        --set-roles "${ROLES?},kube-access"
      ```

   3. Regenerate the identity file. Make sure path/to/identity/file corresponds to the correct output path:

      ```
      $ tctl auth sign \
        --user=username \
        --out=path/to/identity/file \
        --overwrite
      ```

8. Configure the `viewers` group in your Kubernetes cluster to have the built-in `view` ClusterRole. When your Teleport user assumes the `kube-access` role and sends requests to the Kubernetes API server, the Teleport Kubernetes Service impersonates the `viewers` group and proxies the requests.

   Create a file called `viewers-bind.yaml` with the following contents, binding the built-in `view` ClusterRole with the `viewers` group you enabled your Teleport user to access:

   ```
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: viewers-crb
   subjects:
   - kind: Group
     # Bind the group "viewers", corresponding to the kubernetes_groups we assigned our "kube-access" role above
     name: viewers
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: ClusterRole
     # "view" is a default ClusterRole that grants read-only access to resources
     # See: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles
     name: view
     apiGroup: rbac.authorization.k8s.io

   ```

9. Apply the `ClusterRoleBinding` with `kubectl`:

   ```
   $ kubectl apply -f viewers-bind.yaml
   ```

## Next steps

To see all of the options you can set in the values file for the `teleport-kube-agent` Helm chart, consult our [reference guide](https://goteleport.com/docs/reference/helm-reference/teleport-kube-agent.md).