Setting up Kubernetes with eksctl and AWS

eksctl allows you to spin up a Kubernetes cluster in AWS from your command line, to use it you’ll need the AWS command line tools installed on your local machine, as well as kubectl.

You can use brew to install eksctl on macOS

brew tap weaveworks/tap
brew install weaveworks/tap/eksctl

Installation instructions for other platforms can be found here

To allow eksctl permission to run on your AWS account you’ll need to create IAM policies to create and manage the nodes in the Kubernetes cluster, most of them are standard AWS policies but two custom policies are needed.

Get your account id from your ‘My Account’ section in the AWS console, it's a 12 digit number.

For each policy, start at your IAM dashboard in the AWS console, in the policy section click ‘create policy’ and paste in the JSON listed below, remember to replace $ACCOUNT_ID with your own AWS account id.

IamLimitedAccess policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateInstanceProfile",
                "iam:DeleteInstanceProfile",
                "iam:GetInstanceProfile",
                "iam:RemoveRoleFromInstanceProfile",
                "iam:GetRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:AttachRolePolicy",
                "iam:PutRolePolicy",
                "iam:ListInstanceProfiles",
                "iam:AddRoleToInstanceProfile",
                "iam:ListInstanceProfilesForRole",
                "iam:PassRole",
                "iam:DetachRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:GetRolePolicy",
                "iam:GetOpenIDConnectProvider",
                "iam:CreateOpenIDConnectProvider",
                "iam:DeleteOpenIDConnectProvider",
                "iam:ListAttachedRolePolicies",
                "iam:TagRole"
            ],
            "Resource": [
                "arn:aws:iam::$ACCOUNT_ID:instance-profile/eksctl-*",
                "arn:aws:iam::$ACCOUNT_ID:role/eksctl-*",
                "arn:aws:iam::$ACCOUNT_ID:oidc-provider/*",
                "arn:aws:iam::$ACCOUNT_ID:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup",
                "arn:aws:iam::$ACCOUNT_ID:role/eksctl-managed-*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole"
            ],
            "Resource": [
                "arn:aws:iam::$ACCOUNT_ID:role/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:AWSServiceName": [
                        "eks.amazonaws.com",
                        "eks-nodegroup.amazonaws.com",
                        "eks-fargate.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

EksAllAccess policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "*"
        },
        {
            "Action": [
                "ssm:GetParameter",
                "ssm:GetParameters"
            ],
            "Resource": [
                "arn:aws:ssm:*::$ACCOUNT_ID:parameter/aws/*",
                "arn:aws:ssm:*::parameter/aws/*"
            ],
            "Effect": "Allow"
        },
        {
            "Action": [
                "kms:CreateGrant",
                "kms:DescribeKey"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}

Now create an IAM group and add the policies above as well as the standard AWS policies AWSCloudFormationFullAccess and AmazonEC2FullAccess.

Once the group has been created, add an existing user to the group or create a new user, then add them to the group and get the API credentials for the user.

Enter your new credentials to allow your AWS to provision the necessary services

vim  ~/.aws/credentials

Test it out by running

eksctl create cluster --name testK8 --region us-west-2 --nodes 2 --ssh-access

It may take a few minutes to get the nodes up and running but if all went well you should see the cluster on your AWS EKS console or by running

kubectl cluster-info

You can delete your test cluster with

eksctl delete  cluster --name testK8

More about eksctl can be found on their site

Thanks for reading

Please get in touch by email or by twitter if you have any questions or follow ups

Niall McGinness