Kubernetes Guidance Report

Obtain a set of best practice recommendations to enhance cost-efficiency, boost performance, and ensure reliability for your Kubernetes resources. These recommendations are grouped based on three priority levels: High, Moderate, and Low.

Ensure that you follow them in order to maintain high availability and minimize downtime.

Best practices 

Nodes

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. High CPU usage of nodes

Priority:

High

Best practice:

Ensure the node's CPU usage does not exceed 90%.

Description:

Allocate ideal limits to all your resources that tend to overutilize the CPU based on the capacity of the node.

3. High memory usage of nodes

Priority:

High

Best practice:

Ensure the node's memory usage does not exceed 90%.

Description:

Allocate ideal limits to all your resources that tend to overutilize the memory based on the capacity of the node.

4. Low CPU usage of nodes

Priority:

High

Best practice:

Ensure the node's CPU usage does not go below 10%.

Description:

Terminate or scale down underutilized nodes to save money and resources. Allocate lesser resources so operations are balanced.

5. Low memory usage of nodes

Priority:

High

Best practice:

Ensure the node's memory usage does not go below 10%.

Description:

Terminate or scale down underutilized nodes to save money and resources. Allocate lesser resources so operations are balanced.

Pods

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. Pods without controllers

Priority:

Low

Best practice:

Deploy pods with appropriate controllers.

Description:

Pods without controllers can prove a potential threat, and they cannot be used for scaling. These controllers limit pod access to reduce the risk of a breach. In the event of a node failure, these pods will not be rescheduled.

3. Workloads running in the default namespace

Priority:

Low

Best practice:

Move workloads to a separate namespace or project

Description:

Different namespaces for workloads can help organize objects and enable the creation of logical partitions inside the Kubernetes cluster, and they are also helpful for security purposes.

Deployments

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. Missing CPU requests

Priority:

Medium

Best practice:

Configure CPU requests to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container CPU requests. Configuring CPU requests to all workloads will enable the cluster to spread loads evenly.

3. Missing CPU limits

Priority:

Medium

Best practice:

Configure CPU limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container CPU limits. Configuring CPU limits to all workloads will enable the cluster to spread loads evenly.

4. Missing memory requests

Priority:

Medium

Best practice:

Configure memory requests to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory requests. Configuring memory requests to all workloads will enable the cluster to spread loads evenly.

5. Missing memory limits

Priority:

Medium

Best practice:

Configure memory limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory limits. Configuring memory limits to all workloads will enable the cluster to spread loads evenly.

6. Missing liveness probe

Priority:

Medium

Best practice:

Configure a liveness probe, which restarts the container if its status is not ready.

Description:

Restarting the container is as important as knowing its status, which indicates that it is not ready to receive traffic.

7. Missing readiness probe

Priority:

Medium

Best practice:

Configure a readiness probe, which indicates the ready status of the container.

Description:

If the readiness of the container is not known and if it is not in a ready state, then traffic might stagnate.

8. Image tag not specified

Priority:

High

Best practice:

Specifying a stable tag provides clarity and control in managing container images.

Description:

Tags allow you to control when updates to your application are applied, and they also reduce the risk of unexpected behavior due to differences in image versions.

9. ImagePullPolicy not set to Always

Priority:

Low

Best practice:

Set your ImagePullPolicy to Always to ensure the latest version of the image.

Description:

When the ImagePullPolicy is set to Always, you can ensure the latest version of the image is deployed every time a Kubernetes pod is started.

10. RunAsRoot allowed

Priority:

High

Best practice:

Ensure that RunAsRoot is not allowed.

Description:

If RunAsRoot is allowed, the root user will have all privileges, and security might be compromised.

11. RunAsPrivileged allowed

Priority:

High

Best practice:

Ensure that RunAsPrivileged is not allowed.

Description:

If RunAsPrivileged is allowed, the container will have access to all the hosts' system resources, including kernel features and devices.

12. AllowPrivilegedEscalations allowed

Priority:

High

Best practice:

Ensure that AllowPrivilegedEscalations is not allowed.

Description:

If AllowPrivilegedEscalations is allowed, a random process may gain more privileges than its parent process.

13. ReadOnlyRootFileSystem not set

Priority:

High

Best practice:

Set ReadOnlyRootFileSystem to True.

Description:

If ReadOnlyRootFileSystem is not set to True, the container can access and edit the mounted host volumes.

14. Workloads running in the default namespace

Priority:

Low

Best practice:

Move workloads to a separate namespace or project

Description:

Different namespaces for workloads can help organize objects and enable the creation of logical partitions inside the Kubernetes cluster, and they are also helpful for security purposes.

15. Running one replica per deployment

Priority:

Medium

Best practice:

Create multiple replicas for a deployment for fault tolerance.

Description:

If the current replica fails, the other replicas will sustain the operation of the workload. It is ideal to create multiple replicas for a deployment.

DaemonSets

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. Missing CPU requests

Priority:

Medium

Best practice:

Configure CPU requests to workloads.

Description:

Cluster cannot spread loads across nodes evenly when workloads do not specify container CPU requests.

3. Missing CPU limits

Priority:

Medium

Best practice:

Configure CPU limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container CPU limits.

4. Missing memory requests

Priority:

Medium

Best practice:

Configure memory requests to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory requests.

5. Missing memory limits

Priority:

Medium

Best practice:

Configure memory limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory limits.

6. Missing liveness probe

Priority:

Medium

Best practice:

Configure a liveness probe, which restarts the container if its status is not ready.

Description:

Restarting the container is as important as knowing its status, which indicates that it is not ready to receive traffic.

7. Missing readiness probe

Priority:

Medium

Best practice:

Configure a readiness probe, which indicates the ready status of the container.

Description:

If the readiness of the container is not known and if it is not in a ready state, then traffic might stagnate.

8. Image tag not specified

Priority:

High

Best practice:

Specifying a stable tag provides clarity and control in managing container images.

Description:

Tags allow you to control when updates to your application are applied, and they also reduce the risk of unexpected behavior due to differences in image versions.

9. ImagePullPolicy not set to Always

Priority:

Low

Best practice:

Set your ImagePullPolicy to Always to ensure the latest version of the image.

Description:

When the ImagePullPolicy is set to Always, you can ensure the latest version of the image is deployed every time a Kubernetes pod is started.

10. RunAsRoot allowed

Priority:

High

Best practice:

Ensure that RunAsRoot is not allowed.

Description:

If RunAsRoot is allowed, the root user will have all privileges, and security might be compromised.

11. RunAsPrivileged allowed

Priority:

High

Best practice:

Ensure that RunAsPrivileged is not allowed.

Description:

If RunAsPrivileged is allowed, the container will have access to all the hosts' system resources, including kernel features and devices.

12. AllowPrivilegedEscalations allowed

Priority:

High

Best practice:

Ensure that AllowPrivilegedEscalations is not allowed.

Description:

If AllowPrivilegedEscalations is allowed, a random process may gain more privileges than its parent process.

13. ReadOnlyRootFileSystem not set

Priority:

High

Best practice:

Set ReadOnlyRootFileSystem to True.

Description:

If ReadOnlyRootFileSystem is not set to True, the container can access and edit the mounted host volumes.

14. Workloads running in the default namespace

Priority:

Low

Best practice:

Move workloads to a separate namespace or project

Description:

Different namespaces for workloads can help organize objects and enable the creation of logical partitions inside the Kubernetes cluster, and they are also helpful for security purposes.

StatefulSets 

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. Missing CPU requests

Priority:

Medium

Best practice:

Configure CPU requests to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container CPU requests.

3. Missing CPU limits

Priority:

Medium

Best practice:

Configure CPU limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container CPU limits.

4. Missing memory requests

Priority:

Medium

Best practice:

Configure memory requests to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory requests.

5. Missing memory limits

Priority:

Medium

Best practice:

Configure memory limits to workloads.

Description:

Clusters cannot spread loads across nodes evenly when workloads do not specify container memory limits.

6. Missing liveness probe

Priority:

Medium

Best practice:

Configure a liveness probe, which restarts the container if its status is not ready

Description:

Restarting the container is as important as knowing its status, which indicates that it is not ready to receive traffic.

7. Missing readiness probe

Priority:

Medium

Best practice:

Configure a readiness probe, which indicates the ready status of the container.

Description:

If the readiness of the container is not known and if it is not in a ready state, then traffic might stagnate.

8. Image tag not specified

Priority:

High

Best practice:

Specifying a stable tag provides clarity and control in managing container images.

Description:

Tags allow you to control when updates to your application are applied and also reduce the risk of unexpected behavior due to differences in image versions.

9. ImagePullPolicy not set to Always

Priority:

Low

Best practice:

Set your ImagePullPolicy to Always to ensure the latest version of the image.

Description:

When the ImagePullPolicy is set to Always, you can ensure the latest version of the image is deployed every time a Kubernetes pod is started.

10. RunAsRoot allowed

Priority:

High

Best practice:

Ensure that RunAsRoot is not allowed.

Description:

If RunAsRoot is allowed, the root user will have all privileges, and security might be compromised.

11. RunAsPrivileged allowed

Priority:

High

Best practice:

Ensure that RunAsPrivileged is not allowed.

Description:

If RunAsPrivileged is allowed, the container will have access to all the hosts' system resources, including kernel features and devices.

12. AllowPrivilegedEscalations allowed

Priority:

High

Best practice:

Ensure that AllowPrivilegedEscalations is not allowed.

Description:

If AllowPrivilegedEscalations is allowed, a random process may gain more privileges than its parent process.

13. ReadOnlyRootFileSystem not set

Priority:

High

Best practice:

Set ReadOnlyRootFileSystem to True.

Description:

If ReadOnlyRootFileSystem is not set to True, the container can access and edit the mounted host volumes.

14. Workloads running in the default namespace

Priority:

Low

Best practice:

Move workloads to a separate namespace or project.

Description:

Different namespaces for workloads can help organize objects and enable the creation of logical partitions inside the Kubernetes cluster, and they are also helpful for security purposes.

Jobs

1. Resources without labels

Priority:

Low

Best practice:

Assign appropriate labels to workloads.

Description:

Tracking resources without labels will adversely impact time and resource management. Assigning labels will improve the efficiency of resource handling.

2. Job not clearing up after completion

Priority:

Low

Best practice:

Configure the job with the ttlSecondsAfterFinished parameter to clean it up after completion.

Description:

The ttlSecondsAfterFinished parameter will limit the lifetime of job objects that have finished their expected operation. If jobs are not cleared up, it will impact cluster operations.