Security is one of the key areas of focus when adopting Kubernetes across the enterprise, and it has evolved over the past year or so. With many large enterprises evolving from the waterfall model to agile based frameworks like SAFe, IT teams are forced to release infrastructure and applications more rapidly, and “shift left” aspects of security and testing are moving up in the development and infrastructure build lifecycle. Identifying and enforcing baseline security policies early in the Kubernetes CI/CD lifecycle are critical to mitigating risks early in the process, rather than having security catch up later in the release cycle.
While third-party Kubernetes security tools such as Twistlock, AquaSec, etc., offer registry scanning and runtime protection of container images, tools like HashiCorp Sentinel and Open Policy Agent (OPA) facilitate “policy as code” and enable tight security controls to be part of deployment artifacts.
This article outlines possible control policies at different layers of Kubernetes.
Open Policy Agent
OPA is a lightweight, general-purpose policy engine that can be co-located with your Kubernetes service. OPA is deployable as a sidecar, host-level daemon or library on a Kubernetes cluster.
Services offload policy decisions to OPA by executing queries. OPA evaluates policies and data to produce query results (which are sent back to the client). Policies are written in a high-level declarative language, and can be loaded into OPA via either the filesystem or well-defined APIs.
In the context of Kubernetes, OPA implements admission control rules that validate Kubernetes resources during create, update and delete operations, and enforce policies on the fly without recompiling any of the services that offload policy decisions to it.
Utilities such as conftest help you write tests against structured configuration data such as Kubernetes deployment files, Terraform code, etc., and provide code analysis against OPA rules as part of the CI process.
Written in Rego, OPA empowers security professionals to focus on query results rather than on execution. OPA teams have released the online OPA playground and plugins to build intuitive policies that support rapid prototyping and development.
Consider a scenario to block the deployment if a specific label is missing. For example, the deployment below does not have the “environment” label in the pod definition.
An OPA policy can be defined to validate all the pod deployments, and to halt the deployment if the metadata.label.environment parameter is missing in the incoming requests.
At the deployment time of the pod definition, OPA flags the deployment with a custom message configured in the policy.
Deployments will succeed once the “environment” label is added to the input file.
OPA integration with CI/CD pipelines delivers granular security and compliance controls to the Kubernetes deployments. Standard OPA policies might include:
- Mandate SSL to all Kubernetes services
- Deny launching containers with root access
- Allow signed images only on production namespaces
- Allow images from private registry only
- Allow only X and Y images to a specific namespace
- Limit CPU and memory requests
- Allow only user X to deploy a particular namespace
Network Policies
By default, all pods in a Kubernetes cluster can communicate with each other without any restrictions. Network policies help you to isolate the services running in pods from each other, to limit the blast radius and improve the overall security posture. Kubernetes supports a plethora of Layer 2 and Layer 3 network plugins from ISVs and cloud service providers today, and each has its own pros and cons. Service meshes such as Istio or Envoy present application or Layer 7 protection, such as user authorization, TLS, etc.
For example, the network policy below whitelists inbound DB connections from pods that match specific labels, e.g., Application: Web
Network policies can be standardized to:
- By default, deny all traffic between the pods
- Allow traffic between specific pods
- Allow traffic from specific namespaces
Pod Security Policies
While OPA policies validate Kubernetes deployments, and network policies provide traffic isolation between the services, a Pod Security Policy (PSP) allows you to control access to the host operating systems. A PSP supports a multitude of OS level security controls such as SELinux, AppArmor, SecComp, etc. The policy below enforces containers to run as non-root users to prevent any privilege escalations.
With a Pod Security Policy (PSP), you could enforce:
- Run-time permissions for a container
- Permitted actions on the kernel space
- Allowed AppArmor or SecComp profiles
- SELinux modes
- Allowed host paths and volumes
Conclusion:
Kubernetes provides many primitives, such as pods, private registries, labels, namespaces, etc., to isolate workloads with required security controls, but connecting these dots at scale is a daunting task. Kubernetes policy plugins such as OPA, network policies and Pod Security Policies are designed to solve these challenges.
