Technical Report on Container Security (V)-1

março 13, 2019 | Adeline Zhang

Security Tools—Open-Source Security Tool Kubernetes

In addition to commercial software, open-source software projects can also provide some security functions. This document describes several open-source projects that are usually used for protection of non-critical business.

Kubernetes’s Native Security Policies

Kubernetes’s network policies provide L3/L4-level (IP address/port) network isolation. Kubernetes does not provide network functions, but opens network interfaces and uses network plug-ins (such as Flannel[25], Calico[26], Contiv[i], Cilium[ii], and Kube-router[iii]) to support the execution of network policies.

Istio

Istio is an open-source framework jointly developed by Google, IBM, and Lyft to manage, secure, and monitor microservices. It creates a service mesh to manage service-to-service communication, including routing, authentication, authorization, and encryption. The following figure shows the Istio architecture.

Figure 5.1 Istio architecture

In a word, Istio can be defined as a simple way to build a network of services deployed. It provides load balancing, service-to-service authentication, monitoring, and the like, without requiring any changes in service code.

As shown in the preceding figure, Istio consists of a control plane and a data plane.

(1) Data plane

Envoy[iv]: As a high-performance proxy written in C++, Envoy mediates inbound and outbound traffic for all services within the service mesh.

(2) Control plane

Mixer: Mixer executes access control and usage policies across the service mesh and collects telemetry data from Envoy and other services.
Pilot: collects and verifies configuration policies and spreads them to Istio components. Independent of the underlying platform, it extracts environment-specific implementation details from Mixer and Envoy and provides abstract representations of user services for them.
Citadel: Citadel provides strong service-to-service and end-user authentication with interactive TLS, built-in identity, and credential management.

Meanwhile, Istio provides a security mechanism which, without any modification to service code, secure services by using a sidecar container deployed near the container to which the microserivce belongs. This mechanism provides authentication and authorization functions.

(1) Authentication

Istio provides two types of authentication:

Transmission authentication (service-to-service authentication): Istio provides bidirectional TLS authentication as a complete authentication solution.
Source authentication (authentication of end users): Istio authenticates the original client requesting to be an end user or device. With JSON Web Token (JWT), Auth0, Firebase Auth, Google Auth, and custom authentication methods, Istio simplifies the developer experience and easily implements request authentication.

In the preceding circumstances, Istio stores authentication policies in the Istio Config Store through a custom Kubernetes API. Pilot, when appropriate, will keep the latest status and key for each proxy. Besides, Istio supports authentication in license mode to help users learn about what impact policy changes will have on the user’s security status before taking effect.

With authentication policies, mesh operators can specify authentication requirements for services that receive requests in the Istio mesh. Figure 5.2 shows Istio’s authentication architecture. Administrators set authentication policies using the yaml configuration file and deploy them on Namespace Foo and Namespace Bar. After deployment, these policies will be stored in Istio Config Store which is monitored by Pilot. Once changes occur to policies, Pilot will convert new policies into appropriate configurations and notify the Envoy Sidecar proxy, which is at the same position as the service instance, of how to execute the required authentication mechanism. For example, the policy defined in Namespace Foo targets SvcA and therefore will work for SvcA’s Envoy Sidecar proxy. Likewise, two policies created in Namespace Bar respectively target ALL and SvcB and therefore work for proxies of the two services.

Figure 5.2 Authentication architecture of Istio

2 Authorization

Istio’s authorization feature is also known as role-based access control (RBAC) which provides namespace-level, service-level, and method-level access control for services in an Istio mesh. This function has the following characteristics:

Role-based semantics, which makes for ease of use.
Service-to-service and end user-to-service authorization.
High flexibility as it allows users to define attributes such as bindings between conditions and roles.
High performance as Istio authorization is enforced natively on Envoy.

Figure 5.3 shows the official Istio authorization architecture. The network administrator can set the Istio authorization policy using the yaml configuration file and deploy it. After deployment, this policy is stored in Istio Config Store and under the monitoring of Pilot. Once finding any changes are made to the policy, Pilot will re-obtain the policy and dispatch it to the Envoy Sidecar proxy which is at the same position as the service instance. Each Envoy Sidecar proxy will run an authorization engine which authorizes requests at runtime. When a request arrives at the proxy, its running authorization engine will evaluate the request context against the authorization policy and return the authorization result (ALLOW or Deny).

Figure 5.3 Istio authorization architecture

Grafeas

Grafeas is a tool launched in 2017 by Google in collaboration with JFrog and IBM among others for auditing and governing the software supply chain.

With DevOps and microservices constantly develop, software is delivered at an increasingly rapid pace, with delivered binaries (including Docker images) showing exponential increase. Multiple languages for software development differ materially in data specifications. For example, the GO language uses URLs to define its own component addresses; Java has its unique specifications; Docker identifies each layer with a Sha256 number. Such specification differences make it difficult to determine the quality of and vulnerabilities in binaries of multiple languages included in container images before these images are delivered for production.

Grafeas was originally designed for container security quality management. The official definition, however, suggests that Grafeas goes beyond its original purpose to provide an open-source component metadata API for auditing and monitoring software components in a centralized way. Software mentioned here can be container images or packages in WAR, JAR or ZIP format.

Figure 5.4 shows a software supply chain (code, build, test, deploy, and operate) of multi-side collaboration in the container environment before images are delivered for production. In each stage, Grafeas collects key metadata via its API. Especially in the deployment stage, Grafeas can conduct effective checks on the compliance with final security specifications.

Figure 5.4 Flowchart of auditing and governing a software supply chain with Grafeas

Security policies provided by Grafeas can be enforced through integration with third-party tools. Such policies can be used for security management of CI/CD pipelines rather than containers at runtime.

Clair

Clair is an open-source container analysis tool released by CoreOS in November 2015 to fix security vulnerabilities in container images. Clair makes a static analysis of container images, associating image contents with public vulnerability databases and finally forming analysis reports.

Figure 5.5 Clair architecture

Clair consists of a detector, fetcher, notifier, and PostSQL database among other modules. For image detection, Clair first extracts characteristics of an image and then matches them against the Common Vulnerabilities and Exposures (CVE). The detailed process is as follows:

For an image to be detected, the user invokes a RESTful API of Clair, which will upload all layers of the image and, for each layer, create a worker.
Each worker starts a detector to scan the received layer.
The detector first decompresses the compressed layer package that is uploaded to obtain the operating system type and version as well as the name and version of the installed application.
The updater starts a fetcher which will search the CVE database for vulnerability information of the obtained operating system type to check the existence of vulnerabilities in the current operating system and applications.

To handle vulnerabilities, Clair will start a Notifier+WebHook asynchronous process in which WebHook keeps crawling up-to-date vulnerability information from CVE websites, then notifies newly exposed CVE vulnerabilities to the notifier, and finally updates the vulnerability information to the database.

Figure 5.6 Image scanning process

Users can execute simple command lines with Clairctl[v], the lightweight Clair client tool, to upload and scan images as well as output reports.

CIS Benchmarks

The Center for Internet Security (CIS)[vi] is a nonprofit entity that harnesses the power of a global IT community to safeguard private and public organizations against cyber threats. CIS Benchmarks is a recognized global standard and best practice for securing IT systems and data against attacks. CIS Benchmarks provide a wide variety of security assessment standards covering operating systems, system software, cloud providers, and mobile devices among others.

Docker CIS Benchmark

Docker and CIS co-produce a security benchmark document on security audit of container environments. Five versions[27] are available for this document:

CIS V1.0.0 for Docker 1.11.0 Benchmark
CIS V1.0.0 for Docker 1.12.0 Benchmark
CIS V1.0.0 for Docker 1.13.0 Benchmark
CIS V1.0.0 for Docker 1.6 Benchmark
CIS V1.1.0 for Docker Community Edition Benchmark

For example, CIS V1.0.0 for Docker 1.13.0 Benchmark, released on January 19, 2017, presents more than 100 security configuration and audit methods covering host configuration, Docker configuration, Docker daemon configuration files, container image and Build file, container runtime, and Docker security operations among others.

The Docker community open-sourced this tool[vii] which supports CIS Docker Community Edition Benchmark V1.1.0.

Figure 5.7 Docker Benchmark Security

Docker Hub provides the image of this Benchmark which needs to run on Docker 1.13.0 or later. Users can use the following command to rapidly deploy this program:

# docker run -it –net host –pid host –userns host –cap-add audit_control \

-e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \

-v /var/lib:/var/lib \

-v /var/run/docker.sock:/var/run/docker.sock \

-v /usr/lib/systemd:/usr/lib/systemd \

-v /etc:/etc –label docker_bench_security \

docker/docker-bench-security

Kubernetes CIS Benchmark

Meanwhile, CIS provides security Kubernetes benchmark documents to audit the operating environment of Kubernetes. Currently, two versions are available for such documents: CIS v1.0.0 for Kubernetes 1.6 Benchmark and CIS v1.2.0 for Kubernetes 1.8 Benchmark[28]. The latter, CIS V1.2.0, is the latest version which supports Kubernetes 1.8.

CIS v1.2.0 for Kubernetes 1.8 Benchmark, released in October 2017, provides more than 100 security audit check items covering API server configuration, Scheduler configuration, controller configuration, and Etcd among others.

Many organizations have open-sourced Kubernetes’s compliance and audit check tools such as NeuVector’s NeuVector CIS Kubernetes Benchmark[viii] NeuVector kubernetes-cis-benchmark and Aqua’s kube-bench[ix] (both tools support Kubernetes 1.6 and 1.8).

Figure 5.8 NeuVector CIS Kubernetes Benchmark

TUF and Notary

The Update Framework (TUF)[x] and Notary[xi] are two security incubation projects that are accepted in to the Cloud Native Computing Foundation (CNCF)[xii]. TUF is an open-source security and verification standard, while Notary is an implementation of TUF. The Update Framework was originally developed by New York University professor Justin Cappos and his team at Tandon School of Engineering. Notary was originally an image signing system component for Docker1.8.

Notary adopts a server/client architecture in which the server end runs and maintains the trusted collection of a container, while the client end interacts with the server. In this way, Notary implements more secure content delivery and verification in a simple way.

Usually, we can use the Transport Layer Security (TLS) protocol to secure the communication of both ends. This method, however, is less than perfect as any compromises to the server end will lead valid contents to be replaced by malicious ones.

With the aid of Notary, container image publishers could sign container images with strongly secure keys for encryption. In the wake of encryption, images upon publication are first pushed to the Notary’s trusted collection. In this case, a visitor can set up communication with the server in Notary and access the appropriate image as long as he or she obtains the public key of the publisher through a security channel.

Figure 5.9 Notary workflow

SPIFFE

The Secure Production Identity Framework for Everyone (SPIFFE)[xiii] is an open and community-driven service security framework actively promoted by Google. It is used for identifying and securing communications between web-based services.

Currently, most service implementations are bound to IP addresses. An IP address, however, may run a number of services. For example, an IP address runs more than one container and each container has its own task. The problem is that how we can know which container is legal or illegal. Here is another example: Service A serves service B. How can we know service A truly serves service B rather than being used for other malicious purposes?

SPIFFE defines how services identify themselves to each other. Actually, this framework uses an X.509 certificate to verify the identity of each object in the production environment. Such an X.509 certificate contains a SPIFFE ID which identifies an object and takes the form of Uniform Resource Identifiers (URIs).

SPIRE (short for SPIFFE Runtime Environment) is an open-source implementation of SPIFFE. As a tool-chain, SPIRE is used to establish trust between software systems across a wide variety of hosting platforms. Specifically, SPIRE exposes the SPIFFE Workload API, which can prove the identities of running software systems before issuing SPIFFE IDs and SVIDs to them. This in turn allows two workloads to establish trust between each other, for example by establishing an mTLS connection or by signing and verifying a JWT token.

SPIFFE is hosted by CNCF as a sandbox-level project.

Open Policy Agent (OPA)

Open Policy Agent (OPA)[xiv] is an open-source general-purpose policy engine which specifies how to enforce policies in a cloud native environment. OPA enables unified and context-aware policy enforcement. Such engine, when integrated with a service, allows users to establish and deploy custom policies to control the policy-based function of this service, as shown in the following figure. OPA is hosted by CNCF as a sandbox-level project.