Technical Report on Container Security (IV)

Container Security Protection – Application Security

• Application Security

The ecosystem of the container technology is gradually established and various solutions become available in specific segments of containers, both of which lay a solid foundation for the container deployment. On the basis of the enterprise container deployment, the emergence of business processes revolving around container applications, especially application logic-oriented microservice architecture, brings challenges to the application security.

• Microservice Security

The preceding sections describe the basic concept and framework of microservices. Common components of the container microservice architecture mainly consist of the API server and container-based microservice applications.

Microservice security protection includes the following aspects:

• The authentication and authorization mechanism of the API server
• The security mechanism for guaranteeing the integrity and confidentiality of the communication between microservices in the architecture
• Interface specification of microservices
• High availability (HA) of microservices

Details about security protection of microservices are as follows.

(1) Authentication and authorization

With the conversion from the monolithic architecture to the microservice architecture, various microservices are exposed via REST APIs, which brings privilege control-related security issues to the system, mainly the callers’ authentication and API-level operation privilege control issues. These two issues are in a particular order. The system first authenticates the user identity and then checks whether this user has the operation privilege corresponding to his/her specific privilege for the requested operation.

• Authentication

Currently, there are two common authentication schemes: session-based authentication and OAuth2 protocol token-based authentication. The latter is more applicable to microservice scenarios, because it has such advantages as stateless server side, high performance, support for mobile devices, and seamless integration with Spring Security. For different scenarios, OAuth2 provides different grant types: authorization code, implicit, password credential, and client credential, with the latter two as the common grant types.

• Operation privilege control

Currently, there are mainly two protection solutions. One is Apache Shiro[i], which is a powerful, flexible and open-source security framework that performs authentication, authorization, session management, and cryptography and is widely recommended due to its ease-of-use and seamless integration with Spring.

The other solution is Spring Security[ii]. Since it has a powerful community, supports OAuth and OpenId, and covers all functions of Shiro, Spring Security is strongly recommended by developers. Its disadvantage lies in its steep learning curve.

(2) Secure communication

In microservices, there are a great number of application services, which contain internal or external data transmission. Considering efficiency, many REST APIs adopt unreliable HTTP-based communication mode, bringing such risks as information disclosure, information theft, and information forgery. For a large-scale microservice architecture, the secure and reliable SSL/TLS communication protocol can be used to encrypt data transmission via user-oriented API pipes, in a bid to provide security assurance.

(3) Security specification of microservice interfaces

In the microservice architecture, services are provided via remote procedure calls (RPCs) or REST APIs/web service interfaces, with HTTP/HTTPS-based REST API as the mainstream. Developing security interface specifications and designing APIs for ensuring the security of microservices are necessary measures to guarantee the security of system applications. APIs can be designed according to the following guidelines:

• Checking the validity of request packets sent to the API gateway
• Filtering requests sent to the API gateway according to the API whitelist policy
• Checking the validity of the format and content of the data sent to the API gateway
• Monitoring operations contained in requests, disrupting sensitive operations, and initiating the authorization
• Limiting the number of access requests to the API gateway from the same user

(4) HA of microservices

Many services in the microservice architecture are mutually dependent. Since each service is likely to get offline and the number of microservices increases, the failure rate increases accordingly. Besides, network timeout and congestion happen from time to time, which can cause service unavailability.

We recommend to improve the availability of microservices from the following perspectives:

• Setting the timeout for service calls, so as to avoid resources being used by abnormal service call threads
• Building independent thread pools for different service traffic, so that resource consumption of one type of abnormal traffic will not affect other legitimate traffic
• Limiting the number of concurrent threads and the call rate of each service interface, guaranteeing the availability of core services
• Configuring a breaker to stop abnormal services timely
• Improving the callback processing for call failures between microservices by lowering service-specific privileges
• Monitoring the availability of microservices and setting a procedure for handling abnormal microservices
  • DevOps Security

As the methodology in the field of software lifecycle management, DevOps is adopted by more and more IT enterprises. Its value is undoubted. As the security issues facing DevOps have been described in preceding sections, this section details how to ensure the DevOps security.

• CI/CD Security Suggestions

The development and O&M teams adopts an automatic process of continuous integration and continuous deployment (CI/CD) to improve cooperation efficiency and accelerate product development with faster iteration flows. Security suggestions for CI/CI are as follows:

1. Execute code in an isolated and controlled Docker container, protecting the CI server from being threatened.
2. Back up code and programs in the process of CI, avoiding unnecessary debugging operations due to an environment error.
3. If login authentication is involved in the CI/CD process, brute-force password cracking possibly occurs. Therefore, the maximum number of failed login attempts, SMS verification code, or third-party authentication integration should be configured for authentication.
4. During the process of CI/CD, manual authorization should be performed for critical test scenarios and deployment scenarios.

• DevSecOps

Speaking of DevOps, we have to mention DevSecOps[iii]. The former is an abbreviation of “development” and “operations”, and the latter is an abbreviation of “development”, “security” and “operations”. As its name indicates, IT security must be included in the entire process of development and operations. The core is that the responsibilities of all IT members (including development, O&M, and security teams) should be seen in the entire lifecycle (from development to O&M).

Figure 4.8 DevSecOps

The key of the DevOps security lies in the risk control and management, so attention should be paid to the following aspects during the Security and Risk Management (SRM):

• Seamlessly integrate security and compliance testing into DevOps, so that developers do not have to separate from the CI/CD environment.
• Scan all open-source or third-party components for known vulnerabilities and improper configurations. Ideally, you are advised to make a complete asset list for the analysis of all software components.
• Do not try to remove all unknown vulnerabilities in custom code, which may increase false positives. Developers should be those who are the most rigorous and confident.
• Encourage use of new types of tools and methods, so as to reduce the friction between developers. For example, interactive application security testing (IAST) can take place of the traditional static and dynamic testing.
• Use the security champion model to involve the information security team in the DevOps process.
• Use a unified specification for processing all automated scripts, templates, images, and designs, making sure that all source code is covered.

According to Gartner[iv], by 2019, more than 70% of enterprise DevSecOps initiatives will have incorporated automated security vulnerability and configuration scanning for open-source components and commercial packages, up from less than 10% in 2016. By 2021, DevSecOps practices will be embedded in 80% of rapid development teams, up from 15% in 2017. Though, in most cases, DevOps does not properly handle security and compliance issues, we are happy to see that enterprises are taking it more and more seriously.

DevSecOps implementation suggestions are as follows:

• Use security test tools and processes catering for developers.

Do not force DevOps developers to adapt themselves to traditional security processes. Instead, security should be integrated to developers’ tools and processes. For example, security test tools can be integrated into developers’ development environment (IDE) and CI/CD tool chains; security and compliance scanning platforms can be built via APIs, making all security tools and services automatic; threat modeling tools can be included as part of non-functional requirements.

• Do not try to eliminate all vulnerabilities during the development.

There is no absolute security. For DevSecOps, what you should do is make continuous risk and trust assessment and prioritize application vulnerabilities. In the implementation of DevSecOps, information security personnel must admit that it is impossible to have no vulnerability, especially when such pursuit obviously hinders the pace of new service development and innovation.

Instead, the runtime protection mechanism can be used to remediate known and low-risk vulnerabilities or unknown vulnerabilities. For example, in the runtime environment, a network- and host-based intrusion protection system (IPS) and web application firewall (WAF) can be used together to monitor the application performance and security and provide in-depth protection at the application layer, including load balancing, denial-of-service (DoS) protection, and distributed denial-of-service (DDoS) protection.

• Identify and remove known critical vulnerabilities.

Generally, modern software system are assembled but not developed. Developers can collect and integrate a number of pre-compiled components, code libraries, containers, and frameworks from open-source source code and code repositories (such as GitHub and Docker Hub). Therefore, the wide usage of open-source software poses severe challenges on the security and the focus of security scanning also shifts.

From the perspective of security, identifying known vulnerabilities in known code is much easier than identifying unknown vulnerabilities in custom code. Among various implementation methods, the simplest way is to match files with the vulnerability base. Security vendors should adjust their scanning capabilities by integrating such operations into DevSecOps for an automatic implementation. Docker, as security vendor, also provide such integration capability. The importance of best practice should not be underestimated.

• Enhance developers’ common sense of basic secure coding

Each member of the DevOps team, especially developers, should receive security training, in a bid to prevent security risks at the very beginning rather than detect and handle such risks after events.

For example, the software system architecture should be securely designed. Input data should be detected, preventing SQL injection and cross-site scripting (XSS) attacks. Databases, configuration files, and network traffic should be detected.

• Prohibit the use of known vulnerable components in the source code.

In the real development environment, code repositories downloaded by developers may contain old and known vulnerable software versions. To address this issue, some vendors provide tools such as an open-source firewall to disclose the security trend of code repositories to developers, so as to help them decide which software version to use. In this way, developers can have a clear understanding of which components and code repositories have severe vulnerabilities and will not download and use them.

Organizations that do not allow developers to directly user open-source software should consider using a secure code storage model. The information security team should cooperate with the development team to build, audit, and maintain component storage libraries within their organization.

Besides, some institutions ask developers to use qualified, approved, and standardized code repositories or components (such as authentication, authorization, key management, encryption, clickjacking protection, and input filtering), with the purpose of further reducing risks.

In a word, DevSecOps is such a concept that emphasizes the integration of DevOps and security, which brings a challenge to traditional software development modes and security management modes but impressive security gains at the same time.

(To be continued)

[i] Shiro, https://shiro.apache.org/

[ii] Spring Security, https://spring.io/projects/spring-security

[iii] What is DevSecOps, https://www.redhat.com/en/topics/devops/what-is-devsecops

[iv] 10 Things to Get Right for Successful DevSecOps, https://www.gartner.com/doc/reprints?id=1-4I8FBLQ&ct=171016&st=sg