Overview & Analysis of a Threat Intelligence Ecosystem

Overview & Analysis of a Threat Intelligence Ecosystem

fevereiro 6, 2017 | Adeline Zhang
Threat Analysis

Authors: Richard Zhao, CTO & Cody Mercer, Senior Intelligence Threat Researcher

Security Event Investigation and Threat Intelligence

Over a year ago I purposed the three main tenants encompassing a successful Threat Intelligence framework:

  1. Define a system infrastructure for security event disclosure and case analysis.
  2. Clearly delineate security disclosure responsibilities to respective parties.
  3. Cultivate a security data and response platform.

These observed measures will gradually promote industry standard security practices at the strategic level in terms of effectiveness, accuracy, correctness, and timeliness.

My previous document which is a companion to this report may be viewed here.

An effective security posture with secure defense against advanced threats relies on threat intelligence data. The progressive state of evolving TI constructs has begun to gain notoriety within the cyber-security world and is now considered a primary staple and an industry standard in defense capabilities.

At the present moment, threat intelligence may be consumed or obtained via various channels to include collective intel community gatherings, cyber-security chat forums, and IT industry reports. Additionally, TI subscriptions are readily available and are a highly recognized means of not only sharing TI data, but ingesting it as well. All of the above-mentioned outlets demonstrate a successful means to accumulate viable TI data.

However, it is important to recognize that great challenges still exist within the TI arena that prevents establishing a successful ecosystem. Obstacles remain evident when developing the foundation for an ecological system of threat intelligence to include collection, analysis, accumulation, sharing, timeliness, and application.

Threat intelligence may be sourced from four aspects across the industry:

  1. Data that is purchased and shared within the industry and alliances.
  2. Operating procedures of the security protection systems.
  3. Independent research in the field of TI.
  4. Security event investigation and tracking activities.

The last element supports live, continuous, and verifiable sources of TI which is essential to the entire threat intelligence ecology. Extensive event analysis that covers a broad range of industries and regions along with shared threat intelligence and best practices will prove to be powerful and effective weapon against known and unknown IOCs’.

Let’s use the American service provider Verizon as an example. Verizon has been releasing annual data security situation reports for many years. Initially, only Verizon collected data for such reports, but currently a total of over 70 organizations contribute to this report to include the United States Department of Homeland Security (DHS), US-CERT, Secret Service, and many other security vendors and service providers. Security events that are covered by such reports have steadily increased since 2013, with nearly 80,000 in 2015. Moreover, note that the identifiable data leak events included in the annual reports increased from 761 in 2010 to 2122 in 2015.

Figure 1: Accumulation of data included in Data Breach Investigations Report (DBIR)

The preceding table is an indication on how the American counterpart additions have proven to be a practicable and useful source of threat intelligence.

As the Cyber Security Law and Amendment to Criminal Law are released and come into force, those who are convicted of serious cyber security events will be held legally responsible. The Amendment to Criminal Law requires that the following be added to Article 286 in the Criminal Law, “Internet service providers that cause serious consequences arising from disclosure of user information, will be sentenced to fixed-term of imprisonment of not more than three years, criminal detention, or control, in addition to fines; or are to be fined”. Evidently the new legislation will treat cyber security events in a more severe manner and encourage owners and operators of information systems to pay closer attention to nefarious acts with an increase in the investment of cyber security controls.

As the new cyber security legislation further increases the price to be paid for security events that are reported and revealed, it is becomes more difficult to effectively investigate. Often incentives for professionally analyzed events and correlation are absent which undermines the foundation for sustainable development of the threat intelligence ecology.

“Living Loop” and “Dead Loop”

You often find quarreling among cyber-attackers and red/blue security teams. When the strength of one side gains momentum, the other side may lie low. After an attack is initiated, a security protection system may:

  1. fail to detect
  2. discover acts of intrusion
  3. or accurately detect the attack

In the first case, the attacker easily achieves his or her purpose of gaining profits and therefore is eager for greater notoriety. At the same time, the attacker obtains more “intelligence” about the defending side, preparing himself or herself for further attacks. In the second case, the protection system detects other IOCs’ of the attack and subsequently, the defending side may ignore such symptoms as is described in the first case.

Alternatively, the defending side may locate the source by tracking the attack, and then upgrade the threat intelligence and detection system for accurate detection and prevention of this particular attack. Thus, evolving into the third case as previously mentioned were the attack is blocked and ends in a wild-goose chase. Although, it is possible that the attack source is located and the attacker may be brought to justice. During this process, the defending side through adequate investigation and forensics enriches its threat intelligence database. Additionally, the investigation promotes a greater understanding of attack resources, tactics, techniques, and procedures (TTP). Moreover, it becomes increasingly more difficult for attackers to develop new attack vectors and tools that risk detection. The following figure illustrates the “living loop” and “dead loop”.

Figure 2: “Living loop” and “dead loop” in a cyber attack

As indicated in the first and second case, both prove to promote increased attacker profits, increased frequency of attacks, and greater adversity in cyber security environments. Seemingly, the third case and the investigation of the second case give rise to a reinforced protection capability with an increased possibility of locating the attack source. As a result, the cyber security environment will be clean under effective governance in what is called the “living loop”. Figure 2 indicate that the most clear-cut divide between the “living loop” and “dead loop” is the detection of symptoms of an attack event. The defending side may choose to ignore the symptoms or initiate an investigation depending on its capabilities, resources, and the priority given to the input-output ratio.

 Ignore or Investigate?

Complete security is not possible and is a widely accepted notion universally acknowledged in the security industry. Understandably, “compromises” inevitably occur in a network attributing to defects in the protection system. These “compromises” often fall into one of the four categories:

  1. may be perceived
  2. may be ignored
  3. may be investigated
  4. may be analyzed for the root cause

The scenes of a compromised asset usually conveys a lot of information about attackers and investigating such a scene calls for a large amount of resources. The output provided supports information relevant to root cause analysis, attacker profile information, TTP’s, and the cause of protection failures. In contrast, the preceding output and the value of such output are quite uncertain for the small to medium-sized security operations teams. Therefore, small and medium-sized enterprises with limited security capacities do not have the capabilities that large enterprises do and are unable to conduct proper investigations. This is one of the predicaments of cyber security protection. The professional division of duties is conducive to the resolution of the “incapability” issue. The issue of “unwillingness” can be resolved by appropriately rewarding those who collect more threat intelligence following root cause analysis instead of punishing those with lack of proper security protocols.

Threat Intelligence and Cyber Insurance

By reference to rules governing traffic accidents and insurance, the introduction of the “insurance” role in the cyber security ecosystem will help break the “externality” and “lemon-market” traps. The United States has made great progress in cyber-security insurance with at least 50 insurance companies providing cyber insurance products by 2014. 24% of American businesses have bought cyber insurance with a combined cost of $2 billion. Figure 3 depicts the introduction of the “insurance” role into the cyber-security realm and falls into two main categories:

  1. before the security event
  2. after the security event

Figure 3 Threat intelligence and cyber insurance

The regulatory authority outlines and defines the corresponding necessary rules. An information system operator independently develops security products and services, or buys such products and services from a professional security provider to strengthen their network protection system. Additionally, the operator buys the cyber insurance to cover all other “residual risks”. The insurance company assesses the operator’s current security depth and the “reliability” of the security products and services adopted by the operator before calculating the “insurance rate”. In the case of a security compromise, the information system operator asks the professional service provider to investigate and analyze the event, and then submits the detailed event information and analysis result to the insurance company for compensation. After confirming the event cause and treatment measures, the insurance company pays the compensation and updates its case library, threat intelligence database, and insurance rate accordingly.

Through an alternative form of “redemption”, the information system operator becomes willing to “investigate” rather than “ignore” security events. The professional security provider can obtain the first-hand “threat intelligence” for investigation and forensics and evaluate the “effectiveness” of an entities security products and services. Moreover, it is objectively verified by the security event analysis review and reflected in the defined “insurance rate”. In this way, the ecological security protection system can defend against security threats more effectively.


Sun Zi once said, “Know the enemy and know yourself, and you can fight a hundred battles with no danger of defeat.” Threat intelligence is the key for you to “know the enemy and know yourself” in the network attack and defense field. Seemingly, the continuous accumulation of security event investigation data and analysis is paramount in effective, actionable threat intelligence.

Undoubtedly, it is extremely important to legislatively increase the “punishment” for recognized security compromises while also upholding cyber security awareness of information system owners and operators. Cyber security events are inevitable and we should encourage investigation and forensics techniques while enforcing efficient analysis processes and results. This will improve the current situation where, in China, the basic data of cyber security is deficient and relevant standards and best practices lack data support and verification.

To achieve this goal, it is necessary to design and build the corresponding regulatory and ecological environments such as introducing the cyber insurance role to realize case accumulation and sustainable development of threat intelligence through cyber security event claims.

Download WhitePaper