Structured threat hunting: One way Microsoft Threat Experts prioritizes customer defense>
Microsoft Blog
Todayâs threat landscape is incredibly fast-paced. New campaigns surface all the time, and the amount of damage that they can cause is not always immediately apparent. Security operations centers (SOCs) must be equipped with the tools and insight to identify and resolve potentially high-impact threats before attackers set up persistence mechanisms, exfiltrate data, or deploy payloads such as ransomware.

Microsoft Threat Experts allows organizations to collaborate with Microsoft analysts to benefit from their expertise in tackling critical incidents. This collaborative relationship also gives Microsoft analysts the opportunity to gain invaluable insight into real-world threats, how attackers operate inside enterprise networks, and how security operations teams function. This creates an environment conducive to mutual learning and innovation, which helps improve our processes, protections, and services.

This process starts with Microsoft analysts formulating hypotheses to explain suspicious behavior discovered within our data. If the hypotheses pass our initial quality checks, we perform an automated hunt for and collect observations that could ultimately confirm or deny our suspicions.

Once weâve gathered more evidence, the observations are grouped into potential threats and run through a variety of computations to evaluate the possible impact. One of the key figures we use for evaluating potential threats is the amount of diversity we see across our observations. A more diverse set of observations indicates that a potential threat is more likely to have a broad impact.

As the first step in our process, threat hunters formulate a hypothesis around data related to a potential threat, such as, âThe attacker remotely executed code by exploiting a vulnerability in a system process.â After validating the soundness of the hypothesis by measuring the signal-to noise ratio and using known data sets to ensure that the accuracy is within acceptable limits, the hypothesis is modeled in our hunting systems, which automatically perform data collection, correlation, and enrichment.

These automated systems collect observations through our telemetry, from multiple devices and often from different stages of an attack. Each observation represents a single instance of a hypothesis â in our example, here is the system process, here is what the system process did, and here are the arguments that were passed to it. Examining the observations associated with the hypothesis helps analysts determine if that instance of the hypothesis is valid.

Using diversity to prioritize potential threats How can we measure the diversity of different aspects of a potential threat. Microsoft Threat Experts uses entropy, borrowed from information theory. Entropy measures how many bits (or yes/no questions) it would take, on average, to identify a random element of a set if we know the elements of the set.

A set containing elements that are all alike has zero entropy. Meanwhile, a set with more distinct elements, or the distinct elements in more equal proportions, will have higher entropy.

Calculating the final priority score Since our final score takes into consideration many different kinds of information about the potential threat, we need a way to combine these values. To do this, we convert each the results of each calculation into a p-value.

A p-value represents the percentage of potential threats weâd expect to have a certain value or larger in that category. For example, if only 5% of the MITRE technique entropy values were larger than the value from our calculation, then the p-value for our potential threatâs hypothesis entropy would be 0.05.

Validating the potential threat The final prioritization score is used to sort potential threats, so that the most critical potential threats are analyzed the most quickly. When a potential threat is ready, a dedicated team of security experts look over the results and perform deep analysis to determine the threatâs validity.
Link: https://www.microsoft.com/security/blog/2021/12/02/structured-threat-hunting-one-way-microsoft-threat-experts-prioritizes-customer-defense/