Real-time threat detection in the cloud

Real-time threat detection in the cloud

Organizations have moved business-critical applications to the cloud, and attackers have followed suit. 2020 was a inflection point; the first year we saw more breaches and incidents of assets in the cloud than on premises. We know there are bad actors out there; if you are operating in the cloud, how do you detect threats?

The cloud is different. Services are no longer confined to one place with one entrance or one exit.

DevOps/Cloud-Native Live!  Boston

Traditionally, services have been deployed in data centers on servers that were close to each other, physically interconnected, and data had only one way in or out of that data center. Security was based on a perimeter; our realm was easily protected through firewalls like a medieval city surrounded by high thick walls, limiting traffic and attacks through solid gates and defending through fine loopholes.

Aerial view of Monteriggioni ( by Maurizio Moro5153, July 14, 2020. Creative Commons BY 3.0 License.

Today, services are distributed and operated in environments with limited perimeters. Developers, operators and users are located all over the world. Having all of those users access services through a single location will impact productivity and the user experience. Your services are no longer confined to one place where there is only one way in or out. If before we compared our infrastructure with a medieval town, now it is more like an amusement park.

Blurry Rides Amusement Park Amusement Fair

An amusement park packed with rides, multiple entrances and exits, and many more possibilities for the actors to behave unexpectedly. A distributed infrastructure based on cloud technologies requires detection of threats from myriad sources. So many actors interacting in so many different ways increases the number of potential events and the amount of information that needs to be monitored.

Threat detection: a delicate balance

A common approach to threat detection starts with sending logs to a centralized repository, then looks for indicators of suspicious behavior or configuration changes that increase risk. That takes time, it’s like trying to identify a moving target. Copying records out of the cloud and storing an additional copy can be an expensive and operational headache. And, more importantly, this approach delays the ability to detect and respond to threats.

Obviously, the closer the monitoring tools are to the source of an event, the better the response time. However, this could add complexity and increase costs. Also, there are still too many steps involved in this pipeline. Couldn’t this be improved in some way?

Inspect logs in real time with flow detection

What if instead of trying to police a fortified city with well-defined entry points, we started thinking about how to police activity within that amusement park? Imagine smart security cameras constantly on the alert and looking for anomalous behavior, reacting accordingly and triggering alarms when necessary. Translated to cloud infrastructures, that means the most accurate way to monitor cloud security is through flow detection.

Flow sensing is an ongoing process that collects, analyzes, and reports data in motion. With a streaming detection process, logs are inspected in real time. This real-time detection allows you to identify unexpected changes in permissions and access rights to services, as well as unusual activities that may indicate the presence of an intruder or, in the worst case, data leakage. Based on that idea, the open source community offers a solution: Falco.

Falco is an open source runtime security tool, often described as a security camera for modern cloud infrastructure. Falco is an incubation-hosted tier project at the Cloud Native Computing Foundation (CNCF). Originally designed to monitor workloads, Falco focuses on collecting system calls from running endpoints, such as hosts or containers running applications, and collecting granular data from the source in those containers, understanding the details of what What do the apps do?

Obviously, not everything in our infrastructure is hosts and containers. Organizations also benefit from numerous external services offered by their cloud provider(s). Fortunately, the cloud provider also makes it easy to share valuable information generated by each service, and that information is useful for monitoring. This is where Falco behaves differently than conventional alternatives: since the open source project can consume additional types of data, it can ingest and digest that information in real time to generate alerts on the fly.

Consider how this works in an Amazon Web Services (AWS) environment, for example. Almost everything that happens in AWS is tracked and recorded in the AWS cloud version of logs, called CloudTrail. By monitoring CloudTrail logs, you can detect unexpected behavior, configuration changes, intrusions, and data theft, not only from existing services but also from newly launched ones. Connecting Falco to CloudTrail gives you the flexibility to manage your rules in one place. Not having to store your logs externally also reduces bandwidth and storage costs.

Respond to threats faster

Time is a critical factor in reducing risk. Real-time log analysis and leveraging flow detection allows you to immediately spot suspicious activity and trigger an alert for further investigation.

The goal of flow detection, and of projects like Falco, is to align an effective security approach with the realities of modern architectures. Doing so enables faster detection time and equips security teams with evolving, community-driven innovation, all of which are critical to effective security with modern stacks. The reality is that the cloud is a new beast and enables a new approach to everything, including security. This is your chance to get it right!

Leave a Comment