Securing the edge: Harnessing Falco's power with Elastic Security for cloud workload protection

In the modern IT ecosystem, securing containerized applications in environments like Kubernetes is vital. Tools like Falco help address this need by providing tooling that can be integrated within Elastic Security. Falco is a cloud-native security tool that provides runtime security across hosts, containers, Kubernetes, and cloud environments. It leverages pre-defined, customizable Falco rules on Linux kernel events and other data sources through plugins, enabling the detection of abnormal behavior, potential security threats, and compliance violations. 

Expanding on our recent announcement of extended protections for cloud using CNCF open source security tools, this blog delves into how we've strengthened Elastic's capabilities by integrating with Falco.

Alternatively, for testing via Kubernetes — in this case, locally via Minikube — you can use the following. For more details, see Falco’s Quick Start Guide.

Once you have triggered a rule, check the Falcosidekick logs for successful POST requests to Elasticsearch. See examples below for expected output:

With these successful requests, you should now be able to observe the forwarded event in Kibana.

For in-depth information on setting up Falco with Elastic, consult Elastic’s setup instructions and the provided links mentioned above.

Generally speaking, the data you will see in Elastic Security starts pre-filtered through Falco’s rules. However, you can add some very open-ended Falco rules if you want to see something more akin to a raw telemetry feed. A word of caution: Doing this can be very resource intensive depending on how open-ended your rules are and how many events occur on a given machine.

The next section will explain the nuances of Falco’s detection rulesets and how to enable them.

The default /etc/falco/falco.yaml configuration file will then pick up on these rules.

After restarting the Falco service, you can immediately start experimenting with Falco's security monitoring rulesets.

In the next section, we will take a look at the fields that are available in Falco alert documents and how these fields can enable effective threat detection.

After this step, the necessary Falco ingest pipelines are installed and the Falco alerts should be properly parsed.

The Falco ingest pipeline contains several processors. The processor tagged as painless_map_event_type checks the evt.type field from the Falco output, converts it to lowercase, and maps it to a predefined set of categories based on specific syscall names or actions. If the syscall in the log matches one of the defined sets, it assigns an appropriate event type like access, admin, change, creation, etc. If no match is found, the event type defaults to info.

The full ingest pipeline can be found in Kibana > Stack Management > Ingest Pipelines > logs-falco.alerts-x.x.x.

Analyzing this ingest pipeline gives us a good idea of what kind of event types are available. Here’s an overview of the event types that we convert from Falco alerts to ECS and some of the syscalls that each category matches:

• Access: faccessat, read, open, etc.

• Admin: bdflush, ptrace, reboot, etc.

• Allowed: fallocate, finit_module.

• Change: llseek, chmod, chdir, ioctl, etc.

• Connection: accept, connect, socket, etc.

• Creation: clone, creat, fork, link, mkdir, etc.

• Deletion: delete_module, rmdir, unlink, etc.

• End: exit, shutdown, kill, etc.

• Group: fanotify_init, setgroups.

• Info: getpid, getcwd, stat, etc.

• Installation: utrap_install.

• Protocol: ipc.

• Start: execve, execv, swapon.

• User: userfaultfd.

Let's look at three practical examples to get a good idea of what kind of data we can expect to ingest from Falco:

1. Unexpected UDP Traffic

2. Launch Suspicious Network Tool on Host

3. Read sensitive file untrusted

In the first example, a UDP netcat reverse shell was initiated. This generated the “Unexpected UDP Traffic” rule. The entire event is displayed below:

After parsing, the event is ingested and looks like this:

Key fields available in this alert include:

• rule.name: The name of the Falco rule that triggered

• process.executable: The name of the process executable that initiated the event

• process.command_line: The process command line that triggered the alert

• falco.output_fields.fd.name: File description (FD) full name. If the FD is a file, this field contains the full path. If the FD is a socket, this field contains the connection tuple

• falco.output_fields.evt.type: The event's name (from the original Falco event)

• event.type: The converted event type (parsed by the ingest pipeline)

• falco.output_fields.fd.type: Type of FD; it can be file, directory, ipv4, pipe, or event

• falco.output_fields.fd.l4proto: The IP protocol of a socket; it can be tcp, udp, icmp, or raw

Similarly, we can look at the process execution event for this Netcat execution, as this triggered the “Launch Suspicious Network Tool on Host” rule. The full event:

This event is parsed by the ingest pipeline and shows up as:

For this process event, we see that the execve event is translated to the event.type value start. Additionally, we can see the parent process (process.parent.name) and the user (process.user.id) that executed it.

Finally, let’s take a look at the “Read sensitive file untrusted” alert that we triggered when reading the /etc/shadow file earlier. Presented below is the original alert.

After parsing, the following document is generated:

Here we see the falco.output_fields.fd.name containing the full path to the accessed file. Additionally, we can see the openat syscall being converted to the access event.type. Finally, we can see that the container.name is host, thus this event occurred on the host system.

To get a complete list of the supported fields with a corresponding description of what each field contains, visit Falco’s Supported Fields for Conditions and Outputs documentation.

Now that we have a decent understanding of the most important Falco alert fields, we can take a look at the alerts that are generated through Falco in the Elastic Security alert overview.

This rule queries any of the logs-* indices for documents that contain the following:

As the Falco documents are ingested with event.kind of alert, this rule promotes these documents to the Elastic Security Alerts overview as displayed below:

We can interact with these alerts similarly to how we would normally interact with any other alert. This means that we can assign users to the alert and assign the alert to a case:

Additionally, we can add rule exceptions to any Falco rule:

With the integration of Falco alerts into Elasticsearch and the use of ECS for standardization, centralized alert management and analysis become not only feasible but also highly efficient.

Falco detection rule alerts: 

• Basic Interactive Reconnaissance

• Redirect STDOUT/STDIN to Network Connection in Container

• System procs network activity

• Terminal shell in container

Node discovery
Identify the nodes's privileges and explore its environment.

• First, we can examine the environment variables to see if we can uncover any secrets. Depending on the secrets discovered, the attacker can pivot to take advantage of what information is uncovered.

From here, we can see that we are likely in a Kubernetes environment due to the environment variables. We can then use the default values expected to be on the pod to make calls to the internal Kubernetes API and see what permissions we have:

The attacker may also try to pivot to other pods with potentially greater access. To follow this course, one may try to list the pods via an API request:

Falco detection rule alerts: 

• Contact K8S API Server From Container

Pod escape via abusing mount points

• Now, it’s determined that the compromised host is a Kubernetes pod and does not have sufficient privileges to manipulate the pod API endpoint. The attacker may look to abuse trusted mount points from the Kubernetes host. One place to look for potential points of attack is in /proc/mounts. If found, the attacker may use this as a means of persistence through establishing a cron job or other form of recurring reverse shell.

From this output, we can see that the /etc/ directory has logical volume mappings for a number of the files. While this is not definitive, this can be an indication that this directory is mounted from another location, possibly from the Kubernetes host. To test this, the attacker then attempts to establish persistence in this directory by creating a cron job for a reverse shell.

This will create a cron job on the Kubernetes host that will not be lost when the pod is redeployed. Fortunately, Falco can detect these actions:

 Falco detection rule alerts: 

• Schedule Cron Jobs

Ptrace execution of a payload

• With persistence established, the attacker may now detonate their intended payload with the reverse shell to mitigate risk. For this example, we will attempt to run process injections to gain control of an ongoing process. For demonstrative purposes, we will use the Infector tool, specifically the infect.c file. An attacker using this tool could replace the shellcode constant in infect.c with the shellcode of a Meterpreter payload. In our case, the attacker will generate a payload on a separate machine using msfvenom.

Now, the attacker would take this shellcode output and replace the SHELLCODE variable in infect.c. Afterward the attacker would compile this file and run it to inject the payload into a desired PID, in this case 10247.

Falco detection rule alerts: 

• PTRACE attached to process

In this scenario, Falco was able to detect each core step our simulated attacker took. Let’s now take a look at an endpoint protection scenario.

Falco detection rule alerts: 

• Launch Suspicious Network Tool in Container

• Non sudo setuid

Discovery of the container

• The attacker checks for privileges, permissions, and potential privilege escalation opportunities or Docker escapes:

Falco detection rule alerts: 

• Basic Interactive Reconnaissance

• Non sudo setuid

Docker escape using nsenter

• Leveraging the container's privileges, the attacker escapes to the host using nsenter.

Falco detection rule alerts: 

• Change thread namespace

Persistence on the host

• The attacker establishes persistence by creating a new user and scheduling a cron job:

Falco detection rule alerts: 

• Unexpected UDP Traffic

• Execution from /dev/shm

• Write below etc

• Schedule Cron Jobs

• System procs network activity

Discovery of the host

• The attacker maps out the host environment, searching for lateral movement and sensitive information stealing opportunities.

Falco detection rule alerts: 

• Read sensitive file untrusted

• Basic Interactive Reconnaissance

• Schedule Cron Jobs

Through these simulations, we've demonstrated how Falco can detect various malicious activities at different stages of an attack lifecycle. Each step of the attack triggers unique Falco rules, allowing for prompt detection and alerting within the Elastic Security environment. This showcases the power of Falco's fine-grained syscall monitoring in conjunction with Elastic Security's centralized management for effective threat response.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.