System process metricset

edit

The System process metricset provides process statistics. One document is provided for each process.

This metricset is available on:

  • FreeBSD
  • Linux
  • macOS
  • Windows

Configuration

edit
processes

When the process metricset is enabled, you can use the processes option to define a list of regexp expressions to filter the processes that are reported. For more complex filtering, you should use the processors configuration option. See Processors for more information.

The following example config returns metrics for all processes:

metricbeat.modules:
- module: system
  metricsets: ["process"]
  processes: ['.*']
process.cgroups.enabled

When the process metricset is enabled, you can use this boolean configuration option to disable cgroup metrics. By default cgroup metrics collection is enabled.

The following example config disables cgroup metrics on Linux.

metricbeat.modules:
- module: system
  metricsets: ["process"]
  process.cgroups.enabled: false
process.cmdline.cache.enabled
This metricset caches the command line args for a running process by default. This means if you alter the command line for a process while this metricset is running, these changes are not detected. Caching can be disabled by setting process.cmdline.cache.enabled: false in the configuration.
process.env.whitelist

This metricset can collect the environment variables that were used to start the process. This feature is available on Linux, Darwin, and FreeBSD. No environment variables are collected by default because they could contain sensitive information. You must configure the environment variables that you wish to collect by specifying a list of regular expressions that match the variable name.

metricbeat.modules:
- module: system
  metricsets: ["process"]
  process.env.whitelist:
  - '^PATH$'
  - '^SSH_.*'
process.include_cpu_ticks

By default the cumulative CPU tick values are not reported by this metricset (only percentages are reported). Setting this option to true will enable the reporting of the raw CPU tick values (for user, system, and total CPU time).

metricbeat.modules:
- module: system
  metricsets: ["process"]
  process.include_cpu_ticks: true
process.include_per_cpu
By default metrics per cpu are reported when available. Setting this option to false will disable the reporting of these metrics.
process.include_top_n
These options allow you to filter out all processes that are not in the top N by CPU or memory, in order to reduce the number of documents created. If both the by_cpu and by_memory options are used, the union of the two sets is included.
process.include_top_n.enabled
Set to false to disable the top N feature and include all processes, regardless of the other options. The default is true, but nothing is filtered unless one of the other options (by_cpu or by_memory) is set to a non-zero value.
process.include_top_n.by_cpu
How many processes to include from the top by CPU. The processes are sorted by the system.process.cpu.total.pct field. The default is 0.
process.include_top_n.by_memory
How many processes to include from the top by memory. The processes are sorted by the system.process.memory.rss.bytes field. The default is 0.

Monitoring Hybrid Hierarchy Cgroups

edit

The process metricset supports both V1 and V2 (sometimes called unfied) cgroups controllers. However, on systems that are running a hybrid hierarchy, with both V1 and V2 controllers, metricbeat will only report one of the hierarchies for a given process. Is a process has both V1 and V2 hierarchies associated with it, metricbeat will check to see if the process is attached to any V2 controllers. If it is, it will report cgroups V2 metrics. If not, it will report V1 metrics.

A workaround is also required if metricbeat is running inside docker on a hybrid system. Within docker, metricbeat won’t be able to see any V2 cgroups components. If you wish to monitor cgroups V2 from within docker on a hybrid system, you must mount the unified sysfs hierarchy (usually /sys/fs/cgroups/unified) inside the container, and then use system.hostfs to specify the filesystem root within the container.

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

Fields

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "system.process",
        "duration": 115000,
        "module": "system"
    },
    "metricset": {
        "name": "process",
        "period": 10000
    },
    "process": {
        "args": [
            "/tmp/go-build2412913392/b001/process.test",
            "-test.paniconexit0",
            "-test.timeout=10m0s",
            "-test.v=true",
            "-data",
            "-test.run=TestData"
        ],
        "command_line": "/tmp/go-build2412913392/b001/process.test -test.paniconexit0 -test.timeout=10m0s -test.v=true -data -test.run=TestData",
        "cpu": {
            "pct": 0.0013,
            "start_time": "2022-01-31T21:06:54.000Z"
        },
        "executable": "/tmp/go-build2412913392/b001/process.test",
        "memory": {
            "pct": 0.0004
        },
        "name": "process.test",
        "parent": {
            "pid": 3334369
        },
        "pgid": 3334369,
        "pid": 3334614,
        "state": "sleeping",
        "working_directory": "/home/alexk/go/src/github.com/elastic/beats/metricbeat/module/system/process"
    },
    "service": {
        "type": "system"
    },
    "system": {
        "process": {
            "cgroup": {
                "cgroups_version": 2,
                "cpu": {
                    "id": "session-27.scope",
                    "path": "/user.slice/user-1000.slice/session-27.scope",
                    "pressure": {
                        "full": {
                            "10": {
                                "pct": 0
                            },
                            "300": {
                                "pct": 0
                            },
                            "60": {
                                "pct": 0
                            },
                            "total": 3431214
                        },
                        "some": {
                            "10": {
                                "pct": 0
                            },
                            "300": {
                                "pct": 0
                            },
                            "60": {
                                "pct": 0
                            },
                            "total": 3626431
                        }
                    },
                    "stats": {
                        "system": {
                            "norm": {
                                "pct": 0
                            },
                            "ns": 70940728,
                            "pct": 0
                        },
                        "usage": {
                            "norm": {
                                "pct": 0
                            },
                            "ns": 502653872,
                            "pct": 0
                        },
                        "user": {
                            "norm": {
                                "pct": 0
                            },
                            "ns": 431713143,
                            "pct": 0
                        }
                    }
                },
                "id": "session-27.scope",
                "memory": {
                    "id": "session-27.scope",
                    "mem": {
                        "events": {
                            "high": 0,
                            "low": 0,
                            "max": 0,
                            "oom": 0,
                            "oom_kill": 0
                        },
                        "low": {
                            "bytes": 0
                        },
                        "usage": {
                            "bytes": 2462191616
                        }
                    },
                    "memsw": {
                        "events": {
                            "fail": 0,
                            "high": 0,
                            "max": 0
                        },
                        "low": {
                            "bytes": 0
                        },
                        "usage": {
                            "bytes": 0
                        }
                    },
                    "path": "/user.slice/user-1000.slice/session-27.scope",
                    "stats": {
                        "active_anon": {
                            "bytes": 19554304
                        },
                        "active_file": {
                            "bytes": 4419584
                        },
                        "anon": {
                            "bytes": 2368811008
                        },
                        "anon_thp": {
                            "bytes": 205520896
                        },
                        "file": {
                            "bytes": 48955392
                        },
                        "file_dirty": {
                            "bytes": 0
                        },
                        "file_mapped": {
                            "bytes": 14315520
                        },
                        "file_thp": {
                            "bytes": 0
                        },
                        "file_writeback": {
                            "bytes": 0
                        },
                        "htp_collapse_alloc": 28,
                        "inactive_anon": {
                            "bytes": 2370981888
                        },
                        "inactive_file": {
                            "bytes": 24166400
                        },
                        "kernel_stack": {
                            "bytes": 2621440
                        },
                        "major_page_faults": 45,
                        "page_activate": 12197,
                        "page_deactivate": 0,
                        "page_faults": 3623174,
                        "page_lazy_free": 0,
                        "page_lazy_freed": 0,
                        "page_refill": 0,
                        "page_scan": 0,
                        "page_steal": 0,
                        "page_tables": {
                            "bytes": 22589440
                        },
                        "per_cpu": {
                            "bytes": 0
                        },
                        "shmem": {
                            "bytes": 20369408
                        },
                        "shmem_thp": {
                            "bytes": 0
                        },
                        "slab": {
                            "bytes": 13215832
                        },
                        "slab_reclaimable": {
                            "bytes": 8919768
                        },
                        "slab_unreclaimable": {
                            "bytes": 4296064
                        },
                        "sock": {
                            "bytes": 16384
                        },
                        "swap_cached": {
                            "bytes": 0
                        },
                        "thp_fault_alloc": 91,
                        "unevictable": {
                            "bytes": 0
                        },
                        "workingset_activate_anon": 0,
                        "workingset_activate_file": 0,
                        "workingset_node_reclaim": 0,
                        "workingset_refault_anon": 0,
                        "workingset_refault_file": 1381,
                        "workingset_restore_anon": 0,
                        "workingset_restore_file": 0
                    }
                },
                "path": "/user.slice/user-1000.slice/session-27.scope"
            },
            "cmdline": "/tmp/go-build2412913392/b001/process.test -test.paniconexit0 -test.timeout=10m0s -test.v=true -data -test.run=TestData",
            "cpu": {
                "start_time": "2022-01-31T21:06:54.000Z",
                "system": {
                    "ticks": 140
                },
                "total": {
                    "norm": {
                        "pct": 0.0013
                    },
                    "pct": 0.0158,
                    "ticks": 340,
                    "value": 340
                },
                "user": {
                    "ticks": 200
                }
            },
            "fd": {
                "limit": {
                    "hard": 524288,
                    "soft": 524288
                },
                "open": 11
            },
            "memory": {
                "rss": {
                    "bytes": 23404544,
                    "pct": 0.0004
                },
                "share": 14946304,
                "size": 1725992960
            },
            "state": "sleeping",
            "num_threads": 42
        }
    },
    "user": {
        "name": "alexk"
    }
}