Skip to content

ROX-33217: Instrument inode tracking on directory being created path mkdir#465

Open
JoukoVirtanen wants to merge 18 commits intomainfrom
jv-ROX-33217-instrument-inode-tracking-on-directory-being-created-path_mkdir
Open

ROX-33217: Instrument inode tracking on directory being created path mkdir#465
JoukoVirtanen wants to merge 18 commits intomainfrom
jv-ROX-33217-instrument-inode-tracking-on-directory-being-created-path_mkdir

Conversation

@JoukoVirtanen
Copy link
Copy Markdown
Contributor

@JoukoVirtanen JoukoVirtanen commented Apr 1, 2026

Description

Directory creation events need to be handled correctly. When a directory is created in a tracked directory its inode should be added to a hash set in kernel space. In user space an entry needs to be added into a map with the inode as the key and file path as the value.

An alternative approach can be found at #449

Checklist

  • Investigated and inspected CI test results
  • Updated documentation accordingly

Automated testing

  • Added unit tests
  • Added integration tests
  • Added regression tests

If any of these don't apply, please comment below.

Testing Performed

Checked metrics

$ grep path_mkdir */metrics 
test_mkdir_ignored/metrics:# HELP stackrox_fact_kernel_path_mkdir_events Events processed by the path_mkdir LSM hook.
test_mkdir_ignored/metrics:# TYPE stackrox_fact_kernel_path_mkdir_events counter
test_mkdir_ignored/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="RingbufferFull"} 0
test_mkdir_ignored/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Added"} 0
test_mkdir_ignored/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Error"} 0
test_mkdir_ignored/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Ignored"} 2
test_mkdir_ignored/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Total"} 3
test_mkdir_nested/metrics:# HELP stackrox_fact_kernel_path_mkdir_events Events processed by the path_mkdir LSM hook.
test_mkdir_nested/metrics:# TYPE stackrox_fact_kernel_path_mkdir_events counter
test_mkdir_nested/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="RingbufferFull"} 0
test_mkdir_nested/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Added"} 0
test_mkdir_nested/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Total"} 3
test_mkdir_nested/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Ignored"} 0
test_mkdir_nested/metrics:stackrox_fact_kernel_path_mkdir_events_total{label="Error"} 0
$ grep d_instantiate */metrics 
test_mkdir_ignored/metrics:# HELP stackrox_fact_kernel_d_instantiate_events Events processed by the d_instantiate LSM hook.
test_mkdir_ignored/metrics:# TYPE stackrox_fact_kernel_d_instantiate_events counter
test_mkdir_ignored/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Error"} 0
test_mkdir_ignored/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Added"} 2
test_mkdir_ignored/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Ignored"} 36
test_mkdir_ignored/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="RingbufferFull"} 0
test_mkdir_ignored/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Total"} 37
test_mkdir_nested/metrics:# HELP stackrox_fact_kernel_d_instantiate_events Events processed by the d_instantiate LSM hook.
test_mkdir_nested/metrics:# TYPE stackrox_fact_kernel_d_instantiate_events counter
test_mkdir_nested/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="RingbufferFull"} 0
test_mkdir_nested/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Added"} 6
test_mkdir_nested/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Error"} 0
test_mkdir_nested/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Total"} 115
test_mkdir_nested/metrics:stackrox_fact_kernel_d_instantiate_events_total{label="Ignored"} 112

@JoukoVirtanen JoukoVirtanen marked this pull request as ready for review April 1, 2026 23:30
@JoukoVirtanen JoukoVirtanen requested a review from a team as a code owner April 1, 2026 23:30
struct inode* parent_inode_ptr = BPF_CORE_READ(parent_dentry, d_inode);
inode_key_t parent_inode = inode_to_key(parent_inode_ptr);

if (should_track_mkdir(parent_inode, path) != PARENT_MONITORED) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally had

if (should_track_mkdir(parent_inode, path) == NOT_MONITORED) {

However, test_nonrecursive_wildcard::test_wildcard.py failed with that change. The reason was that was the inodes of all directories created in that test were tracked and subsequently all files in those directories were tracked regardless if they matched the wildcard pattern or not.

#define FMODE_PWRITE ((fmode_t)(1 << 4))
#define FMODE_CREATED ((fmode_t)(1 << 20))

// File type constants from linux/stat.h
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a permalink to the definition?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines +309 to +314
umode_t mode = BPF_CORE_READ(inode, i_mode);
if (!S_ISDIR(mode)) {
bpf_map_delete_elem(&mkdir_context, &pid_tgid);
m->d_instantiate.ignored++;
return 0;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check if m->d_instantiate.ignored actually increases, I have a feeling this condition should never be met.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m->d_instantiate.ignored increases, but I don't know if it is due to this or other locations with m->d_instantiate.ignored++;.


m->d_instantiate.total++;

if (inode == NULL) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should check if inode can actually be null, if so, we probably still want to try and cleanup the mkdir_context map entry.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is now cleanup even when the inode is null.

}

// d_instantiate doesn't support bpf_d_path, so we use false and rely on the stashed path from path_mkdir
__submit_event(event, m, FILE_ACTIVITY_CREATION, filename, inode, parent_inode, false);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling __submit_event will cause a full event to be filled in, including process information. However, mkdir is mostly needed for inode tracking and probably not interesting enough to send an event over gRPC, so we might want to look into having an abbreviated version that will just submit a FILE_ACTIVITY_MKDIR event that will be processed by HostScanner and dropped after adding to the userspace map.

Comment on lines +86 to +96
struct {
__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
__type(key, __u32);
__type(value, struct mkdir_context_t);
__uint(max_entries, 1);
} mkdir_context_heap SEC(".maps");

__always_inline static struct mkdir_context_t* get_mkdir_context() {
unsigned int zero = 0;
return bpf_map_lookup_elem(&mkdir_context_heap, &zero);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be unused.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, but now I'm wondering why we are not just getting an entry from mkdir_context directly and filling that in, instead of getting an entry from mkdir_context_heap and then copying it over.

There's a few things you could try for when the entry doesn't actually exist.

  • If you drop BPF_F_NO_PREALLOC from mkdir_context, I'm fairly certain you should always get a value back.
  • You can try using bpf_map_lookup_elem and if it fails, create a new entry passing value as NULL.
  • If that doesn't work, you can just pass a pointer to a default value from a const global, which will end up in .rodata section.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you also need to add your metrics here:

fact/fact-ebpf/src/lib.rs

Lines 123 to 127 in eb0b3a9

m.file_open = m.file_open.accumulate(&other.file_open);
m.path_unlink = m.path_unlink.accumulate(&other.path_unlink);
m.path_chmod = m.path_chmod.accumulate(&other.path_chmod);
m.path_chown = m.path_chown.accumulate(&other.path_chown);
m.path_rename = m.path_rename.accumulate(&other.path_rename);

I will look into improving how we handle these, it is getting very repetitive and having it scattered among multiple files is very error prone.

Comment on lines +23 to +25
os.mkdir(level1)
os.mkdir(level2)
os.mkdir(level3)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] We can probably just use os.makedirs on level3.

from event import Event, EventType, Process


def test_mkdir_nested(monitored_dir, server):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can parametrize the final directory with UTF-8 characters?

Comment on lines +86 to +96
struct {
__uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
__type(key, __u32);
__type(value, struct mkdir_context_t);
__uint(max_entries, 1);
} mkdir_context_heap SEC(".maps");

__always_inline static struct mkdir_context_t* get_mkdir_context() {
unsigned int zero = 0;
return bpf_map_lookup_elem(&mkdir_context_heap, &zero);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, but now I'm wondering why we are not just getting an entry from mkdir_context directly and filling that in, instead of getting an entry from mkdir_context_heap and then copying it over.

There's a few things you could try for when the entry doesn't actually exist.

  • If you drop BPF_F_NO_PREALLOC from mkdir_context, I'm fairly certain you should always get a value back.
  • You can try using bpf_map_lookup_elem and if it fails, create a new entry passing value as NULL.
  • If that doesn't work, you can just pass a pointer to a default value from a const global, which will end up in .rodata section.

@JoukoVirtanen JoukoVirtanen force-pushed the jv-ROX-33217-instrument-inode-tracking-on-directory-being-created-path_mkdir branch from 4a6a257 to b0ef4fe Compare April 6, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants