Re: Verifier error: variable stack access var_off

Teng Qin

Firstly, note that bpf_probe_read_str adds an extra \0 to the read string. So your max should be sizeof(data.argv) - 1 instead in order for data.argv[len] = ' ' to work (from Verifier's perspective, logically you don't need that extra delimiter~)

Then, Yonghong had a patch a few month ago addressing very similar issue. See the example in patch series
bpf: improve verifier ARG_CONST_SIZE_OR_ZERO semantics
Does your Kernel have those patches?

However, even with all those the data.argv[len] = ' ' part still fails with something about stack offset not being fixed. I will try debug more to see how to fix that. For now, you can use a per-CPU array of size 1 for the data instead of allocating it on the stack. The following works for me:
#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
#include <linux/fs.h>
#define ARGSIZE  128
struct data_t {
    char argv[ARGSIZE];
BPF_PERCPU_ARRAY(mem, struct data_t, 1);
// Here's what I'm trying to do. Let's say this has:
//   __argv[0] = "ls"
//   __argv[1] = "-l"
// I'm trying to create a buffer with "ls -l", by doing bpf_probe_read_str() for
// each element into the buffer, while keeping track of the length of
// the previous read so I can insert a space delimiter at that offset,
// and begin the next read after the delimiter.
int on_event(struct pt_regs *ctx,
    const char __user *filename,
    const char __user *const __user *__argv,
    const char __user *const __user *__envp)
    int zero = 0;
    struct data_t* data = mem.lookup(&zero);
    if (!data)
      return 0;
    uint64_t max = sizeof(data->argv) - 1;
    const char *argp = NULL;
    bpf_probe_read(&argp, sizeof(argp), (void *)&__argv[0]);
    uint64_t len = bpf_probe_read_str(&(data->argv), max, argp);
    len &= 0xffffffff; // to avoid: "math between fp pointer and register errs"
    bpf_trace_printk("len: %d\\n", len); // sanity check: len is indeed valid
    data->argv[len] = ' ';
    events.perf_submit(ctx, data, len);
    return 0;

Join to automatically receive all group messages.