Re: Building BPF programs and kernel persistence
Thanks for the reply Andrii.
Managed to get a build working outside of the kernel tree for BPF programs.
The two major things that I learned were that first, the order in which files are
included in the build command is more important than I previously thought.
The second thing was learning how clang deals with asm differently than gcc.
I had to use samples/bpf/asm_goto_workaround.h to fix those errors.
The meat of the makefile is as follows:
CLANGINC := /usr/lib/llvm-10/lib/clang/10.0.0/include
INC_FLAGS := -nostdinc -isystem $(CLANGINC)
EXTRA_FLAGS := -O3 -emit-llvm
linuxhdrs := /usr/src/linux-headers-$(shell uname -r)
LINUXINCLUDE := -include $(linuxhdrs)/include/linux/kconfig.h \
-include asm_workaround.h \
COMPILERFLAGS := -D__KERNEL__ -D__ASM_SYSREG_H \
-D__BPF_TRACING__ -D__TARGET_ARCH_$(ARCH) \
# Builds all the targets from corresponding .c files
$(CC) $(INC_FLAGS) $(COMPILERFLAGS) \
$(LINUXINCLUDE) $(LIBBPF_HDRS) \
$(EXTRA_FLAGS) -c $< -o - | $(LLC) -march=bpf -filetype obj -o $@
I wanted to include that sample for whatever soul in the future wants to tread the
same path with similar systems experience levels.
I still get about 100+ warnings when building that are the same as or similar to:
/usr/src/linux-headers-5.4.0-26-generic/arch/x86/include/asm/atomic.h:194:9: warning: unused variable '__ptr' [-Wunused-variable]
return arch_cmpxchg(&v->counter, old, new);
/usr/src/linux-headers-5.4.0-26-generic/arch/x86/include/asm/msr.h:100:26: warning: variable 'low' is uninitialized when used here [-Wuninitialized]
return EAX_EDX_VAL(val, low, high);
I suspect that these warnings come from my aggressive warning flags during
compilation rather than from actual issues in the kernel.
I have been looking at the commits surrounding the pinning of bpf_link. It looks like it's only
working in kernel 5.7? I did actually go through and attempt to attach links for kprobes,
tracepoints, and raw_tracepoints in kernel 5.4 but, as you suggested, it seems unsupported.
I have yet to try on kernel 5.5-5.7 so I'll take a look this week or next.
As I mentioned before, with basic functionality in place here, I'm interested in working on
some sort of BPF tutorial similar to the XDP tutorial (https://github.com/xdp-project/xdp-tutorial)
with perhaps a more in-depth look at the technology included as well.
I'm still fuzzy on the relationship between bpf(2) and perf(1). Would it be correct to say that for
tracepoints, kprobes, and uprobes BPF leverages perf "under the hood" while for XDP and tc,
this is more like classic BPF in that it's implementation doesn't involve perf?
If that's the case then is the bpf_link object the tool to bridge BPF and perf? I noticed that when
checking for pinned BPF programs with bpftool in kernel 5.4 that unless a kprobe, tracepoint,
or uprobe is listed in "bpftool perf list", the program doesn't seem to be running. Is the use of
perf to load BPF programs potentially a way to make them "headless" instead of pinning the bpf_link objects?
Regardless, I'm excited to have a more reliable build system than I have in the past. I think I'll start looking more into CO-RE and libbpf on kernels 5.5-5.7.
Hope everyone is staying healthy out there,
On Thu, May 14, 2020 at 5:51 PM Andrii Nakryiko <andrii.nakryiko@...> wrote:
On Mon, May 11, 2020 at 10:06 AM <mayfieldtristan@...> wrote: