When using kprobe in bcc, I can get param directly like `int kprobe__tcp_set_state(struct pt_regs *ctx, struct sock *sk, int state)`
But it seems not to work in kretprobe, I've found that I can get first param by using `struct sock *sk = (void*)ctx->bx`
but I can't get the second param through `ctx->cx`.
Am I get the wrong register? I'm in x86-64

On Fri, Aug 7, 2020 at 12:45 AM <forrest0579@...> wrote:

When using kprobe in bcc, I can get param directly like `int kprobe__tcp_set_state(struct pt_regs *ctx, struct sock *sk, int state)`
But it seems not to work in kretprobe, I've found that I can get first param by using `struct sock *sk = (void*)ctx->bx`
but I can't get the second param through `ctx->cx`.
Am I get the wrong register? I'm in x86-64

You can't do it reliably with kretprobe. kretprobe is executed right
before the function is exiting, by that time all the registers that
contained input parameters could have been used for something else. So
you got lucky with struct sock * here, but as a general rule you
shouldn't rely on this. You either have to pair kprobe with kretprobe
and store input arguments, or take a look at fexit program type, it is
just like kretprobe, but faster and guarantees input arguments are
preserved.

On Fri, Aug 7, 2020 at 11:31 AM, Andrii Nakryiko wrote:

You can't do it reliably with kretprobe. kretprobe is executed right
before the function is exiting, by that time all the registers that
contained input parameters could have been used for something else. So
you got lucky with struct sock * here, but as a general rule you
shouldn't rely on this. You either have to pair kprobe with kretprobe
and store input arguments, or take a look at fexit program type, it is
just like kretprobe, but faster and guarantees input arguments are
preserved.

Thanks for reply.
It seems fexit it a new feature and I'm using linux v4.15, so fexit can't help here.
kretprobe with kprobe is an option and I've found a lot examples in bbc, but I am also wondering if it is always right to use pid_tgid as key to store params and get it from kretprobe.
I am wondering if there is a chance that following case would happen:

0. attach kprobe program in tcp_set_state, store params in HASHMAP using pid_tgid as key; attach kretprobe in tcp_set_state, lookup params using pid_tgid
1. kprobe program triggered twice with same pid_tgid before kretprobe executed, so can only get the last params

I have this concern because I'm using golang and the two goroutines may map to one thread in kernel. If one goroutine gets interrupted when executing tcp_set_state, another one would have a chance to execute tcp_set_state with the same pid_tgid.

On Sun, Aug 9, 2020 at 8:10 PM <forrest0579@...> wrote:

On Fri, Aug 7, 2020 at 11:31 AM, Andrii Nakryiko wrote:

You can't do it reliably with kretprobe. kretprobe is executed right
before the function is exiting, by that time all the registers that
contained input parameters could have been used for something else. So
you got lucky with struct sock * here, but as a general rule you
shouldn't rely on this. You either have to pair kprobe with kretprobe
and store input arguments, or take a look at fexit program type, it is
just like kretprobe, but faster and guarantees input arguments are
preserved.

Thanks for reply.
It seems fexit it a new feature and I'm using linux v4.15, so fexit can't help here.
kretprobe with kprobe is an option and I've found a lot examples in bbc, but I am also wondering if it is always right to use pid_tgid as key to store params and get it from kretprobe.
I am wondering if there is a chance that following case would happen:

0. attach kprobe program in tcp_set_state, store params in HASHMAP using pid_tgid as key; attach kretprobe in tcp_set_state, lookup params using pid_tgid
1. kprobe program triggered twice with same pid_tgid before kretprobe executed, so can only get the last params

I have this concern because I'm using golang and the two goroutines may map to one thread in kernel. If one goroutine gets interrupted when executing tcp_set_state, another one would have a chance to execute tcp_set_state with the same pid_tgid.

I don't think golang can interrupt thread while it's being executed in
the kernel. So from the golang perspective I wouldn't worry, the
kernel will execute both kprobe and corresponding kretprobe before
golang runtime can do anything about that. But in general, it's
possible to attach kprobe to a kernel function that could be called
multiple times for a given thread, at which point pid_tgid won't be
enough. This cannot happen for syscalls and many other kernel
functions, though. I would imagine that's not the case for
tcp_set_state either. But please double check kernel sources to be
absolutely sure.