On Wed, May 6, 2020 at 9:26 AM <vigs.prof@...> wrote:

Hello - I am looking to trace ip_forward_finish. The intent is to trace latency of all TCP connections going through a linux based gateway router. Hence thought of tracing ip_forward_finish kernel function. And capture the time-stamp of SYN, SYN-ACK and ACK messages at the router.

The issue is accessing iphdr inside the trace function crashes with the below error:

bpf: Failed to load program: Permission denied
0: (79) r6 = *(u64 *)(r1 +96)
1: (b7) r1 = 0
2: (6b) *(u16 *)(r10 -24) = r1
3: (bf) r3 = r6
4: (07) r3 += 192
5: (bf) r1 = r10
6: (07) r1 += -24
7: (b7) r2 = 2
8: (85) call bpf_probe_read#4
9: (69) r1 = *(u16 *)(r10 -24)
10: (55) if r1 != 0x8 goto pc+7
R0=inv(id=0) R1=inv8 R6=inv(id=0) R10=fp0
11: (69) r1 = *(u16 *)(r6 +196)
R6 invalid mem access 'inv'

You did not show the code which actually caused the problem.

bpf_probe_read(&ip_Hdr, sizeof(ip_Hdr), (void*)ip_hdr(skb));

if ( (ip_Hdr.protocol != IPPROTO_TCP))
return 0;

return 0;
}

There must be code after "if ( (ip_Hdr.protocol != IPPROTO_TCP)) return 0;" .
You may need bpf_probe_read() for memory accesses there.

HINT: The invalid mem access 'inv' error can happen if you try to dereference memory without first using bpf_probe_read() to copy it to the BPF stack. Sometimes the bpf_probe_read is automatic by the bcc rewriter, other times you'll need to be explicit.

The code fragment I originally had was as below and the crash occurs when an access to ip_Hdr->protocol is made. And I also checked that ip_Hdr is not null.

int trace_forward_finish(struct pt_regs *ctx,struct net *net, struct sock *sk, struct sk_buff *skb)
{

if (skb->protocol != htons(ETH_P_IP)) return 0;

struct iphdr* ip_Hdr = (struct iphdr *) skb_network_header(skb);

if (ip_Hdr->protocol != IPPROTO_TCP)
return 0;


/// Other code

}

Per the HINT in the message, I did try to change to bpf_probe_read but still the same outcome

int trace_forward_finish(struct pt_regs *ctx,struct net *net, struct sock *sk, struct sk_buff *skb)
{
if (skb->protocol != htons(ETH_P_IP)) return 0;

struct iphdr ip_Hdr;
bpf_probe_read(&ip_Hdr, sizeof(ip_Hdr), (void*)ip_hdr(skb));

if ( (ip_Hdr.protocol != IPPROTO_TCP))
return 0;

return 0;
}

Any help would be appreciated.