Hello everyone,

A couple of questions. I’m using XDP with BCC on a Fedora 25 with kernel 4.13.0-0.rc5.git0.2.fc27.x86_64 with e1000 driver. Basic XDP examples appear to work, I was able to create my own simple example counting TCP SYN packets, so the setup is at least partially correct.

I’m playing around now with modifying TCP payload on the fly and failing miserably. The function that is supposed to swap two u16s in TCP payload is below

static inline int swapu16(uint16_t *start, void *data_end, int off1, int off2) {
int len = (uint16_t*)data_end - start;
uint16_t poff1, poff2;

if (((void*)&start[off1] > data_end) || ((void*)&start[off2] > data_end))
return 0;

poff1 = start[off1];
poff2 = start[off2];

//start[off2] = poff1;
//start[off1] = poff2;

return 1;
}

and it gets called with start pointing to the beginning of the payload (I’m reasonably sure that is correct).

If I *uncomment* either of the two lines in the function I get verifier errors suggesting I use bpf_probe_read. First question

1. Is what I’m attempting even possible - am I allowed to modify the packet in XDP hook? If no this may be a short conversation.

2. If it is possible, is there a canonical way of doing it compared to what I’m trying to do?

3. I actually attempted to use bpf_probe_read() (even though with the code structured as above, it appears they are inserted automatically, because the BPF program is installed properly with those two lines commented out), like in the code shown below:

static inline int swapu16(uint16_t *start, void *data_end, int off1, int off2) {
int len = (uint16_t*)data_end - start;
uint16_t poff1, poff2;

if (((void*)&start[off1] > data_end) || ((void*)&start[off2] > data_end))
return 0;

bpf_probe_read(&poff1, sizeof(uint16_t), (void*)&start[off1]);
//poff1 = start[off1];
//poff2 = start[off2];

//start[off2] = poff1;
//start[off1] = poff2;

return 1;
}

I get a verifier error
….
60: (2d) if r1 > r2 goto pc+5
R1=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2 R2=pkt_end R3=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2 R4=imm0,min_value=0,max_value=0,min_align=2147483648 R5=pkt(id=0,off=42,r=42) R6=pkt(id=1,off=0,r=20),aux_off_align=2 R7=imm0,min_value=0,max_value=0,min_align=2147483648 R10=fp
61: (bf) r1 = r10
62: (07) r1 += -32
63: (b7) r2 = 2
64: (85) call bpf_probe_read#4
unknown func bpf_probe_read#4

which is really strange. In fact it appears I’m unable to invoke any of the helper bpf functions explicitly.

Many thanks for your suggestions.

-ilya

Ilya Baldin

On Aug 24, 2017, at 12:43 PM, Brenden Blanco <bblanco@...> wrote:

On Thu, Aug 24, 2017 at 9:26 AM, Daniel Borkmann via iovisor-dev
<iovisor-dev@...> wrote:

On 08/24/2017 06:17 PM, Y Song wrote:

CC to bcc mailing list as well.

bpf_probe_read is not allowed in XDP programs.

Your comparison may need to comparison not just starting offset, but
also including the memory size so the
whole write won't fall beyond the "data_end".

Regarding to bcc translates "start[off2]" to bpf_probe_read, it is
possible that bcc rewriter tries to infer bpf_probe candidate and
finds this one ...

I was wondering about this last one, and where this suggestion
comes from exactly (bcc for sure?). How does the error message
look like?

The error comes from here:
https://github.com/iovisor/bcc/blob/master/src/cc/libbpf.c#L210

But as Yonghong said, you cannot use bpf_probe_read from XDP and hence
should ignore the warning.

I would suggest rewriting your check to be something like:

 if (((void*)&start[off1 + 1] > data_end) || ((void*)&start[off2 + 1]

data_end))

   return 0;

On Thu, Aug 24, 2017 at 8:22 AM, Ilya Baldin <ibaldin@...> wrote:

Hello everyone,

A couple of questions. I’m using XDP with BCC on a Fedora 25 with kernel
4.13.0-0.rc5.git0.2.fc27.x86_64 with e1000 driver. Basic XDP examples appear
to work, I was able to create my own simple example counting TCP SYN
packets, so the setup is at least partially correct.

I’m playing around now with modifying TCP payload on the fly and failing
miserably. The function that is supposed to swap two u16s in TCP payload is
below

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
  int len = (uint16_t*)data_end - start;
  uint16_t poff1, poff2;

  if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
    return 0;

  poff1 = start[off1];
  poff2 = start[off2];

  //start[off2] = poff1;
  //start[off1] = poff2;

  return 1;
}

and it gets called with start pointing to the beginning of the payload
(I’m reasonably sure that is correct).

If I *uncomment* either of the two lines in the function I get verifier
errors suggesting I use bpf_probe_read. First question

1. Is what I’m attempting even possible - am I allowed to modify the
packet in XDP hook? If no this may be a short conversation.

2. If it is possible, is there a canonical way of doing it compared to
what I’m trying to do?

3. I actually attempted to use bpf_probe_read() (even though with the
code structured as above, it appears they are inserted automatically,
because the BPF program is installed properly with those two lines commented
out), like in the code shown below:

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
  int len = (uint16_t*)data_end - start;
  uint16_t poff1, poff2;

  if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
    return 0;

  bpf_probe_read(&poff1, sizeof(uint16_t), (void*)&start[off1]);
  //poff1 = start[off1];
  //poff2 = start[off2];

  //start[off2] = poff1;
  //start[off1] = poff2;

  return 1;
}

I get a verifier error
….
60: (2d) if r1 > r2 goto pc+5
 R1=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
R2=pkt_end R3=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
R4=imm0,min_value=0,max_value=0,min_align=2147483648
R5=pkt(id=0,off=42,r=42) R6=pkt(id=1,off=0,r=20),aux_off_align=2
R7=imm0,min_value=0,max_value=0,min_align=2147483648 R10=fp
61: (bf) r1 = r10
62: (07) r1 += -32
63: (b7) r2 = 2
64: (85) call bpf_probe_read#4
unknown func bpf_probe_read#4

which is really strange. In fact it appears I’m unable to invoke any of
the helper bpf functions explicitly.

Many thanks for your suggestions.

-ilya

Ilya Baldin

_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

On Aug 24, 2017, at 3:25 PM, Y Song <ys114321@...> wrote:

On Thu, Aug 24, 2017 at 12:17 PM, Ilya Baldin <ibaldin@...> wrote:

Thank you, everyone for the replies. This explains about bpf_probe_read.

I tried the approach below (changed the check to offset +1) however that
didn’t help, I still get the same error and BCC refuses to load the program

R0=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
R1=inv,min_value=6,max_value=65541,min_align=1,aux_off_align=2 R2=pkt_end
R3=pkt(id=1,off=0,r=20),aux_off_align=2
R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2 R10=fp
65: (69) r1 = *(u16 *)(r6 +0)
R6 invalid mem access 'inv'

Note that:
R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
"inv" means unknown value.

Could you double check your program?

HINT: The invalid mem access 'inv' error can happen if you try to
dereference memory without first using bpf_probe_read() to copy it to the
BPF stack. Sometimes the bpf_probe_read is automatic by the bcc rewriter,
other times you'll need to be explicit.

and it is the ‘write’ statements that are causing it as in when I try to
write into the packet

start[off1] = val;

is when I get the error.

Is the solution to use the tc loader instead of bcc? Less convenient, but
doable, I suppose.

-ilya

Ilya Baldin


On Aug 24, 2017, at 12:43 PM, Brenden Blanco <bblanco@...> wrote:

On Thu, Aug 24, 2017 at 9:26 AM, Daniel Borkmann via iovisor-dev
<iovisor-dev@...> wrote:

On 08/24/2017 06:17 PM, Y Song wrote:


CC to bcc mailing list as well.

bpf_probe_read is not allowed in XDP programs.

Your comparison may need to comparison not just starting offset, but
also including the memory size so the
whole write won't fall beyond the "data_end".

Regarding to bcc translates "start[off2]" to bpf_probe_read, it is
possible that bcc rewriter tries to infer bpf_probe candidate and
finds this one ...



I was wondering about this last one, and where this suggestion
comes from exactly (bcc for sure?). How does the error message
look like?


The error comes from here:
https://github.com/iovisor/bcc/blob/master/src/cc/libbpf.c#L210

But as Yonghong said, you cannot use bpf_probe_read from XDP and hence
should ignore the warning.

I would suggest rewriting your check to be something like:

if (((void*)&start[off1 + 1] > data_end) || ((void*)&start[off2 + 1]

data_end))

  return 0;



On Thu, Aug 24, 2017 at 8:22 AM, Ilya Baldin <ibaldin@...> wrote:


Hello everyone,

A couple of questions. I’m using XDP with BCC on a Fedora 25 with kernel
4.13.0-0.rc5.git0.2.fc27.x86_64 with e1000 driver. Basic XDP examples appear
to work, I was able to create my own simple example counting TCP SYN
packets, so the setup is at least partially correct.

I’m playing around now with modifying TCP payload on the fly and failing
miserably. The function that is supposed to swap two u16s in TCP payload is
below

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
 int len = (uint16_t*)data_end - start;
 uint16_t poff1, poff2;

 if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
   return 0;

 poff1 = start[off1];
 poff2 = start[off2];

 //start[off2] = poff1;
 //start[off1] = poff2;

 return 1;
}

and it gets called with start pointing to the beginning of the payload
(I’m reasonably sure that is correct).

If I *uncomment* either of the two lines in the function I get verifier
errors suggesting I use bpf_probe_read. First question

1. Is what I’m attempting even possible - am I allowed to modify the
packet in XDP hook? If no this may be a short conversation.

2. If it is possible, is there a canonical way of doing it compared to
what I’m trying to do?

3. I actually attempted to use bpf_probe_read() (even though with the
code structured as above, it appears they are inserted automatically,
because the BPF program is installed properly with those two lines commented
out), like in the code shown below:

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
 int len = (uint16_t*)data_end - start;
 uint16_t poff1, poff2;

 if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
   return 0;

 bpf_probe_read(&poff1, sizeof(uint16_t), (void*)&start[off1]);
 //poff1 = start[off1];
 //poff2 = start[off2];

 //start[off2] = poff1;
 //start[off1] = poff2;

 return 1;
}

I get a verifier error
….
60: (2d) if r1 > r2 goto pc+5
R1=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
R2=pkt_end R3=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
R4=imm0,min_value=0,max_value=0,min_align=2147483648
R5=pkt(id=0,off=42,r=42) R6=pkt(id=1,off=0,r=20),aux_off_align=2
R7=imm0,min_value=0,max_value=0,min_align=2147483648 R10=fp
61: (bf) r1 = r10
62: (07) r1 += -32
63: (b7) r2 = 2
64: (85) call bpf_probe_read#4
unknown func bpf_probe_read#4

which is really strange. In fact it appears I’m unable to invoke any of
the helper bpf functions explicitly.

Many thanks for your suggestions.

-ilya

Ilya Baldin



_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

On Aug 24, 2017, at 3:32 PM, Brenden Blanco <bblanco@...> wrote:

On Thu, Aug 24, 2017 at 12:25 PM, Y Song <ys114321@...> wrote:

On Thu, Aug 24, 2017 at 12:17 PM, Ilya Baldin <ibaldin@...> wrote:

Thank you, everyone for the replies. This explains about bpf_probe_read.

I tried the approach below (changed the check to offset +1) however that
didn’t help, I still get the same error and BCC refuses to load the program

R0=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
R1=inv,min_value=6,max_value=65541,min_align=1,aux_off_align=2 R2=pkt_end
R3=pkt(id=1,off=0,r=20),aux_off_align=2
R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2 R10=fp
65: (69) r1 = *(u16 *)(r6 +0)
R6 invalid mem access 'inv'

Here is a full BCC version that worked for me:

#!/usr/bin/env python
import bcc
text = """
static inline int swapu16(uint16_t *start, void *data_end, int off1, int off2) {
 int len = (uint16_t*)data_end - start;
 uint16_t poff1, poff2;

 if (((void*)&start[off1 + 1] > data_end) || ((void*)&start[off2 + 1]

data_end))

   return 0;

 poff1 = start[off1];
 poff2 = start[off2];

 start[off2] = poff1;
 start[off1] = poff2;

 return 1;
}
int on_xdp_rx(struct xdp_md *xdp) {
 void *data_end = (void *)(long)xdp->data_end;
 void *data = (void *)(long)xdp->data;

 if (!swapu16(data, data_end, 4, 6))
   return XDP_DROP;
 return XDP_PASS;
}
"""
b = bcc.BPF(text=text)
b.attach_xdp("eno1", b.load_func("on_xdp_rx", b.XDP), 0)

Note that:
R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
"inv" means unknown value.

Could you double check your program?

HINT: The invalid mem access 'inv' error can happen if you try to
dereference memory without first using bpf_probe_read() to copy it to the
BPF stack. Sometimes the bpf_probe_read is automatic by the bcc rewriter,
other times you'll need to be explicit.

and it is the ‘write’ statements that are causing it as in when I try to
write into the packet

start[off1] = val;

is when I get the error.

Is the solution to use the tc loader instead of bcc? Less convenient, but
doable, I suppose.

-ilya

Ilya Baldin


On Aug 24, 2017, at 12:43 PM, Brenden Blanco <bblanco@...> wrote:

On Thu, Aug 24, 2017 at 9:26 AM, Daniel Borkmann via iovisor-dev
<iovisor-dev@...> wrote:

On 08/24/2017 06:17 PM, Y Song wrote:


CC to bcc mailing list as well.

bpf_probe_read is not allowed in XDP programs.

Your comparison may need to comparison not just starting offset, but
also including the memory size so the
whole write won't fall beyond the "data_end".

Regarding to bcc translates "start[off2]" to bpf_probe_read, it is
possible that bcc rewriter tries to infer bpf_probe candidate and
finds this one ...



I was wondering about this last one, and where this suggestion
comes from exactly (bcc for sure?). How does the error message
look like?


The error comes from here:
https://github.com/iovisor/bcc/blob/master/src/cc/libbpf.c#L210

But as Yonghong said, you cannot use bpf_probe_read from XDP and hence
should ignore the warning.

I would suggest rewriting your check to be something like:

if (((void*)&start[off1 + 1] > data_end) || ((void*)&start[off2 + 1]

data_end))

  return 0;



On Thu, Aug 24, 2017 at 8:22 AM, Ilya Baldin <ibaldin@...> wrote:


Hello everyone,

A couple of questions. I’m using XDP with BCC on a Fedora 25 with kernel
4.13.0-0.rc5.git0.2.fc27.x86_64 with e1000 driver. Basic XDP examples appear
to work, I was able to create my own simple example counting TCP SYN
packets, so the setup is at least partially correct.

I’m playing around now with modifying TCP payload on the fly and failing
miserably. The function that is supposed to swap two u16s in TCP payload is
below

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
 int len = (uint16_t*)data_end - start;
 uint16_t poff1, poff2;

 if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
   return 0;

 poff1 = start[off1];
 poff2 = start[off2];

 //start[off2] = poff1;
 //start[off1] = poff2;

 return 1;
}

and it gets called with start pointing to the beginning of the payload
(I’m reasonably sure that is correct).

If I *uncomment* either of the two lines in the function I get verifier
errors suggesting I use bpf_probe_read. First question

1. Is what I’m attempting even possible - am I allowed to modify the
packet in XDP hook? If no this may be a short conversation.

2. If it is possible, is there a canonical way of doing it compared to
what I’m trying to do?

3. I actually attempted to use bpf_probe_read() (even though with the
code structured as above, it appears they are inserted automatically,
because the BPF program is installed properly with those two lines commented
out), like in the code shown below:

static inline int swapu16(uint16_t *start, void *data_end, int off1, int
off2) {
 int len = (uint16_t*)data_end - start;
 uint16_t poff1, poff2;

 if (((void*)&start[off1] > data_end) || ((void*)&start[off2] >
data_end))
   return 0;

 bpf_probe_read(&poff1, sizeof(uint16_t), (void*)&start[off1]);
 //poff1 = start[off1];
 //poff2 = start[off2];

 //start[off2] = poff1;
 //start[off1] = poff2;

 return 1;
}

I get a verifier error
….
60: (2d) if r1 > r2 goto pc+5
R1=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
R2=pkt_end R3=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
R4=imm0,min_value=0,max_value=0,min_align=2147483648
R5=pkt(id=0,off=42,r=42) R6=pkt(id=1,off=0,r=20),aux_off_align=2
R7=imm0,min_value=0,max_value=0,min_align=2147483648 R10=fp
61: (bf) r1 = r10
62: (07) r1 += -32
63: (b7) r2 = 2
64: (85) call bpf_probe_read#4
unknown func bpf_probe_read#4

which is really strange. In fact it appears I’m unable to invoke any of
the helper bpf functions explicitly.

Many thanks for your suggestions.

-ilya

Ilya Baldin



_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

If I just copy your swapu16 function into my code (and it looks very similar to mine) the result continues to be the same (error).

Here is a gist with python and c-components of my program

https://gist.github.com/ibaldin/716d70d490b44e05d15db59ee983c0c0

Looking at the BPF byte code print out from BCC (below) something strange is indeed going on that I think Yonghong alluded to.

The last instruction is r2 = *(u16*)(r1 + 8), and r1 is invalid. R1 is initially set, but then for some reason becomes invalid:

50: (2d) if r4 > r2 goto pc+35
R1=pkt(id=2,off=0,r=2),aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
51: (57) r1 &= 65535
52: (15) if r1 == 0x0 goto pc+33
R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=14


—— FULL TEXT OF BPF PROGRAM BYTECODE ———


0: (61) r2 = *(u32 *)(r1 +4)
1: (61) r1 = *(u32 *)(r1 +0)
2: (bf) r3 = r1
3: (07) r3 += 14
4: (2d) if r3 > r2 goto pc+81
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=pkt(id=0,off=14,r=14) R10=fp
5: (71) r3 = *(u8 *)(r1 +12)
6: (71) r4 = *(u8 *)(r1 +13)
7: (67) r4 <<= 8
8: (4f) r4 |= r3
9: (15) if r4 == 0xa888 goto pc+2
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=inv56 R4=inv R10=fp
10: (b7) r3 = 14
11: (55) if r4 != 0x81 goto pc+4
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=imm14,min_value=14,max_value=14,min_align=2 R4=inv,min_value=129,max_value=129 R10=fp
12: (b7) r3 = 18
13: (bf) r5 = r1
14: (07) r5 += 18
15: (2d) if r5 > r2 goto pc+70
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
16: (15) if r4 == 0xa888 goto pc+1
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
17: (55) if r4 != 0x81 goto pc+4
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
18: (07) r3 += 4
19: (bf) r5 = r1
20: (0f) r5 += r3
21: (2d) if r5 > r2 goto pc+64
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=22,r=22) R10=fp
22: (55) if r4 != 0x8 goto pc+63
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv,min_value=8,max_value=8 R5=pkt(id=0,off=22,r=22) R10=fp
23: (bf) r4 = r1
24: (0f) r4 += r3
25: (15) if r4 == 0x0 goto pc+60
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=pkt(id=0,off=22,r=22) R5=pkt(id=0,off=22,r=22) R10=fp
26: (bf) r5 = r4
27: (07) r5 += 20
28: (2d) if r5 > r2 goto pc+57
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=pkt(id=0,off=22,r=42) R5=pkt(id=0,off=42,r=42) R10=fp
29: (71) r4 = *(u8 *)(r4 +9)
30: (55) if r4 != 0x6 goto pc+55
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv56,min_value=6,max_value=6 R5=pkt(id=0,off=42,r=42) R10=fp
31: (bf) r4 = r1
32: (0f) r4 += r3
33: (71) r4 = *(u8 *)(r4 +0)
34: (57) r4 &= 15
35: (67) r4 <<= 2
36: (0f) r4 += r3
37: (bf) r3 = r1
38: (0f) r3 += r4
39: (15) if r3 == 0x0 goto pc+46
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=pkt(id=1,off=0,r=0),aux_off_align=2 R4=inv57,min_value=22,max_value=82,min_align=2 R5=pkt(id=0,off=42,r=42) R10=fp
40: (bf) r5 = r3
41: (07) r5 += 20
42: (2d) if r5 > r2 goto pc+43
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=inv57,min_value=22,max_value=82,min_align=2 R5=pkt(id=1,off=20,r=20),aux_off_align=2 R10=fp
43: (69) r5 = *(u16 *)(r3 +12)
44: (77) r5 >>= 2
45: (57) r5 &= 60
46: (0f) r5 += r4
47: (0f) r1 += r5
48: (bf) r4 = r1
49: (07) r4 += 2
50: (2d) if r4 > r2 goto pc+35
R1=pkt(id=2,off=0,r=2),aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
51: (57) r1 &= 65535
52: (15) if r1 == 0x0 goto pc+33
R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
53: (b7) r5 = 0
54: (bf) r0 = r1
55: (07) r0 += 6
56: (b7) r4 = 0
57: (2d) if r0 > r2 goto pc+9
R0=inv,min_value=6,max_value=65541,min_align=1,aux_off_align=2 R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R10=fp
58: (bf) r0 = r1
59: (07) r0 += 10
60: (b7) r4 = 0
61: (2d) if r0 > r2 goto pc+5
R0=inv,min_value=10,max_value=65545,min_align=1,aux_off_align=2 R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R10=fp
62: (69) r2 = *(u16 *)(r1 +8)
R1 invalid mem access 'inv'

-ilya

Ilya Baldin

On Aug 24, 2017, at 5:18 PM, Y Song <ys114321@...> wrote:

This is the problem:
static inline uint16_t parse_tcp_pld(void *data, u64 nh_off, void *data_end) {
 uint16_t *pld = data + nh_off;
 if ((void*)&pld[1] > data_end)
   return 0;
 return pld;
}

return value should be "void *" or "uint16_t *".

In the future, we will try to add more diagnostic information to debug
such issues.


On Thu, Aug 24, 2017 at 12:52 PM, Ilya Baldin <ibaldin@...> wrote:

If I just copy your swapu16 function into my code (and it looks very similar to mine) the result continues to be the same (error).

Here is a gist with python and c-components of my program

https://gist.github.com/ibaldin/716d70d490b44e05d15db59ee983c0c0

Looking at the BPF byte code print out from BCC (below) something strange is indeed going on that I think Yonghong alluded to.

The last instruction is r2 = *(u16*)(r1 + 8), and r1 is invalid. R1 is initially set, but then for some reason becomes invalid:

50: (2d) if r4 > r2 goto pc+35
R1=pkt(id=2,off=0,r=2),aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
51: (57) r1 &= 65535
52: (15) if r1 == 0x0 goto pc+33
R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=14


—— FULL TEXT OF BPF PROGRAM BYTECODE ———


0: (61) r2 = *(u32 *)(r1 +4)
1: (61) r1 = *(u32 *)(r1 +0)
2: (bf) r3 = r1
3: (07) r3 += 14
4: (2d) if r3 > r2 goto pc+81
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=pkt(id=0,off=14,r=14) R10=fp
5: (71) r3 = *(u8 *)(r1 +12)
6: (71) r4 = *(u8 *)(r1 +13)
7: (67) r4 <<= 8
8: (4f) r4 |= r3
9: (15) if r4 == 0xa888 goto pc+2
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=inv56 R4=inv R10=fp
10: (b7) r3 = 14
11: (55) if r4 != 0x81 goto pc+4
R1=pkt(id=0,off=0,r=14) R2=pkt_end R3=imm14,min_value=14,max_value=14,min_align=2 R4=inv,min_value=129,max_value=129 R10=fp
12: (b7) r3 = 18
13: (bf) r5 = r1
14: (07) r5 += 18
15: (2d) if r5 > r2 goto pc+70
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
16: (15) if r4 == 0xa888 goto pc+1
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
17: (55) if r4 != 0x81 goto pc+4
R1=pkt(id=0,off=0,r=18) R2=pkt_end R3=imm18,min_value=18,max_value=18,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=18,r=18) R10=fp
18: (07) r3 += 4
19: (bf) r5 = r1
20: (0f) r5 += r3
21: (2d) if r5 > r2 goto pc+64
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv,min_value=129,max_value=129 R5=pkt(id=0,off=22,r=22) R10=fp
22: (55) if r4 != 0x8 goto pc+63
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv,min_value=8,max_value=8 R5=pkt(id=0,off=22,r=22) R10=fp
23: (bf) r4 = r1
24: (0f) r4 += r3
25: (15) if r4 == 0x0 goto pc+60
R1=pkt(id=0,off=0,r=22) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=pkt(id=0,off=22,r=22) R5=pkt(id=0,off=22,r=22) R10=fp
26: (bf) r5 = r4
27: (07) r5 += 20
28: (2d) if r5 > r2 goto pc+57
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=pkt(id=0,off=22,r=42) R5=pkt(id=0,off=42,r=42) R10=fp
29: (71) r4 = *(u8 *)(r4 +9)
30: (55) if r4 != 0x6 goto pc+55
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=imm22,min_value=22,max_value=22,min_align=2 R4=inv56,min_value=6,max_value=6 R5=pkt(id=0,off=42,r=42) R10=fp
31: (bf) r4 = r1
32: (0f) r4 += r3
33: (71) r4 = *(u8 *)(r4 +0)
34: (57) r4 &= 15
35: (67) r4 <<= 2
36: (0f) r4 += r3
37: (bf) r3 = r1
38: (0f) r3 += r4
39: (15) if r3 == 0x0 goto pc+46
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=pkt(id=1,off=0,r=0),aux_off_align=2 R4=inv57,min_value=22,max_value=82,min_align=2 R5=pkt(id=0,off=42,r=42) R10=fp
40: (bf) r5 = r3
41: (07) r5 += 20
42: (2d) if r5 > r2 goto pc+43
R1=pkt(id=0,off=0,r=42) R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=inv57,min_value=22,max_value=82,min_align=2 R5=pkt(id=1,off=20,r=20),aux_off_align=2 R10=fp
43: (69) r5 = *(u16 *)(r3 +12)
44: (77) r5 >>= 2
45: (57) r5 &= 60
46: (0f) r5 += r4
47: (0f) r1 += r5
48: (bf) r4 = r1
49: (07) r4 += 2
50: (2d) if r4 > r2 goto pc+35
R1=pkt(id=2,off=0,r=2),aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
51: (57) r1 &= 65535
52: (15) if r1 == 0x0 goto pc+33
R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=pkt(id=2,off=2,r=2),aux_off_align=2 R5=inv56,min_value=22,max_value=142,min_align=2,aux_off_align=2 R10=fp
53: (b7) r5 = 0
54: (bf) r0 = r1
55: (07) r0 += 6
56: (b7) r4 = 0
57: (2d) if r0 > r2 goto pc+9
R0=inv,min_value=6,max_value=65541,min_align=1,aux_off_align=2 R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R10=fp
58: (bf) r0 = r1
59: (07) r0 += 10
60: (b7) r4 = 0
61: (2d) if r0 > r2 goto pc+5
R0=inv,min_value=10,max_value=65545,min_align=1,aux_off_align=2 R1=inv,min_value=0,max_value=65535,min_align=1,aux_off_align=2 R2=pkt_end R3=pkt(id=1,off=0,r=20),aux_off_align=2 R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2 R10=fp
62: (69) r2 = *(u16 *)(r1 +8)
R1 invalid mem access 'inv'

-ilya

Ilya Baldin