On 27 Mar 2019, at 16:11, Jiong Wang via Lists.Iovisor.Org <jiong.wang=netronome.com@...> wrote:

On 27 Mar 2019, at 14:53, Simon <contact@...> wrote:

Hi Jiong,
I didn't succeed to generate .i file using bcc, but since severals days I try to rewrite my code without bcc. (directly with bpf C api / clang / iproute2)

I didn't finished yet, but I have a reduced version compared to the one I written for bcc and I face the same issue, so this time I can get .i file easily.

So the .i file is as attachment.
The corresponding code is available here : https://github.com/sbernard31/udploadbalancer/blob/44fe1ea549a55ab23c7d1b70e9651df6f61fb865/ulb.c

Hi Simon,

Thanks for the .i, I prototyped some byteswap code-gen change, but
seems doesn’t help your issue which could narrow down to the following
general code pattern:

unsigned char cal(unsigned int a, unsigned char *b)
{
if (a < 8 || a > 512)
return 0;

return b[a];
}

LLVM is doing some optimisation, instead of generating two separate comparison,
a < 8 and a > 512, it is combining them because a negative value when casted
into unsigned must be bigger than 504, so above code turned into

unsigned char cal(unsigned int a, unsigned char *b)
{
unsigned tmp = a - 8;
if (tmp > 504)
return 0;

return b[a];
}

The consequence of such optimisation is new variable “tmp” is used for comparison
And verifier now know “tmp”'s value range instead of the original “a” which is used
later adding to a packet pointer. A unknown value range of “a” then caused the
verifier rejection.

So, I suspect any code using above c code pattern will likely be rejected.

1. combinable comparisons
2. the variable involved in the comparison used later in places requiring value range

And in your code, after you insert those printk, they made the following
two comparisons non-combinable any more, so udp_len is used for the
comparison and got correct value range to pass the later pkt pointer
addition check.


if (udp_len < 8) {
return XDP_DROP;
}
if (udp_len > 512) {
return XDP_DROP;
}

Regards,
Jiong

The error is still math between pkt pointer and register with unbounded min value is not allowed

The verifier output is pretty much the same :

27: (71) r4 = *(u8 *)(r1 +23)
28: (b7) r0 = 2
29: (55) if r4 != 0x11 goto pc+15
R0=inv2 R1=pkt(id=0,off=0,r=34,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=34,r=34,imm=0) R4=inv17 R5=inv5 R10=fp0,call_-1
30: (07) r3 += 8
31: (b7) r0 = 1
32: (2d) if r3 > r2 goto pc+12
R0=inv1 R1=pkt(id=0,off=0,r=42,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=42,r=42,imm=0) R4=inv17 R5=inv5 R10=fp0,call_-1
33: (69) r3 = *(u16 *)(r1 +38)
34: (dc) r3 = be16 r3
35: (bf) r4 = r3
36: (07) r4 += -8
37: (57) r4 &= 65535
38: (b7) r0 = 1
39: (25) if r4 > 0x1f8 goto pc+5
R0=inv1 R1=pkt(id=0,off=0,r=42,imm=0) R2=pkt_end(id=0,off=0,imm=0) R3=inv(id=0) R4=inv(id=0,umax_value=504,var_off=(0x0; 0x1ff)) R5=inv5 R10=fp0,call_-1
40: (0f) r1 += r3
math between pkt pointer and register with unbounded min value is not allowed

I'm pretty sure the bcc version I used before linked statically clang/llvm v7.0.
Here I use a v6.0.

The funny part ... This modification about just adding some logs/printk makes this error disappear : https://github.com/sbernard31/udploadbalancer/commit/0145538c7b35e2a6bb92225f69a45f4bee120a6d

All of those erifier errors make me a bit crazy (╥_╥)

HTH

Simon


<ulb.i>