|
O Mahony, Billy <billy.o.mahony@...>
Hi All,
I'm doing my version of hello world for eBPF by forwarding eth frames between two nics - eth1 and eth3 on my machine. Using bpf_clone_redirect(). (cc/export/helpers.h says bpf_redirect is not available until a later kernel.)
When I run the python program it generates:
[17:42 GD-WCP my_bpf]$ sudo python p2p.py
bpf: Invalid argument
0: (b7) r2 = 5
1: (b7) r3 = 0
2: (85) call 13
unknown func 13
Traceback (most recent call last):
File "p2p.py", line 86, in <module>
function_eth1_ic = bpf.load_func("eth1_ic", BPF.SOCKET_FILTER)
File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 165, in load_func
raise Exception("Failed to load BPF program %s" % func_name)
Exception: Failed to load BPF program eth1_ic
The bpf program is:
int eth1_ic(struct __sk_buff *skb) {
bpf_clone_redirect(skb, 5, 0);
return -1;
}
I've hard coded the egress ifindex based on output from '$ipi a'
Bpf_clone_redirect is indeed the 13th entry in the bpf_func_id enum but why is it reported as 'unknown' ?
I'm running Ubuntu 14.04 with the kernel upgraded using binary packages to 4.3.0-040300-generic from Mon Nov 2 2015.
Thanks,
Billy.
|
|
|
Brenden Blanco <bblanco@...>
|
|
|
O Mahony, Billy <billy.o.mahony@...>
Hi Brendan,
Thanks, that’s great. I’ll have a look at that example you mention too.
Whereabouts in the code should I look to see which functions are allowable in which bpf program types?
I haven’t looked again recently but iirc in some header or man page there is a TODO for description of the various program types.
If you point me to the relevant area of the code I can do a patch to fill out some of that documentation. I won’t be able to figure out chapter and verse on it
but some basic info would be better than none!
Cheers,
/Billy.
toggle quoted message
Show quoted text
From: Brenden Blanco [mailto:bblanco@...]
Sent: Tuesday, March 1, 2016 6:07 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Tue, Mar 1, 2016 at 9:52 AM, O Mahony, Billy via iovisor-dev <iovisor-dev@...> wrote:
Hi All,
I'm doing my version of hello world for eBPF by forwarding eth frames between two nics - eth1 and eth3 on my machine. Using bpf_clone_redirect(). (cc/export/helpers.h says bpf_redirect is not available until a later kernel.)
When I run the python program it generates:
[17:42 GD-WCP my_bpf]$ sudo python p2p.py
bpf: Invalid argument
0: (b7) r2 = 5
1: (b7) r3 = 0
2: (85) call 13
unknown func 13
Traceback (most recent call last):
File "p2p.py", line 86, in <module>
function_eth1_ic = bpf.load_func("eth1_ic", BPF.SOCKET_FILTER)
File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 165, in load_func
raise Exception("Failed to load BPF program %s" % func_name)
Exception: Failed to load BPF program eth1_ic
You need to use BPF.SCHED_ACT or SCHED_CLS to use these forwarding functions. They are disallowed by the verifier for SOCKET_FILTER programs. The simplest usage (focus on the pyroute2 bits) is probably in tests/cc/test_xlate1.py.
The bpf program is:
int eth1_ic(struct __sk_buff *skb) {
bpf_clone_redirect(skb, 5, 0);
return -1;
}
I've hard coded the egress ifindex based on output from '$ipi a'
Bpf_clone_redirect is indeed the 13th entry in the bpf_func_id enum but why is it reported as 'unknown' ?
I'm running Ubuntu 14.04 with the kernel upgraded using binary packages to 4.3.0-040300-generic from Mon Nov 2 2015.
Thanks,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
Brenden Blanco <bblanco@...>
The best source of truth is in the kernel code.
The whitelist of functions for networking is:
sk_filter_func_proto() tc_cls_act_func_proto()
other whitelists are available around various pieces where you see BPF_FUNC_*
On Wed, Mar 2, 2016 at 1:48 AM, O Mahony, Billy <billy.o.mahony@...> wrote:
Hi Brendan,
Thanks, that’s great. I’ll have a look at that example you mention too.
Whereabouts in the code should I look to see which functions are allowable in which bpf program types?
I haven’t looked again recently but iirc in some header or man page there is a TODO for description of the various program types.
If you point me to the relevant area of the code I can do a patch to fill out some of that documentation. I won’t be able to figure out chapter and verse on it
but some basic info would be better than none!
Cheers,
/Billy.
On Tue, Mar 1, 2016 at 9:52 AM, O Mahony, Billy via iovisor-dev <iovisor-dev@...> wrote:
Hi All,
I'm doing my version of hello world for eBPF by forwarding eth frames between two nics - eth1 and eth3 on my machine. Using bpf_clone_redirect(). (cc/export/helpers.h says bpf_redirect is not available until a later kernel.)
When I run the python program it generates:
[17:42 GD-WCP my_bpf]$ sudo python p2p.py
bpf: Invalid argument
0: (b7) r2 = 5
1: (b7) r3 = 0
2: (85) call 13
unknown func 13
Traceback (most recent call last):
File "p2p.py", line 86, in <module>
function_eth1_ic = bpf.load_func("eth1_ic", BPF.SOCKET_FILTER)
File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 165, in load_func
raise Exception("Failed to load BPF program %s" % func_name)
Exception: Failed to load BPF program eth1_ic
You need to use BPF.SCHED_ACT or SCHED_CLS to use these forwarding functions. They are disallowed by the verifier for SOCKET_FILTER programs. The simplest usage (focus on the pyroute2 bits) is probably in tests/cc/test_xlate1.py.
The bpf program is:
int eth1_ic(struct __sk_buff *skb) {
bpf_clone_redirect(skb, 5, 0);
return -1;
}
I've hard coded the egress ifindex based on output from '$ipi a'
Bpf_clone_redirect is indeed the 13th entry in the bpf_func_id enum but why is it reported as 'unknown' ?
I'm running Ubuntu 14.04 with the kernel upgraded using binary packages to 4.3.0-040300-generic from Mon Nov 2 2015.
Thanks,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
O Mahony, Billy <billy.o.mahony@...>
Hi Brendan,
After looking at the xlate tests, I *think* have managed to get my bpf programs attached to my NICS (or rather attached to a qdisc attached to my NICS - as I
now understand it).
However I still can't forward traffic ingressing one nic to egress the other.
My python program looks like this.
bpf = BPF(src_file = "p2p.c",debug = 0)
function_eth1_ic = bpf.load_func("eth1_ic", BPF.SCHED_ACT)
function_eth3_ic = bpf.load_func("eth3_ic", BPF.SCHED_ACT)
ip = IPRoute()
ifindex_eth1 = ip.link_lookup(ifname="eth1")[0]
ifindex_eth3 = ip.link_lookup(ifname="eth3")[0]
#Add an ingress qdisc to both NICs
ip.tc("add", "ingress", ifindex_eth1, "ffff:")
ip.tc("add", "ingress", ifindex_eth3, "ffff:")
#create actions based on the BPF functions
action_eth1 = {"kind": "bpf", "fd": function_eth1_ic.fd,
"name": function_eth1_ic.name, "action": "ok"}
action_eth3 = {"kind": "bpf", "fd": function_eth3_ic.fd,
"name": function_eth3_ic.name, "action": "ok"}
#add a filter to accept all eth frame. Attach the bpf action? functions to the filter?
#I have no idea what the classid, target and keys parameters mean!
ip.tc("add-filter", "u32", ifindex_eth1, ":1", parent="ffff:",
action=[action_eth1],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
ip.tc("add-filter", "u32", ifindex_eth3, ":1", parent="ffff:",
action=[action_eth3],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
while (True): pass
A couple of things; I can’t see any mention of the filter or action on the output from some tc show cmds:
[13:32 GD-WCP bpf]$ tc qdisc show dev eth3
qdisc mq 0: root
qdisc pfifo_fast 0: parent :1 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
...
qdisc pfifo_fast 0: parent :38 bands 3 priomap 1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc ingress ffff: parent ffff:fff1 ----------------
[13:32 GD-WCP bpf]$ tc class show dev eth3
class mq :1 root
...
class mq :40 root
[13:32 GD-WCP bpf]$ tc filter show dev eth3
<nothing>
I know my query is mainly relating to tc but after looking at a lot of documentation on it I'm not much the wiser about how packet actually traverses tc.
The BPF programs remain the same as earlier with hard-coded value for the ifindex value in bpf_clone_redirect.
Thanks,
Billy.
toggle quoted message
Show quoted text
From: Brenden Blanco [mailto:bblanco@...]
Sent: Wednesday, March 2, 2016 6:55 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
The best source of truth is in the kernel code.
The whitelist of functions for networking is:
other whitelists are available around various pieces where you see BPF_FUNC_*
On Wed, Mar 2, 2016 at 1:48 AM, O Mahony, Billy <billy.o.mahony@...> wrote:
Hi Brendan,
Thanks, that’s great. I’ll have a look at that example you mention too.
Whereabouts in the code should I look to see which functions are allowable in which bpf program types?
I haven’t looked again recently but iirc in some header or man page there is a TODO for description
of the various program types.
If you point me to the relevant area of the code I can do a patch to fill out some of that documentation.
I won’t be able to figure out chapter and verse on it but some basic info would be better than none!
Cheers,
/Billy.
On Tue, Mar 1, 2016 at 9:52 AM, O Mahony, Billy via iovisor-dev <iovisor-dev@...> wrote:
Hi All,
I'm doing my version of hello world for eBPF by forwarding eth frames between two nics - eth1 and eth3 on my machine. Using bpf_clone_redirect(). (cc/export/helpers.h says bpf_redirect is not available until a later kernel.)
When I run the python program it generates:
[17:42 GD-WCP my_bpf]$ sudo python p2p.py
bpf: Invalid argument
0: (b7) r2 = 5
1: (b7) r3 = 0
2: (85) call 13
unknown func 13
Traceback (most recent call last):
File "p2p.py", line 86, in <module>
function_eth1_ic = bpf.load_func("eth1_ic", BPF.SOCKET_FILTER)
File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 165, in load_func
raise Exception("Failed to load BPF program %s" % func_name)
Exception: Failed to load BPF program eth1_ic
You need to use BPF.SCHED_ACT or SCHED_CLS to use these forwarding functions. They are disallowed by the verifier for SOCKET_FILTER programs. The simplest usage (focus on the pyroute2
bits) is probably in tests/cc/test_xlate1.py.
The bpf program is:
int eth1_ic(struct __sk_buff *skb) {
bpf_clone_redirect(skb, 5, 0);
return -1;
}
I've hard coded the egress ifindex based on output from '$ipi a'
Bpf_clone_redirect is indeed the 13th entry in the bpf_func_id enum but why is it reported as 'unknown' ?
I'm running Ubuntu 14.04 with the kernel upgraded using binary packages to 4.3.0-040300-generic from Mon Nov 2 2015.
Thanks,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
Brenden Blanco <bblanco@...>
I will heartily acknowledge that the mechanism to attach bpf programs to an interface is non-intuitive. This has been a source of pain for many. The code in https://github.com/iovisor/iomodules is an attempt to wrap some of this in a rest API, but the target audience for that is other automation tools (container plugins, etc.), so as a learning tool it will be a step back rather than forward. It's also not mature yet, so I'm hesitant to mention it.
See other answers inline.
|
|
|
On 03/03/2016 08:19 PM, Brenden Blanco via iovisor-dev wrote: [...] On Thu, Mar 3, 2016 at 5:52 AM, O Mahony, Billy <billy.o.mahony@...> [...] #add a filter to accept all eth frame. Attach the bpf action? functions
to the filter?
#I have no idea what the classid, target and keys parameters mean!
ip.tc("add-filter", "u32", ifindex_eth1, ":1", parent="ffff:",
action=[action_eth1],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
ip.tc("add-filter", "u32", ifindex_eth3, ":1", parent="ffff:",
action=[action_eth3],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
Yeah, it's a bit of black magic. it's basically creating a match-all rule
(match 0 bytes and offset 0 == true), with the matching action being the
bpf program. The classid and target aren't really meaningful with just one
action and filter, so it's ok to ignore for now.
The clsact qdisc added in 4.5 by Daniel Borkmann is a much better
abstraction for this, and I'll try to upstream some pyroute2 code to
support this.
Plus you may also want to try out cls_bpf with da (direct-action) mode. That will save you the u32 match all classifier config and is faster.
|
|
|
Brenden Blanco <bblanco@...>
|
|
|
On 03/03/2016 10:01 PM, Brenden Blanco wrote: On Thu, Mar 3, 2016 at 11:31 AM, Daniel Borkmann <daniel@...>
wrote:
On 03/03/2016 08:19 PM, Brenden Blanco via iovisor-dev wrote:
[...]
On Thu, Mar 3, 2016 at 5:52 AM, O Mahony, Billy <billy.o.mahony@...
[...]
#add a filter to accept all eth frame. Attach the bpf action? functions
to the filter?
#I have no idea what the classid, target and keys parameters mean!
ip.tc("add-filter", "u32", ifindex_eth1, ":1", parent="ffff:",
action=[action_eth1],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
ip.tc("add-filter", "u32", ifindex_eth3, ":1", parent="ffff:",
action=[action_eth3],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
Yeah, it's a bit of black magic. it's basically creating a match-all rule
(match 0 bytes and offset 0 == true), with the matching action being the
bpf program. The classid and target aren't really meaningful with just one
action and filter, so it's ok to ignore for now.
The clsact qdisc added in 4.5 by Daniel Borkmann is a much better
abstraction for this, and I'll try to upstream some pyroute2 code to
support this. Plus you may also want to try out cls_bpf with da (direct-action) mode.
That will save you the u32 match all classifier config and is faster. Speaking of which, I just opened
https://github.com/svinota/pyroute2/pull/223 to add support in pyroute2
upstream. The docstring in that pull request shows an example usage, which
I'll also add to a testcase in bcc once it merges upstream.
I've done similar additions for go in
https://github.com/vishvananda/netlink/pull/94, if go is your thing. That's awesome, thanks Brenden!
|
|
|
O Mahony, Billy <billy.o.mahony@...>
Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work I'll stick with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT;
However, I still don’t see packets being received by the traffic generator I have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually hello_world.py) I can see the trace statements but not at anything like the rate that I configured the traffic generator to generate packets (100pkts/sec). Also if I vary the rate of traffic ingress on the eth1/3 the ratio of the corresponding trace lines does not change - so not sure what is going on there.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1
filter protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Thanks again,
Billy.
toggle quoted message
Show quoted text
-----Original Message-----
From: Daniel Borkmann [mailto:daniel@...]
Sent: Thursday, March 3, 2016 9:05 PM
To: Brenden Blanco <bblanco@...>
Cc: O Mahony, Billy <billy.o.mahony@...>; iovisor-
dev@...
Subject: Re: [iovisor-dev] unknown func 13
On 03/03/2016 10:01 PM, Brenden Blanco wrote:
On Thu, Mar 3, 2016 at 11:31 AM, Daniel Borkmann
<daniel@...>
wrote:
On 03/03/2016 08:19 PM, Brenden Blanco via iovisor-dev wrote:
[...]
On Thu, Mar 3, 2016 at 5:52 AM, O Mahony, Billy
<billy.o.mahony@...
[...]
#add a filter to accept all eth frame. Attach the bpf action?
functions
to the filter?
#I have no idea what the classid, target and keys parameters mean!
ip.tc("add-filter", "u32", ifindex_eth1, ":1", parent="ffff:",
action=[action_eth1],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
ip.tc("add-filter", "u32", ifindex_eth3, ":1", parent="ffff:",
action=[action_eth3],
protocol=protocols.ETH_P_ALL, classid=1,
target=0x10002, keys=['0x0/0x0+0'])
Yeah, it's a bit of black magic. it's basically creating a
match-all rule (match 0 bytes and offset 0 == true), with the
matching action being the bpf program. The classid and target aren't
really meaningful with just one action and filter, so it's ok to ignore for now.
The clsact qdisc added in 4.5 by Daniel Borkmann is a much better
abstraction for this, and I'll try to upstream some pyroute2 code to
support this.
Plus you may also want to try out cls_bpf with da (direct-action) mode.
That will save you the u32 match all classifier config and is faster.
Speaking of which, I just opened
https://github.com/svinota/pyroute2/pull/223 to add support in
pyroute2 upstream. The docstring in that pull request shows an example
usage, which I'll also add to a testcase in bcc once it merges upstream.
I've done similar additions for go in
https://github.com/vishvananda/netlink/pull/94, if go is your thing.
That's awesome, thanks Brenden!
|
|
|
Brenden Blanco <bblanco@...>
On Fri, Mar 4, 2016 at 10:01 AM, O Mahony, Billy <billy.o.mahony@...> wrote: Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work I'll stick with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT; That's fine, it'll still work.
However, I still don’t see packets being received by the traffic generator I have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually hello_world.py) I can see the trace statements but not at anything like the rate that I configured the traffic generator to generate packets (100pkts/sec). Also if I vary the rate of traffic ingress on the eth1/3 the ratio of the corresponding trace lines does not change - so not sure what is going on there.
Probably arp is failing, your generator is probably doing nothing because it can't resolve the other side.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1
filter protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Is your iproute2 up to date? This usually misses showing the bpf program since it doesn't understand the flags. If you can build iproute2 from source, preferably from the net-next branch, you will see more useful information. I would also include "tc -s" in the output so you can see drops/passes. If you are in fact using the latest iproute2, then the above filters are wrong. It should look more like this: filter protocol all pref 49152 u32 filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2 match 00000000/00000000 at 0 action order 1: bpf on_packet default-action pass index 1 ref 1 bind 1 installed 0 sec used 0 sec Action statistics: Sent 112 bytes 2 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 Thanks again,
Billy.
|
|
|
O Mahony, Billy <billy.o.mahony@...>
Hi Brendan,
Hurrah! I have packets traversing my interfaces via eBPF.
Thanks for all your help.
As I said at one of the iovisor calls I planned to run some basic throughput tests and share the results with you guys. I got some *very* rudimentary figures this afternoon. Though before posting them I'd like to ensure I'm running a reasonably performant setup.
So currently I am running an eBPF action hanging off a u32 filter on the ingress qdisc. Looking at the slide "Hooking in to the Linux network stack (RX)" from bpf_network_examples_2015aug20.pdf it suggests that I could also attach my eBPF program via a TAP device which would avoid the TC overhead.
Are there any examples for that?
Billy.
toggle quoted message
Show quoted text
-----Original Message-----
From: Brenden Blanco [mailto:bblanco@...]
Sent: Friday, March 4, 2016 10:09 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: Daniel Borkmann <daniel@...>; iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Fri, Mar 4, 2016 at 10:01 AM, O Mahony, Billy <billy.o.mahony@...>
wrote:
Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work I'll stick with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT;
That's fine, it'll still work.
However, I still don’t see packets being received by the traffic generator I
have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually hello_world.py) I
can see the trace statements but not at anything like the rate that I
configured the traffic generator to generate packets (100pkts/sec). Also if I
vary the rate of traffic ingress on the eth1/3 the ratio of the corresponding
trace lines does not change - so not sure what is going on there.
Probably arp is failing, your generator is probably doing nothing because it
can't resolve the other side.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter
protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Is your iproute2 up to date? This usually misses showing the bpf program
since it doesn't understand the flags. If you can build
iproute2 from source, preferably from the net-next branch, you will see
more useful information. I would also include "tc -s" in the output so you can
see drops/passes.
If you are in fact using the latest iproute2, then the above filters are wrong. It
should look more like this:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter protocol all pref
49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: bpf on_packet default-action pass
index 1 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 112 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Thanks again,
Billy.
|
|
|
Brenden Blanco <bblanco@...>
On Mon, Mar 7, 2016 at 10:39 AM, O Mahony, Billy <billy.o.mahony@...> wrote: Hi Brendan,
Hurrah! I have packets traversing my interfaces via eBPF. Nice!
Thanks for all your help.
As I said at one of the iovisor calls I planned to run some basic throughput tests and share the results with you guys. I got some *very* rudimentary figures this afternoon. Though before posting them I'd like to ensure I'm running a reasonably performant setup.
So currently I am running an eBPF action hanging off a u32 filter on the ingress qdisc. Looking at the slide "Hooking in to the Linux network stack (RX)" from bpf_network_examples_2015aug20.pdf it suggests that I could also attach my eBPF program via a TAP device which would avoid the TC overhead.
That seems contrary to my own expectations. TC is closer to the metal than SOCKET_FILTER, the ingress qdisc should already be lightweight (lockless for example). The optimizations mentioned earlier in this thread are adding icing to the cake. Besides that, the SOCKET_FILTER program type doesn't allow modification or actions, only capture, so bpf_redirect() wouldn't be available to you there.
Are there any examples for that?
tests/python/test_stat1.py uses a BPF.SOCKET_FILTER program.
Billy.
|
|
|
On 03/07/2016 07:39 PM, O Mahony, Billy via iovisor-dev wrote: Hi Brendan,
Hurrah! I have packets traversing my interfaces via eBPF.
Thanks for all your help.
As I said at one of the iovisor calls I planned to run some basic throughput tests and share the results with you guys. I got some *very* rudimentary figures this afternoon. Though before posting them I'd like to ensure I'm running a reasonably performant setup.
So currently I am running an eBPF action hanging off a u32 filter on the ingress qdisc. Looking at the slide "Hooking in to the Linux network stack (RX)" from bpf_network_examples_2015aug20.pdf it suggests that I could also attach my eBPF program via a TAP device which would avoid the TC overhead. If in terms of tc "overhead" you mean first going through this fake u32 match-all classifier, then, as suggested, you can simply use cls_bpf with direct action mode. Check out the bpf_netdev_conference_2016Feb12.pdf presentation in bpf-docs repo, for example. I presume bcc might support that as well (or soon, depending on the pull reqs?)? Cheers, Daniel Are there any examples for that?
Billy.
-----Original Message-----
From: Brenden Blanco [mailto:bblanco@...]
Sent: Friday, March 4, 2016 10:09 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: Daniel Borkmann <daniel@...>; iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Fri, Mar 4, 2016 at 10:01 AM, O Mahony, Billy <billy.o.mahony@...>
wrote:
Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work I'll stick with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT;
That's fine, it'll still work.
However, I still don’t see packets being received by the traffic generator I
have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually hello_world.py) I
can see the trace statements but not at anything like the rate that I
configured the traffic generator to generate packets (100pkts/sec). Also if I
vary the rate of traffic ingress on the eth1/3 the ratio of the corresponding
trace lines does not change - so not sure what is going on there.
Probably arp is failing, your generator is probably doing nothing because it
can't resolve the other side.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter
protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Is your iproute2 up to date? This usually misses showing the bpf program
since it doesn't understand the flags. If you can build
iproute2 from source, preferably from the net-next branch, you will see
more useful information. I would also include "tc -s" in the output so you can
see drops/passes.
If you are in fact using the latest iproute2, then the above filters are wrong. It
should look more like this:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter protocol all pref
49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: bpf on_packet default-action pass
index 1 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 112 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Thanks again,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
Brenden Blanco <bblanco@...>
On Mon, Mar 7, 2016 at 10:56 AM, Daniel Borkmann <daniel@...> wrote: On 03/07/2016 07:39 PM, O Mahony, Billy via iovisor-dev wrote:
Hi Brendan,
Hurrah! I have packets traversing my interfaces via eBPF.
Thanks for all your help.
As I said at one of the iovisor calls I planned to run some basic
throughput tests and share the results with you guys. I got some *very*
rudimentary figures this afternoon. Though before posting them I'd like to
ensure I'm running a reasonably performant setup.
So currently I am running an eBPF action hanging off a u32 filter on the
ingress qdisc. Looking at the slide "Hooking in to the Linux network stack
(RX)" from bpf_network_examples_2015aug20.pdf it suggests that I could also
attach my eBPF program via a TAP device which would avoid the TC overhead.
If in terms of tc "overhead" you mean first going through this fake u32
match-all
classifier, then, as suggested, you can simply use cls_bpf with direct
action
mode. Check out the bpf_netdev_conference_2016Feb12.pdf presentation in
bpf-docs
repo, for example. I presume bcc might support that as well (or soon,
depending
on the pull reqs?)? Both the pyroute2 and go netlink pull requests have merged, so both clsact and direct-action mode are supported as far as bcc is concerned. Billy mentioned that he wanted to stay on 4.4 kernel for now, so that is the only missing piece.
Cheers,
Daniel
Are there any examples for that?
Billy.
-----Original Message-----
From: Brenden Blanco [mailto:bblanco@...]
Sent: Friday, March 4, 2016 10:09 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: Daniel Borkmann <daniel@...>; iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Fri, Mar 4, 2016 at 10:01 AM, O Mahony, Billy
<billy.o.mahony@...>
wrote:
Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work I'll
stick
with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT;
That's fine, it'll still work.
However, I still don’t see packets being received by the traffic
generator I
have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually
hello_world.py) I
can see the trace statements but not at anything like the rate that I
configured the traffic generator to generate packets (100pkts/sec). Also
if I
vary the rate of traffic ingress on the eth1/3 the ratio of the
corresponding
trace lines does not change - so not sure what is going on there.
Probably arp is failing, your generator is probably doing nothing because
it
can't resolve the other side.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter
protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid
1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Is your iproute2 up to date? This usually misses showing the bpf program
since it doesn't understand the flags. If you can build
iproute2 from source, preferably from the net-next branch, you will see
more useful information. I would also include "tc -s" in the output so
you can
see drops/passes.
If you are in fact using the latest iproute2, then the above filters are
wrong. It
should look more like this:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter protocol
all pref
49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: bpf on_packet default-action pass
index 1 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 112 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Thanks again,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
O Mahony, Billy <billy.o.mahony@...>
toggle quoted message
Show quoted text
-----Original Message-----
From: Brenden Blanco [mailto:bblanco@...]
Sent: Monday, March 7, 2016 7:19 PM
To: Daniel Borkmann <daniel@...>
Cc: O Mahony, Billy <billy.o.mahony@...>; iovisor-
dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Mon, Mar 7, 2016 at 10:56 AM, Daniel Borkmann <daniel@...>
wrote:
On 03/07/2016 07:39 PM, O Mahony, Billy via iovisor-dev wrote:
Hi Brendan,
Hurrah! I have packets traversing my interfaces via eBPF.
Thanks for all your help.
As I said at one of the iovisor calls I planned to run some basic
throughput tests and share the results with you guys. I got some
*very* rudimentary figures this afternoon. Though before posting
them I'd like to ensure I'm running a reasonably performant setup.
So currently I am running an eBPF action hanging off a u32 filter on
the ingress qdisc. Looking at the slide "Hooking in to the Linux
network stack (RX)" from bpf_network_examples_2015aug20.pdf it
suggests that I could also attach my eBPF program via a TAP device which
would avoid the TC overhead.
If in terms of tc "overhead" you mean first going through this fake
u32 match-all classifier, then, as suggested, you can simply use
cls_bpf with direct action mode. Check out the
bpf_netdev_conference_2016Feb12.pdf presentation in bpf-docs repo, for
example. I presume bcc might support that as well (or soon, depending
on the pull reqs?)?
Both the pyroute2 and go netlink pull requests have merged, so both clsact
and direct-action mode are supported as far as bcc is concerned. Billy
mentioned that he wanted to stay on 4.4 kernel for now, so that is the only
missing piece. [[BO'M]] Now that I have something working I feel much happier about compiling a new kernel. I'll checkout the examples and docs you recommend. Thanks, Daniel & Brendan.
Cheers,
Daniel
Are there any examples for that?
Billy.
-----Original Message-----
From: Brenden Blanco [mailto:bblanco@...]
Sent: Friday, March 4, 2016 10:09 PM
To: O Mahony, Billy <billy.o.mahony@...>
Cc: Daniel Borkmann <daniel@...>;
iovisor-dev@...
Subject: Re: [iovisor-dev] unknown func 13
On Fri, Mar 4, 2016 at 10:01 AM, O Mahony, Billy
<billy.o.mahony@...>
wrote:
Hi Daniel, Brendan,
Thanks for the tips. Unless what I'm trying to do just won't work
I'll stick
with kernel 4.4 and the ingress qdisc for now.
So I changed my bpf programs to look like this:
6 int eth1_ic(struct __sk_buff *skb) {
7 bpf_trace_printk ("eth1_ic\n");
8 bpf_redirect(5, 0);
9 return TC_ACT_REDIRECT;
That's fine, it'll still work.
However, I still don’t see packets being received by the traffic
generator I
have attached to eht1 and eth3.
If I also run BPF.trace_print in a separate console (actually
hello_world.py) I
can see the trace statements but not at anything like the rate that
I configured the traffic generator to generate packets
(100pkts/sec). Also if I vary the rate of traffic ingress on the
eth1/3 the ratio of the corresponding trace lines does not change -
so not sure what is going on there.
Probably arp is failing, your generator is probably doing nothing
because it can't resolve the other side.
avahi-daemon-1006 [031] ..s. 10942.915221: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.100833: : eth1_ic
avahi-daemon-1006 [031] ..s. 10943.182141: : eth3_ic
avahi-daemon-1006 [031] ..s. 10943.801170: : eth1_ic
avahi-daemon-1006 [031] ..s. 10944.375068: : eth3_ic
avahi-daemon-1006 [031] ..s. 10944.560597: : eth3_ic
BTW my tc filter output looks like so:
$ tc filter show dev eth3 parent ffff:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter
protocol all pref 49152 u32 fh 800::1 order 1 key ht 800 bkt 0
flowid
1:2
match 00000000/00000000 at 0
action order 1: gact action pass
random type none pass val 0
index 10 ref 1 bind 1
Is your iproute2 up to date? This usually misses showing the bpf
program since it doesn't understand the flags. If you can build
iproute2 from source, preferably from the net-next branch, you will
see more useful information. I would also include "tc -s" in the
output so you can see drops/passes.
If you are in fact using the latest iproute2, then the above filters
are wrong. It should look more like this:
filter protocol all pref 49152 u32
filter protocol all pref 49152 u32 fh 800: ht divisor 1 filter
protocol all pref
49152 u32 fh 800::1 order 1 key ht 800 bkt 0 flowid 1:2
match 00000000/00000000 at 0
action order 1: bpf on_packet default-action pass
index 1 ref 1 bind 1 installed 0 sec used 0 sec
Action statistics:
Sent 112 bytes 2 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Thanks again,
Billy.
_______________________________________________
iovisor-dev mailing list
iovisor-dev@...
https://lists.iovisor.org/mailman/listinfo/iovisor-dev
|
|
|
