Date   

Re: Behavior of bpf_obj_get

Yonghong Song
 

On Thu, May 30, 2019 at 11:26 AM Adam Drescher
<adam.r.drescher@...> wrote:

Thank you for your answer, that makes sense.

This leads me to a follow up question: is there a standard way to see
if a pinned map has been reloaded, without closing and re-opening the
map file every time to check? From your answer, now I know we cannot
If you take a reference count for the pinned map, the map cannot be
removed.
If you are talking about the content change of a pinned map, the applicaiton
has to check the map key/values to find it. Just like shared memory.

do this by monitoring a change in the map's file descriptor. We cannot
use the map ID from bpf_obj_get_info_by_fd, as the info still
corresponds to the old map. As far as I can tell, we don't have access
to the BPF object or program pointers in a standalone userspace
daemon.
You got a map fd, you should use map fd to do standard map lookup/insert/delete
operations.


On Thu, May 30, 2019 at 12:00 PM Y Song <ys114321@...> wrote:

On Thu, May 30, 2019 at 8:40 AM Adam Drescher <adam.r.drescher@...> wrote:

I am seeing unexpected behavior from bpf_obj_get, although this is
likely due to my inexperience with BPF.

In a loader program, I create a pinned map at
"sys/fs/bpf/test/xdp_stats_map". In a separate statistics program, I
access the pinned map via file descriptor -- I got the file descriptor
from a call to bpf_obj_get and provided the pathname above.

In a polling loop, I call bpf_obj_get on the same pathname and compare
this value to the original. However, instead of getting the same file
descriptor, the file descriptor returned by bpf_obj_get increments by
1 each invocation (so it returns 4, 5, 6, 7, ...). I am not doing
anything externally to reload the map or change the file descriptor.
Why is this happening? Looking at samples/bpf/fds_example.c as example
usage of bpf_obj_get, I would expect to get the same file descriptor
back. Any ideas?
No, you won't get the same file descriptor although they pointing to
the same map.
- the original map fd holds a reference count in the kernel.
- bpf_obj_get returns a new fd and map reference count is increased
by 1 as well

You can close the original map fd and the fd returned by bpf_obj_get() would
still be valid, and vice verse.


Relevant code (filename has already been populated elsewhere):

#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
char filename[PATH_MAX];

static void stats_poll(int map_fd, __u32 map_type, int interval)
{
int fd;
while (1) {
fd = bpf_obj_get(filename);
printf("filename: %s\n", filename);
printf("bpf_obj_get: %d - map_fd: %d\n", fd, map_fd);
sleep(interval);
}
}

Relevant output:
- BPF map (bpf_map_type:6) id:42 name:xdp_stats_map key_size:4
value_size:16 max_entries:5
filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 4 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 5 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 6 - map_fd: 3



Re: Behavior of bpf_obj_get

Adam Drescher
 

Thank you for your answer, that makes sense.

This leads me to a follow up question: is there a standard way to see
if a pinned map has been reloaded, without closing and re-opening the
map file every time to check? From your answer, now I know we cannot
do this by monitoring a change in the map's file descriptor. We cannot
use the map ID from bpf_obj_get_info_by_fd, as the info still
corresponds to the old map. As far as I can tell, we don't have access
to the BPF object or program pointers in a standalone userspace
daemon.

On Thu, May 30, 2019 at 12:00 PM Y Song <ys114321@...> wrote:

On Thu, May 30, 2019 at 8:40 AM Adam Drescher <adam.r.drescher@...> wrote:

I am seeing unexpected behavior from bpf_obj_get, although this is
likely due to my inexperience with BPF.

In a loader program, I create a pinned map at
"sys/fs/bpf/test/xdp_stats_map". In a separate statistics program, I
access the pinned map via file descriptor -- I got the file descriptor
from a call to bpf_obj_get and provided the pathname above.

In a polling loop, I call bpf_obj_get on the same pathname and compare
this value to the original. However, instead of getting the same file
descriptor, the file descriptor returned by bpf_obj_get increments by
1 each invocation (so it returns 4, 5, 6, 7, ...). I am not doing
anything externally to reload the map or change the file descriptor.
Why is this happening? Looking at samples/bpf/fds_example.c as example
usage of bpf_obj_get, I would expect to get the same file descriptor
back. Any ideas?
No, you won't get the same file descriptor although they pointing to
the same map.
- the original map fd holds a reference count in the kernel.
- bpf_obj_get returns a new fd and map reference count is increased
by 1 as well

You can close the original map fd and the fd returned by bpf_obj_get() would
still be valid, and vice verse.


Relevant code (filename has already been populated elsewhere):

#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
char filename[PATH_MAX];

static void stats_poll(int map_fd, __u32 map_type, int interval)
{
int fd;
while (1) {
fd = bpf_obj_get(filename);
printf("filename: %s\n", filename);
printf("bpf_obj_get: %d - map_fd: %d\n", fd, map_fd);
sleep(interval);
}
}

Relevant output:
- BPF map (bpf_map_type:6) id:42 name:xdp_stats_map key_size:4
value_size:16 max_entries:5
filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 4 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 5 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 6 - map_fd: 3



Re: bpftrace ustack() pid packing

Daniel Xu
 

Thanks Brendan, that makes sense.

Here's an example that can trigger the fork issue:

$ cat ustack_fork_dup.cpp
#include <iostream>
#include <unistd.h>
#include <stdio.h>

__attribute__((noinline)) void do_sleep() {
::usleep(500000);
}

__attribute__((noinline)) void pathAAAA() {
do_sleep();
}

__attribute__((noinline)) void pathAAA() {
pathAAAA();
}

__attribute__((noinline)) void pathAA() {
pathAAA();
}

__attribute__((noinline)) void pathA() {
pathAA();
}


int main() {
::fork();

while (1) {
pathA();
}
}

$ sudo ./build/src/bpftrace -e 'uprobe:/lib64/libc.so.6:usleep /comm == "a.out"/ { @[ustack(2)] = count() }'
Attaching 1 probe...
^C

@[
usleep+0
pathAAAA()+9
]: 16
@[
usleep+0
pathAAAA()+9
]: 16

The user would expect only a single entry in the map.

Daniel

On Tue, May 28, 2019, at 1:01 PM, Brendan Gregg wrote:
On Tue, May 28, 2019 at 11:35 AM Daniel Xu <dxu@...> wrote:

I'm reading through codegen_llvm.cpp[0] and I noticed that ustack
map keys are packed with a PID. From what I understand, if you run

bpftrace -e 'uprobe:/lib64/libc.so.6:usleep /comm == "a.out"/ { @[ustack(2)] = count() }'

and a.out has forked, it would cause duplicate stack traces to be outputted.
This has caused some confusion in the past with users.

I'm wondering what the intention behind PID packing is.

The stack ID returned by ustack() points to a stack as a list of
addresses. That gets translated sometime later in user-space by
bpftrace. At that point you just have an address, and you need to know
which PID it belongs to, so you can lookup that processes's address
mappings.

If BPF one day can save the stack as a list of strings in the kernel
(meaning the kernel must support user-space symbol lookup), then we
wouldn't need the PID (this has been discussed before).

I don't know about this fork problem you mention.

Brendan




Thanks,
Daniel
---
[0]: https://github.com/iovisor/bpftrace/blob/master/src/ast/codegen_llvm.cpp#L748-L756





Re: Behavior of bpf_obj_get

Yonghong Song
 

On Thu, May 30, 2019 at 8:40 AM Adam Drescher <adam.r.drescher@...> wrote:

I am seeing unexpected behavior from bpf_obj_get, although this is
likely due to my inexperience with BPF.

In a loader program, I create a pinned map at
"sys/fs/bpf/test/xdp_stats_map". In a separate statistics program, I
access the pinned map via file descriptor -- I got the file descriptor
from a call to bpf_obj_get and provided the pathname above.

In a polling loop, I call bpf_obj_get on the same pathname and compare
this value to the original. However, instead of getting the same file
descriptor, the file descriptor returned by bpf_obj_get increments by
1 each invocation (so it returns 4, 5, 6, 7, ...). I am not doing
anything externally to reload the map or change the file descriptor.
Why is this happening? Looking at samples/bpf/fds_example.c as example
usage of bpf_obj_get, I would expect to get the same file descriptor
back. Any ideas?
No, you won't get the same file descriptor although they pointing to
the same map.
- the original map fd holds a reference count in the kernel.
- bpf_obj_get returns a new fd and map reference count is increased
by 1 as well

You can close the original map fd and the fd returned by bpf_obj_get() would
still be valid, and vice verse.


Relevant code (filename has already been populated elsewhere):

#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
char filename[PATH_MAX];

static void stats_poll(int map_fd, __u32 map_type, int interval)
{
int fd;
while (1) {
fd = bpf_obj_get(filename);
printf("filename: %s\n", filename);
printf("bpf_obj_get: %d - map_fd: %d\n", fd, map_fd);
sleep(interval);
}
}

Relevant output:
- BPF map (bpf_map_type:6) id:42 name:xdp_stats_map key_size:4
value_size:16 max_entries:5
filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 4 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 5 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 6 - map_fd: 3



Behavior of bpf_obj_get

Adam Drescher
 

I am seeing unexpected behavior from bpf_obj_get, although this is
likely due to my inexperience with BPF.

In a loader program, I create a pinned map at
"sys/fs/bpf/test/xdp_stats_map". In a separate statistics program, I
access the pinned map via file descriptor -- I got the file descriptor
from a call to bpf_obj_get and provided the pathname above.

In a polling loop, I call bpf_obj_get on the same pathname and compare
this value to the original. However, instead of getting the same file
descriptor, the file descriptor returned by bpf_obj_get increments by
1 each invocation (so it returns 4, 5, 6, 7, ...). I am not doing
anything externally to reload the map or change the file descriptor.
Why is this happening? Looking at samples/bpf/fds_example.c as example
usage of bpf_obj_get, I would expect to get the same file descriptor
back. Any ideas?

Relevant code (filename has already been populated elsewhere):

#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
char filename[PATH_MAX];

static void stats_poll(int map_fd, __u32 map_type, int interval)
{
int fd;
while (1) {
fd = bpf_obj_get(filename);
printf("filename: %s\n", filename);
printf("bpf_obj_get: %d - map_fd: %d\n", fd, map_fd);
sleep(interval);
}
}

Relevant output:
- BPF map (bpf_map_type:6) id:42 name:xdp_stats_map key_size:4
value_size:16 max_entries:5
filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 4 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 5 - map_fd: 3

filename: /sys/fs/bpf/test/xdp_stats_map
bpf_obj_get: 6 - map_fd: 3


Re: how to make redirect_map work?

Mauricio Vasquez
 


On 5/30/19 4:46 AM, forrest0579@... wrote:
Hi all,

Recently I try to build a network topology to test redirect_map, the topology is as below and the build script in the attachment(setup_env.sh).


My test case is run `ping` from netns ns1 to 192.168.2.2 in netns ns2.
I load xdp prog to vhost1 and vhost2. What the prog do is just use redirect_map to redirect package from one ingress to another egress(AFAIK, redirect func is working for that, ifx me if I am wrong).

The result is I can't ping from one netns to another. The xdp prog is in the attachment(xdp_ping.py).

Thanks for helping me!

Forrest

Hello Forrest,

You're using veth interfaces, in this case you have to attach the program in SKB mode, to do it set flags = 1 << 1.

Mauricio.



how to make redirect_map work?

Forrest Chen
 

Hi all,

Recently I try to build a network topology to test redirect_map, the topology is as below and the build script in the attachment(setup_env.sh).


My test case is run `ping` from netns ns1 to 192.168.2.2 in netns ns2.
I load xdp prog to vhost1 and vhost2. What the prog do is just use redirect_map to redirect package from one ingress to another egress(AFAIK, redirect func is working for that, ifx me if I am wrong).

The result is I can't ping from one netns to another. The xdp prog is in the attachment(xdp_ping.py).

Thanks for helping me!

Forrest


minutes: IO Visor TSC/Dev Meeting

Brenden Blanco
 

Hi all,

Thanks for joining in to the call today. As usual, here are my notes.

Cheers,
Brenden

=== Discussion ===

Daniel:
- Some cilium debugging for loadbalancing case
- Fix sendmsg hooks for unconnected udp
- should fix dns resolution issue in cilium

Yonghong:
- Released new bcc version (0.10.0)
- Some new revisions in LLVM for CO-RE support
- bpf_send_signal is complete, support being added to libbpf

Brendan:
- nearing book deadline, send in your last minute reviews

Alexei:
- CFP for Linux Plumbers is open (Sep 9-11, Portugal)
- https://www.linuxplumbersconf.org/event/4/abstracts/
- BPF subtrack will be taking place (some overlap with networking)

Maciej:
- working XSK (AF_XDP socket) support in libbpf
- some issues without an obvious solution, details to be sent to mailing list

=== Attendees ===
Brenden Blanco
Florian Lehner
Paul Chaignon
Dan Siemon
Richard Elling
Andy Gospodarek
Marco Leogrande
Maciej Fijalkowski
Daniel Borkmann
Alexei Starovoitov
Jiong Wang
Jakub Kicinski
Jesper Brouer
Brendan Gregg
Mauricio Vasquez
Joe Stringer


Re: how to set environment to run xdp_redirect_map.py example?

Forrest Chen
 

Thanks a lot, Gary!

Very appreciate for your answer.

Forrest


Re: how to set environment to run xdp_redirect_map.py example?

Gary Lin
 

On Wed, May 29, 2019 at 12:04:53AM -0700, forrest0579@... wrote:
On Tue, May 28, 2019 at 09:20 PM, Gary Lin wrote:


On Tue, May 28, 2019 at 07:32:33PM -0700, forrest0579@... wrote:

Hi all,
What kind of network topology should I set up to make xdp_redirect_map.py
runnable and what user-case this example represent for?
The script does nothing really meaningful but demostrates how to use
XDP_REDIRECT.

It redirects all traffic from one network interface to another, so you
need a machine with at least two XDP-enabled network interfaces. When you
send a packet from eth0 of Machine B to eth0 of Machine A, the script
swaps the src IP and dst IP of the incoming packets and sends them out
through eth1 of Machine A, so the packets will flow like this:

(Machine A) (Machine B)
[ eth0 ]<------[ eth0 ]
| ^
v |
[ eth1 ] ---------+

The script also counts the number of packet processed per second as a
rough performance measurement for XDP_REDIRECT, so you can generate a
huge volume of traffic with the programs like trafgen in Machine B to
do the test.

Cheers,

Gary Lin


PS: I have the same question about xdp_macswap_count.py too

Thanks for helping me!
Forrest


These two examples swap mac instead of IP address.
And as an effect with the given topology, the egress package from MachineA-eth1 has dst mac of MachineB-eth0 and src mac of MachineA-eth0, dst IP of MachineA and src IP of MachineB. Assume that due to the dst mac, the package finally delivered back to MachineB, but the dst IP does not match the IP in MachineB.  I have no idea in which kind of topology the package could be consumed.
Sorry, my memory betrayed me. Those two scripts swap the mac, not IP.
As I said, xdp_redirect_map.py is a demo script, so it doesn't care if
the packets are dropped in Machine B or not. In the real world, users
of XDP_REDIRECT usually modify the IP address or prepend a header
with bpf_xdp_adjust_head(), so that those packets can be passed further.

Another question, after swap mac, shouldn't we re-calculate the l2 csum? or if I modify that field like src/dst IP/Port, how can I re-calculate these csum? Does BPF_FUNC_csum_diff helper function work for this?
AFAIK, the frame checksum is calculated by the network card, so we don't
have to do the calculation.

Cheers,

Gary Lin


Re: how to set environment to run xdp_redirect_map.py example?

Forrest Chen
 

On Tue, May 28, 2019 at 09:20 PM, Gary Lin wrote:
On Tue, May 28, 2019 at 07:32:33PM -0700, forrest0579@... wrote:
Hi all,
What kind of network topology should I set up to make xdp_redirect_map.py runnable and what user-case this example represent for?
The script does nothing really meaningful but demostrates how to use
XDP_REDIRECT.

It redirects all traffic from one network interface to another, so you
need a machine with at least two XDP-enabled network interfaces. When you
send a packet from eth0 of Machine B to eth0 of Machine A, the script
swaps the src IP and dst IP of the incoming packets and sends them out
through eth1 of Machine A, so the packets will flow like this:

(Machine A) (Machine B)
[ eth0 ]<------[ eth0 ]
| ^
v |
[ eth1 ] ---------+

The script also counts the number of packet processed per second as a
rough performance measurement for XDP_REDIRECT, so you can generate a
huge volume of traffic with the programs like trafgen in Machine B to
do the test.

Cheers,

Gary Lin

PS: I have the same question about xdp_macswap_count.py too

Thanks for helping me!
Forrest




These two examples swap mac instead of IP address.
And as an effect with the given topology, the egress package from MachineA-eth1 has dst mac of MachineB-eth0 and src mac of MachineA-eth0, dst IP of MachineA and src IP of MachineB. Assume that due to the dst mac, the package finally delivered back to MachineB, but the dst IP does not match the IP in MachineB.  I have no idea in which kind of topology the package could be consumed.

Another question, after swap mac, shouldn't we re-calculate the l2 csum? or if I modify that field like src/dst IP/Port, how can I re-calculate these csum? Does BPF_FUNC_csum_diff helper function work for this?

Thanks

Forrest


Re: how to set environment to run xdp_redirect_map.py example?

Gary Lin
 

On Tue, May 28, 2019 at 07:32:33PM -0700, forrest0579@... wrote:
Hi all,
What kind of network topology should I set up to make xdp_redirect_map.py runnable and what user-case this example represent for?
The script does nothing really meaningful but demostrates how to use
XDP_REDIRECT.

It redirects all traffic from one network interface to another, so you
need a machine with at least two XDP-enabled network interfaces. When you
send a packet from eth0 of Machine B to eth0 of Machine A, the script
swaps the src IP and dst IP of the incoming packets and sends them out
through eth1 of Machine A, so the packets will flow like this:

(Machine A) (Machine B)
[ eth0 ]<------[ eth0 ]
| ^
v |
[ eth1 ] ---------+

The script also counts the number of packet processed per second as a
rough performance measurement for XDP_REDIRECT, so you can generate a
huge volume of traffic with the programs like trafgen in Machine B to
do the test.

Cheers,

Gary Lin

PS: I have the same question about xdp_macswap_count.py too

Thanks for helping me!
Forrest



reminder: IO Visor TSC/Dev Meeting

Brenden Blanco
 

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am | Pacific Daylight Time (San Francisco, GMT-07:00) | 30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2019&month=5&day=29&hour=18&min=0&sec=0&p1=900


how to set environment to run xdp_redirect_map.py example?

Forrest Chen
 

Hi all,
What kind of network topology should I set up to make xdp_redirect_map.py runnable and what user-case this example represent for? 

PS: I have the same question about xdp_macswap_count.py too

Thanks for helping me!
Forrest


Re: bpftrace ustack() pid packing

Brendan Gregg
 

On Tue, May 28, 2019 at 11:35 AM Daniel Xu <dxu@...> wrote:

I'm reading through codegen_llvm.cpp[0] and I noticed that ustack
map keys are packed with a PID. From what I understand, if you run

bpftrace -e 'uprobe:/lib64/libc.so.6:usleep /comm == "a.out"/ { @[ustack(2)] = count() }'

and a.out has forked, it would cause duplicate stack traces to be outputted.
This has caused some confusion in the past with users.

I'm wondering what the intention behind PID packing is.

The stack ID returned by ustack() points to a stack as a list of
addresses. That gets translated sometime later in user-space by
bpftrace. At that point you just have an address, and you need to know
which PID it belongs to, so you can lookup that processes's address
mappings.

If BPF one day can save the stack as a list of strings in the kernel
(meaning the kernel must support user-space symbol lookup), then we
wouldn't need the PID (this has been discussed before).

I don't know about this fork problem you mention.

Brendan




Thanks,
Daniel
---
[0]: https://github.com/iovisor/bpftrace/blob/master/src/ast/codegen_llvm.cpp#L748-L756



bpftrace ustack() pid packing

Daniel Xu
 

I'm reading through codegen_llvm.cpp[0] and I noticed that ustack
map keys are packed with a PID. From what I understand, if you run

bpftrace -e 'uprobe:/lib64/libc.so.6:usleep /comm == "a.out"/ { @[ustack(2)] = count() }'

and a.out has forked, it would cause duplicate stack traces to be outputted.
This has caused some confusion in the past with users.

I'm wondering what the intention behind PID packing is.

Thanks,
Daniel
---
[0]: https://github.com/iovisor/bpftrace/blob/master/src/ast/codegen_llvm.cpp#L748-L756


Re: Do you know if I can use a bpf file from bcc for snort ?

Dorian ROSSE
 

Python and c files doesn’t success to compiles

 

Provenance : Courrier pour Windows 10

 


De : Y Song <ys114321@...>
Envoyé : Tuesday, May 28, 2019 7:06:50 AM
À : Dorian ROSSE
Cc : iovisor-dev@...
Objet : Re: [iovisor-dev] Do you know if I can use a bpf file from bcc for snort ?
 
On Mon, May 27, 2019 at 4:04 AM Dorian ROSSE <dorianbrice@...> wrote:
>
> Hello everybody,
>
>
> Do you know if I can use a bpf file from bcc for snort ?

You mean a bpf program, right?
Do you mean to have a bpf program to do L7 parsing?
If simple one, it should work. See bcc/examples/networking/http_filter/*.
But since kernel verifier currently does not support loops,
the complex options may not be supported.

>
> Thank you in advance to answer if I can and how to do my ask,
>
> Regards.
>
>
> Dorian ROSSE.
>
>
>
> Provenance : Courrier pour Windows 10
>
>
>


Re: Do you know if I can use a bpf file from bcc for snort ?

Yonghong Song
 

On Mon, May 27, 2019 at 4:04 AM Dorian ROSSE <dorianbrice@...> wrote:

Hello everybody,


Do you know if I can use a bpf file from bcc for snort ?
You mean a bpf program, right?
Do you mean to have a bpf program to do L7 parsing?
If simple one, it should work. See bcc/examples/networking/http_filter/*.
But since kernel verifier currently does not support loops,
the complex options may not be supported.


Thank you in advance to answer if I can and how to do my ask,

Regards.


Dorian ROSSE.



Provenance : Courrier pour Windows 10




Do you know if I can use a bpf file from bcc for snort ?

Dorian ROSSE
 

Hello everybody,


Do you know if I can use a bpf file from bcc for snort ?

Thank you in advance to answer if I can and how to do my ask,

Regards.


Dorian ROSSE.

 

Provenance : Courrier pour Windows 10

 


Re: Facing an error while compiling for bpf using clang

Prashanth Fernando
 

Thank You Daniel for your timely help!

321 - 340 of 2021