This group is locked. No changes can be made to the group while it is locked.
|
First!
Brenden Blanco <bblanco@...>
Welcome IOVisor Devs! This list is now open :)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updates and Meeting Minutes from TSC+Developer bi-weekly
Brenden Blanco <bblanco@...>
Hello, iovisor-dev,
Since this list just got set up, we'll play a bit of catch-up. First, if you haven't yet and would like to, please sign yourself up on http://lists.iovisor.org/mailman/listinfo/iovisor-dev. To send to this mailing list you must register, but the archives can be read by anyone. ================ Wednesday, Sept 02 2015 was the first bi-weekly TSC+Developer meeting. Meeting minutes are unfortunately sparse, however the slides from the discussion are now available in https://github.com/iovisor/bpf-docs under the tsc-meeting-minutes/2015-09-02 folder. ================ Wednesday, Sept 16 2015 was the 2nd bi-weekly TSC+Developer meeting. 1) Keith spent the first half of the meeting presenting Open Daylight's Group Based Policy (GBP) - The presentation slides are available at [1]. - A proposal that comes out of that presentation is to showcase an integration between ODL+GBP on the control plane side and IOV on the data plane side. There was consensus that such an integration is interesting and useful. - We will finalize (offline) the YAML model for the GBP DP - Keith to write a renderer in ODL for IOV-based GBP - PG to write the DP implementation of GBP - Will schedule a meeting early next week to close the data structure/yaml/json details - documentation to be sent to this list 2) Brenden covered thoughts on the design of IOModules, whats been done so far and what is planned. - The presentation slides are available at [2]. - Mihai had comments based on the requirements of the P4->IOVisor compiler discussed during the week. Slides for that are available at [3]. - We have yet to close the definition of an IOModule - So far, we are taking a bottoms-up approach, acknowledging that the design may change when taking other hardwares/backends into account. - Working to close the definition of the network specific definition of IFC (other types to come in the future, e.g. storage). - Need to define and abstract the way IOModules will be connected, the current raw approach of attaching eBPF programs to TC/netdev is too primitive. - IOModule should capture linux building blocks such as LB, router namespace, be able to connect them to eBPF programs seamlessly. - Plan is to make some of these definitions concrete with the GBP implementation. - Mentioned fuse approach as one possible way to create generic, non-networking approach to composing IOModules. - Demo code is available at https://github.com/iovisor/bcc-fuse. - Possible to give eBPF programs permissions - How do we do service insertion? ================ Next meeting (2 weeks): - Present the final design of the GBP - ODL work - ?? John F. to discuss DPDK possibilities - Other topics, please send to the list if you would like to add it to the agenda. ================ References: [1]: https://github.com/iovisor/bpf-docs/blob/master/tsc-meeting-minutes/2015-09-16/iovisor-odl-gbp-module.pdf [2]: https://github.com/iovisor/bpf-docs/blob/master/tsc-meeting-minutes/2015-09-16/iomodules-slides.pdf [3]: https://github.com/iovisor/bpf-docs/blob/master/p4AbstractSwitch.pdf ================ Thanks all, cheers! Brenden Blanco bblanco@... |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
IOVisor help
ashhadsheikh394@...
Hello, I am Ashhad. A CS major undergraduate student. I am researching on iOVISOR, specifically on it's real life performance.
I want to ask that Is it possible to have netlink communication between eBPF code and userspace --- if so how? I want to see if user space programs can communicate with eBPF beyond polling shared maps or not. Waiting for a helpful response Regards |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: IOVisor help
Brenden Blanco <bblanco@...>
Hello Ashhad, It would be good to know what type of use case you have in mind for transferring data between user and kernel space, then perhaps I can answer more accurately. For instance, if using the KPROBE program type, it is possible to use a BPF program as a filter of events that are later accessible through mmap, see "MMAP layout" in http://man7.org/linux/man-pages/man2/perf_event_open.2.html. Or, if using a SOCKET type, the BPF program works as it does traditionally with tcpdump, to filter packets into a socket. A userspace program can listen on said socket. For the SCHED_ACT and SCHED_CLS types, the BPF program can use a helper function to clone or redirect a packet to an ifindex of its choosing. Please let me know which of these categories you are interested in. Thanks, Brenden Blanco On Tue, Sep 22, 2015 at 4:57 AM, Ashhad Sheikh via iovisor-dev <iovisor-dev@...> wrote: Hello, I am Ashhad. A CS major undergraduate student. I am researching on iOVISOR, specifically on it's real life performance. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: IOVisor help
Ashhad Sheikh <ashhadsheikh394@...>
Hello Brenden, Thanks for your helpful replay. These are actually interesting things you mentioned I will surely look into these. Using SOCKET was my initial idea but I want to know that are they full duplex or not?(What if user space wants to send back something) I simple want to send traffic from data plane to user space and back to data plane after some processing. Also it will be highly appreciable if you please link me any example related to SOCKETS. Regards Ashhad On Wed, Sep 23, 2015 at 10:00 AM Brenden Blanco <bblanco@...> wrote:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: IOVisor help
Yonghong Song
Ashhad, At this moment, you can look at the following two tests:Test #10: py_test_brb Test #11: py_test_brb2 On Tue, Sep 29, 2015 at 3:35 AM, Ashhad Sheikh via iovisor-dev <iovisor-dev@...> wrote:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updates and Meeting Minutes from TSC+Developer bi-weekly (2015/9/30)
Brenden Blanco <bblanco@...>
Hello iovisor-dev,
Today was the biweekly IOVisor TSC/Dev call, for which the meeting minutes are included below. We primarily covered status for the previous two weeks. === Meetups === On 2015/9/21, Huawei hosted an IOVisor Meetup through http://www.meetup.com/Silicon-Valley-Linux-Technology - Presenters: Pere Monclus, Brenden Blanco, Brendan Gregg, Jianwen Pi - Great attendance (~100 people), interesting questions and interactive audience. Possible meetup next month hosted by Cisco, stay tuned === Blog posts === Brendan Gregg "bcc: Taming Linux 4.3+ Tracing Superpowers" http://www.brendangregg.com/blog/2015-09-22/bcc-linux-4.3-tracing.html Interesting discussion on hackernews related to this: systemtap vs dtrace vs bcc https://news.ycombinator.com/item?id=10262216 Suchakra Sharma "Linux Networking, Tracing and IO Visor, a New Systems Performance Tool for a Distributed World" http://thenewstack.io/comparing-dtrace-iovisor-new-systems-performance-platform-advance-linux-networking-virtualization/ === Code changes === BCC: - from Mihai Budiu: Initial P4->eBPF (BCC) compiler - from Brendan Gregg: Many tracing tools updates from Brendan Gregg, showcased at meetup - biolatency, funclatency, tcpv4connect, opensnoop, killsnoop - from Brenden Blanco: Tracing bugfixes and enhancements - nested probe reads, built in histogram support - ability to consume perf events in python - from Suchakra Sharma: Kprobe features, bugfixes, and documentation updates LLVM: - various bugfixes for eBPF backend === GBP Efforts === Keith is Working with Huawei to merge code to improve the renderer, once this merges will ease the GBP<->IOVisor implementation detailed last meeting. Plan to start coding prototypes this week, with working json examples provided to the team for lower level work to build off of. === Additional discussion topics === Keith suggested that we create a top-level page on github.com/iovisor, contents to be a wiki page, news, updates, etc. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
callchains & args
Brendan Gregg
G'Day, I wrote a long script that, as a basic example, tries to do the following in C:} - tests/cc/test_callchain.py uses a callback and a b.kprobe_poll() loop to fetch the callchain. - Many other examples use a b.trace_fields() loop to fetch the printk output. I haven't found a way to do both at the same time. I'd like the callchain with the output of printk together. Thoughts? Is this just another example of pushing bpf_trace_printk() too far? Could the callback arguments be extended to be more than "pid, callchain"? If the "return 1" and callback method is reading the raw perf_event, is there a way to read the fmt string? (which is usually set to something useful for tracepoints). Could there be a bpf_trace_fmt(), to customize such a string for kprobes? thanks, Brendan |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: callchains & args
Brenden Blanco <bblanco@...>
On Mon, Oct 05, 2015 at 01:32:14PM -0700, Brendan Gregg via iovisor-dev wrote:
G'Day,There can always be a python thread for this, but I don't believe there would be a way to correlate the two events, since they'll come in at different times and with different chances of being rate limited. In a nutshell: yes. Where I'd like to eventually get is one poll loop, which is the kprobe_poll/perf one. The trace_pipe based approach should eventually die, but if we do it right the tools/examples won't notice (we can migrate trace_fields() to map onto kprobe_poll). Just the underlying implementation will change. The information in the printk can in theory be returned over the ring buffer, but there is still some kernel work to be done. Multiple people have asked for this, so I would place it high on the priority list with a good chance of it making it into a near-future kernel. Yes, take a look at what is available in linux/perf_event.h: enum perf_event_sample_format { PERF_SAMPLE_* ... } If you have something specific in there that would be useful, we can add code for it. I don't have time to implement the whole list at the moment. This is the TBD kernel work. I want this too :)
_______________________________________________ |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: callchains & args
Brendan Gregg
On Mon, Oct 5, 2015 at 1:53 PM, Brenden Blanco <bblanco@...> wrote:
On Mon, Oct 05, 2015 at 01:32:14PM -0700, Brendan Gregg via iovisor-dev wrote:I dug through the list and realized that the actual integer I want toG'Day,There can always be a python thread for this, but I don't believe there emit in this case -- which is not ctx->ax like my simple example, but is the thread blocked time -- can already be exposed as PERF_SAMPLE_PERIOD. I blogged about it previously: http://www.brendangregg.com/blog/2015-02-26/linux-perf-off-cpu-flame-graph.html So maybe my particular case can be served with PERF_SAMPLE_PERIOD support. Although there would need to be a way to set it from bcc/eBPF. Alternately, I could roughly associate kprobe_poll() and trace_fields() given PERF_SAMPLE_TIME, and maybe PERF_SAMPLE_CPU (along with the PID). Brendan |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
IO Visor @OpenStack Summit Tokyo Collaboration Day
Valentina Alaria <vale@...>
All, IO Visor project got three 40 minutes slots at the upcoming Collaboration Day at the OS Summit in Tokyo, Japan at the end of the month (Thursday 10/29). If you (or your company) are attending and are interested in participating and presenting (or co-presenting) please let me know ASAP as we have to submit abstracts and speakers by tomorrow COB. We have identified the following topics for the three slots (but please feel free to send feedback/idea in): Slot 1: IO Visor Project Introduction and BPF architecture Slot 2: IO Visor and Tracing: Use Cases and Applications Slot 3: IO Visor and Networking: Use Cases and Applications As said those are 40 minutes slot so they are good for up to 2 presenters and you can certainly include a live demo (much appreciated by the OS community). Looking forward to hear from you. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Invitation: IOvisor TSC & Dev Members call @ Every 2 weeks from 11am to 12pm on Wednesday (pmonclus@plumgrid.com)
Pere Monclus
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updated Invitation: IOvisor TSC & Dev Members call @ Wed Oct 14, 2015 11am - 12pm (pmonclus@plumgrid.com)
Pere Monclus
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updated Invitation: IOvisor TSC & Dev Members call @ Wed Oct 14, 2015 11am - 12pm (pmonclus@plumgrid.com)
Pere Monclus
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updated Invitation: IOvisor TSC & Dev Members call @ Wed Oct 14, 2015 11am - 12pm (pmonclus@plumgrid.com)
Pere Monclus
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Updated Invitation: IOvisor TSC & Dev Members call @ Wed Oct 14, 2015 11am - 12pm (pmonclus@plumgrid.com)
Pere Monclus
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: Invitation: IOvisor TSC & Dev Members call @ Every 2 weeks from 11am to 12pm on Wednesday (pmonclus@plumgrid.com)
Uri Elzur
Hi
Assume more than few maybe at the OpenStack event. Can we skip this meeting?
Thx
Uri (“Oo-Ree”)
C: 949-378-7568
-----Original Appointment----- From: Pere Monclus [mailto:pmonclus@...]
Sent: Tuesday, August 25, 2015 6:45 PM To: Pere Monclus; John Zannos; Elzur, Uri; prem@...; Prasun Kapoor; Alexei Starovoitov; wardd@...; yunsong.lu@...; aclark@...; bkanekar@...; Ed Doe; Brenden Blanco; developer@...; mc3124@...; krb@...; christopher.price@...; Neela Jacques; Sushil Singh; jianwen.pi@...; Bhushan Kanekar Subject: Invitation: IOvisor TSC & Dev Members call @ Every 2 weeks from 11am to 12pm on Wednesday (pmonclus@...) When: Wednesday, October 28, 2015 11:00 AM-12:00 PM (UTC-08:00) Pacific Time (US & Canada). Where: google hangouts
<< File: invite.ics >>
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: Invitation: IOvisor TSC & Dev Members call @ Every 2 weeks from 11am to 12pm on Wednesday (pmonclus@plumgrid.com)
Pere Monclus
agree, I can't make it tomorrow either, but next week in Tokyo is a good place to meet. On Thu, Oct 22, 2015 at 6:25 PM, Elzur, Uri <uri.elzur@...> wrote:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
BPF dev env
O Mahony, Billy <billy.o.mahony@...>
Hi All,
is a Virtual Box VM a good environment to start compiling and investigating the various BPF *networking* examples at iovisor.github.com ? Regards, Billy. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Re: BPF dev env
Brenden Blanco <bblanco@...>
Hi Billy, Yes, a VBox will work just fine to develop in. I personally test in both physical and kvm environments. AWS also works. So far, most of the examples are self-contained on a single machine, running in namespaces, etc. Thanks, let us know if you have any troubles getting the environment working! -Brenden On Tue, Oct 27, 2015 at 3:00 AM, O Mahony, Billy via iovisor-dev <iovisor-dev@...> wrote: Hi All, |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||