minutes: IO Visor TSC and Dev Members Call

Brenden Blanco <bblanco@...>

Hi All,

Below, please find the meeting minutes from today.

Meeting devoted to XDP discussion.

With Alexei's help, I have been working on furthering support for BPF in drivers.

Added TX support to mlx4 driver
 - packet-in + IP rewrite + stat increment + packet-out => 10 Mpps
 - packet-in + stat increment + packet-out => 12.9 Mpps

The approach steals $NUMCPU tx queues from the driver and allows rx thread to write to them.
Hacky, more work to be done.
No skb alloc is needed.
Tx completion is handled in the normal way (napi), but operating on page instead of skb.

Jesper noted that further improvements to the page allocator should benefit this approach directly.

Was able to use direct packet access patch from Alexei, improves performance.
Enabled packet write as well.

Try at your own risk.


We also discussed thoughts on future API for XDP

Break XDP api into two pieces
== generic part - program + api
 - input and output well defined, OS independent, SW/HW independent
 - in here goes the use case, e.g. router, load balancer
 - for instance, the presence of ifindex in one of these programs would violate OS independence

== non-generic - program + specific implementation
 - use case - mlx has fancy qos, create bpf program for translating hw requirements into api that generic program can consume (bpf defined map? tlv metadata struct?)
 - generic prog tail calls into non-generic
 - need to avoid: creating hw specific helper functions - these become new ABI

== other thoughts
 - want to create something that is not kernel abi
 - want to exploit hw specific features
 - can access driver memory directly?
 - used to configure driver via map?
 - avoid vendor lock-in through implementation, encourage vendor competition on features

repository (see iomodules)
firmware compiler?: frontend and vendor specific backend

how to jit generic side of programs into hw/firmware?
 complex (llvm?) - may not live in kernel

Is bpf to hw like switchdev (network offload) or more like hw provisioning?
 - consensus seemed to prefer hw provisioning

Tom Herbert
Alexei Starovoitov
Keith Burns
Ari Saha
Brendan Gregg
Daniel Borkmann
Jesper Brouer
Brenden Blanco
ChengChun Tu
Prem Jonnalagadda
John Fastabend