Re: CPU Concurrency Issues


Yonghong Song
 

On Tue, Aug 20, 2019 at 12:05 PM Arnaldo Carvalho de Melo
<arnaldo.melo@...> wrote:

Em Tue, Aug 20, 2019 at 11:22:50AM -0700, Yonghong Song escreveu:
On Tue, Aug 20, 2019 at 11:02 AM Matthew Ahrens <mahrens@...> wrote:
On Tue, Aug 20, 2019 at 10:47 AM Yonghong Song <ys114321@...> wrote:
Also we have per cpu counter to prevent when bpf program interrupted then another bpf to run for tracing programs.
I think that means that if an interrupt fires while the bpf program is run, the interrupt will run, but if the interrupt causes another tracing event to fire, the associated bpf program will not run (i.e. the event will be ignored / dropped). Is that right?
Yes. See
https://github.com/torvalds/linux/blob/master/kernel/trace/bpf_trace.c#L88-L97
Yonghong, I thought there would be some counter to at least let users
know that drop happened, has this ever surfaced? Or is there some way to
know about those drops that I'm missing?
You did not miss anything. Currently, there are no counters to count those
drops due to nmi or due to bpf program already running on that cpu.

There is effort by Daniel Xu to expose nhit/nmisses counters
from k/uprobe trace infra. Even kprobe is not a miss, bpf program may not
fire due to the above reasons.
https://lore.kernel.org/bpf/20190820214819.16154-1-dxu@dxuuu.xyz/T/#t

debugfs has k/uprobe_profile to count nhit/nmisses from k/uprobe trace infra.

We could add a counter into trace_event_call->event to count hit/miss. The hit
can also be counted by bpf program itself. The "miss" should be rare, and
most bpf programs e.g. in bcc are designed to tolerate occasional probe miss,
which should not affect much on the final aggregation results.

How strongly do you feel such a bpf prog hit/miss counter for tracing programs
is needed?


- Arnaldo

Join iovisor-dev@lists.iovisor.org to automatically receive all group messages.