Topics

[PATCH RFC] bpf: Symbol sizes and types in object file

Paul Chaignon
 

I am sending here as an RFC instead of LLVM's mailing list as I'm not sure
this is the intended behavior, and the "fix" may therefore not be needed.

Clang-compiled object files currently don't include the symbol sizes and
types. Some tools however need that information. For example, ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (working copy)
@@ -30,8 +30,8 @@
WeakRefDirective = "\t.weak\t";

UsesELFSectionDirectiveForBSS = true;
- HasSingleParameterDotFile = false;
- HasDotTypeDotSizeDirective = false;
+ HasSingleParameterDotFile = true;
+ HasDotTypeDotSizeDirective = true;

SupportsDebugInformation = true;
ExceptionsType = ExceptionHandling::DwarfCFI;

Alexei Starovoitov
 

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm not sure
this is the intended behavior, and the "fix" may therefore not be needed.

Clang-compiled object files currently don't include the symbol sizes and
types. Some tools however need that information. For example, ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (working copy)
@@ -30,8 +30,8 @@
WeakRefDirective = "\t.weak\t";

UsesELFSectionDirectiveForBSS = true;
- HasSingleParameterDotFile = false;
- HasDotTypeDotSizeDirective = false;
+ HasSingleParameterDotFile = true;
+ HasDotTypeDotSizeDirective = true;

SupportsDebugInformation = true;
ExceptionsType = ExceptionHandling::DwarfCFI;
make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm generated elf file?

Yonghong Song
 

On Mon, Sep 17, 2018 at 3:46 PM Alexei Starovoitov
<alexei.starovoitov@...> wrote:

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm not sure
this is the intended behavior, and the "fix" may therefore not be needed.

Clang-compiled object files currently don't include the symbol sizes and
types. Some tools however need that information. For example, ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (working copy)
@@ -30,8 +30,8 @@
WeakRefDirective = "\t.weak\t";

UsesELFSectionDirectiveForBSS = true;
- HasSingleParameterDotFile = false;
- HasDotTypeDotSizeDirective = false;
+ HasSingleParameterDotFile = true;
+ HasDotTypeDotSizeDirective = true;

SupportsDebugInformation = true;
ExceptionsType = ExceptionHandling::DwarfCFI;
make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm generated elf file?
Currently, BTF does not deal with symbols, so that is why it is okay.
llvm-objdump does not need symbol size either. It only needs symbol offset.
that is why we did not have issues so far.

In my future proposed func support in BTF, I also need offset only.

In your use case, the size is merely to satisfy conversion to CTF or there
are some other needs? What did you do with CTF?

Yutaro Hayakawa
 

Hello,

Paul, thank you for your help. I’m Yutaro a reporter of this issue.


On Mon, Sep 17, 2018 at 3:46 PM Alexei Starovoitov
<alexei.starovoitov@...> wrote:

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm not sure
this is the intended behavior, and the "fix" may therefore not be needed.

Clang-compiled object files currently don't include the symbol sizes and
types.  Some tools however need that information.  For example, ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (working copy)
@@ -30,8 +30,8 @@
    WeakRefDirective = "\t.weak\t";

    UsesELFSectionDirectiveForBSS = true;
-    HasSingleParameterDotFile = false;
-    HasDotTypeDotSizeDirective = false;
+    HasSingleParameterDotFile = true;
+    HasDotTypeDotSizeDirective = true;

    SupportsDebugInformation = true;
    ExceptionsType = ExceptionHandling::DwarfCFI;

make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm generated elf file?

Currently, BTF does not deal with symbols, so that is why it is okay.
llvm-objdump does not need symbol size either. It only needs symbol offset.
that is why we did not have issues so far.

In my future proposed func support in BTF, I also need offset only.

In your use case, the size is merely to satisfy conversion to CTF or there
are some other needs? What did you do with CTF?

Describing the background, I’m trying to port eBPF to FreeBSD (https://github.com/YutaroHayakawa/generic-ebpf)

What I want to do with CTF is very similar to that of BTF, but I currently focusing on assisting verification. Since FreeBSD
already has library to deal with CTF, we prefer to use CTF instead of BTF.

In FreeBSD, we usually use tool called ctfconvert (https://www.freebsd.org/cgi/man.cgi?query=ctfconvert&sektion=1&apropos=0&manpath=FreeBSD+10.0-RELEASE) to generate CTF.

And the problem is ctfconvert is aware of symbol type and size. It cannot generate correct CTF if they were missing.

To solve this issue I need the change.

Regards,
Yutaro

Paul Chaignon
 

On Wed, Sept 19, 2018 at 8:17 AM, Yonghong Song wrote:
On 9/18/18 12:18 AM, Yutaro Hayakawa wrote:
Hello,

Paul, thank you for your help. I’m Yutaro a reporter of this issue.


On Mon, Sep 17, 2018 at 3:46 PM Alexei Starovoitov
<alexei.starovoitov@... <mailto:alexei.starovoitov@...>>
wrote:

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm
not sure
this is the intended behavior, and the "fix" may therefore not be
needed.

Clang-compiled object files currently don't include the symbol sizes and
types.  Some tools however need that information.  For example,
ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...
<mailto:paul.chaignon@...>>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...
<mailto:yhayakawa3720@...>>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (working copy)
@@ -30,8 +30,8 @@
    WeakRefDirective = "\t.weak\t";

    UsesELFSectionDirectiveForBSS = true;
-    HasSingleParameterDotFile = false;
-    HasDotTypeDotSizeDirective = false;
+    HasSingleParameterDotFile = true;
+    HasDotTypeDotSizeDirective = true;

    SupportsDebugInformation = true;
    ExceptionsType = ExceptionHandling::DwarfCFI;
make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm
generated elf file?
Currently, BTF does not deal with symbols, so that is why it is okay.
llvm-objdump does not need symbol size either. It only needs symbol
offset.
that is why we did not have issues so far.

In my future proposed func support in BTF, I also need offset only.

In your use case, the size is merely to satisfy conversion to CTF or there
are some other needs? What did you do with CTF?
Describing the background, I’m trying to port eBPF to FreeBSD
(https://github.com/YutaroHayakawa/generic-ebpf)

What I want to do with CTF is very similar to that of BTF, but I
currently focusing on assisting verification. Since FreeBSD
already has library to deal with CTF, we prefer to use CTF instead of BTF.

In FreeBSD, we usually use tool called ctfconvert
(https://www.freebsd.org/cgi/man.cgi?query=ctfconvert&sektion=1&apropos=0&manpath=FreeBSD+10.0-RELEASE
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.freebsd.org_cgi_man.cgi-3Fquery-3Dctfconvert-26sektion-3D1-26apropos-3D0-26manpath-3DFreeBSD-2B10.0-2DRELEASE&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=iNlZa9ix8b8zAn4H53xuaTcSjVa5mlX0j4jNsEgF7gY&s=7c-EOY-jp0QTHfqm121Y9ODTslqckSpZBzQSrP7Qi6o&e=>)
to generate CTF.

And the problem is ctfconvert is aware of symbol type and size. It
cannot generate correct CTF if they were missing.

To solve this issue I need the change.
Yutaro/Paul,

Thanks for explanation. Now I understand your use case.
The change looks good to me. Please send me and Alexei a formal patch
through standard "git send-email" or just an email with
all necessary patch information, and I will apply
to llvm trunk.
Don't we need to cc llvm-commits? I can also send via Phabricator if
that's fine with you.


Yonghong


Regards,
Yutaro

Yonghong Song
 

On Wed, Sep 19, 2018 at 1:20 AM Paul Chaignon <paul.chaignon@...> wrote:

On Wed, Sept 19, 2018 at 8:17 AM, Yonghong Song wrote:
On 9/18/18 12:18 AM, Yutaro Hayakawa wrote:
Hello,

Paul, thank you for your help. I’m Yutaro a reporter of this issue.


On Mon, Sep 17, 2018 at 3:46 PM Alexei Starovoitov
<alexei.starovoitov@... <mailto:alexei.starovoitov@...>>
wrote:

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm
not sure
this is the intended behavior, and the "fix" may therefore not be
needed.

Clang-compiled object files currently don't include the symbol sizes and
types. Some tools however need that information. For example,
ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...
<mailto:paul.chaignon@...>>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...
<mailto:yhayakawa3720@...>>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h (working copy)
@@ -30,8 +30,8 @@
WeakRefDirective = "\t.weak\t";

UsesELFSectionDirectiveForBSS = true;
- HasSingleParameterDotFile = false;
- HasDotTypeDotSizeDirective = false;
+ HasSingleParameterDotFile = true;
+ HasDotTypeDotSizeDirective = true;

SupportsDebugInformation = true;
ExceptionsType = ExceptionHandling::DwarfCFI;
make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm
generated elf file?
Currently, BTF does not deal with symbols, so that is why it is okay.
llvm-objdump does not need symbol size either. It only needs symbol
offset.
that is why we did not have issues so far.

In my future proposed func support in BTF, I also need offset only.

In your use case, the size is merely to satisfy conversion to CTF or there
are some other needs? What did you do with CTF?
Describing the background, I’m trying to port eBPF to FreeBSD
(https://github.com/YutaroHayakawa/generic-ebpf)

What I want to do with CTF is very similar to that of BTF, but I
currently focusing on assisting verification. Since FreeBSD
already has library to deal with CTF, we prefer to use CTF instead of BTF.

In FreeBSD, we usually use tool called ctfconvert
(https://www.freebsd.org/cgi/man.cgi?query=ctfconvert&sektion=1&apropos=0&manpath=FreeBSD+10.0-RELEASE
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.freebsd.org_cgi_man.cgi-3Fquery-3Dctfconvert-26sektion-3D1-26apropos-3D0-26manpath-3DFreeBSD-2B10.0-2DRELEASE&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=iNlZa9ix8b8zAn4H53xuaTcSjVa5mlX0j4jNsEgF7gY&s=7c-EOY-jp0QTHfqm121Y9ODTslqckSpZBzQSrP7Qi6o&e=>)
to generate CTF.

And the problem is ctfconvert is aware of symbol type and size. It
cannot generate correct CTF if they were missing.

To solve this issue I need the change.
Yutaro/Paul,

Thanks for explanation. Now I understand your use case.
The change looks good to me. Please send me and Alexei a formal patch
through standard "git send-email" or just an email with
all necessary patch information, and I will apply
to llvm trunk.
Don't we need to cc llvm-commits? I can also send via Phabricator if
that's fine with you.
You can also cc llvm-commits.


Yonghong


Regards,
Yutaro

Yutaro Hayakawa
 

Hello Paul, Yonghong

Thank you very much for your help and understand.

Regards,
Yutaro


2018/09/19 23:08、Y Song <ys114321@...>のメール:

On Wed, Sep 19, 2018 at 1:20 AM Paul Chaignon <paul.chaignon@...> wrote:

On Wed, Sept 19, 2018 at 8:17 AM, Yonghong Song wrote:
On 9/18/18 12:18 AM, Yutaro Hayakawa wrote:
Hello,

Paul, thank you for your help. I’m Yutaro a reporter of this issue.


On Mon, Sep 17, 2018 at 3:46 PM Alexei Starovoitov
<alexei.starovoitov@... <mailto:alexei.starovoitov@...>>
wrote:

On Mon, Sep 17, 2018 at 11:29:13PM +0200, Paul Chaignon wrote:
I am sending here as an RFC instead of LLVM's mailing list as I'm
not sure
this is the intended behavior, and the "fix" may therefore not be
needed.

Clang-compiled object files currently don't include the symbol sizes and
types.  Some tools however need that information.  For example,
ctfconvert
uses that information to generate FreeBSD's CTF representation from ELF
files.
With this patch, that information is included in object files.

Signed-off-by: Paul Chaignon <paul.chaignon@...
<mailto:paul.chaignon@...>>
Reported-by: Yutaro Hayakawa <yhayakawa3720@...
<mailto:yhayakawa3720@...>>
---
Index: lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h
===================================================================
--- lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (revision 341679)
+++ lib/Target/BPF/MCTargetDesc/BPFMCAsmInfo.h        (working copy)
@@ -30,8 +30,8 @@
   WeakRefDirective = "\t.weak\t";

   UsesELFSectionDirectiveForBSS = true;
-    HasSingleParameterDotFile = false;
-    HasDotTypeDotSizeDirective = false;
+    HasSingleParameterDotFile = true;
+    HasDotTypeDotSizeDirective = true;

   SupportsDebugInformation = true;
   ExceptionsType = ExceptionHandling::DwarfCFI;

make sense to me, but I wonder why pahole's dwarf->btf converter
doesn't have this issue.
Could you describe what exactly are you trying to do with llvm
generated elf file?

Currently, BTF does not deal with symbols, so that is why it is okay.
llvm-objdump does not need symbol size either. It only needs symbol
offset.
that is why we did not have issues so far.

In my future proposed func support in BTF, I also need offset only.

In your use case, the size is merely to satisfy conversion to CTF or there
are some other needs? What did you do with CTF?

Describing the background, I’m trying to port eBPF to FreeBSD
(https://github.com/YutaroHayakawa/generic-ebpf)

What I want to do with CTF is very similar to that of BTF, but I
currently focusing on assisting verification. Since FreeBSD
already has library to deal with CTF, we prefer to use CTF instead of BTF.

In FreeBSD, we usually use tool called ctfconvert
(https://www.freebsd.org/cgi/man.cgi?query=ctfconvert&sektion=1&apropos=0&manpath=FreeBSD+10.0-RELEASE
<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.freebsd.org_cgi_man.cgi-3Fquery-3Dctfconvert-26sektion-3D1-26apropos-3D0-26manpath-3DFreeBSD-2B10.0-2DRELEASE&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=DA8e1B5r073vIqRrFz7MRA&m=iNlZa9ix8b8zAn4H53xuaTcSjVa5mlX0j4jNsEgF7gY&s=7c-EOY-jp0QTHfqm121Y9ODTslqckSpZBzQSrP7Qi6o&e=>)
to generate CTF.

And the problem is ctfconvert is aware of symbol type and size. It
cannot generate correct CTF if they were missing.

To solve this issue I need the change.

Yutaro/Paul,

Thanks for explanation. Now I understand your use case.
The change looks good to me. Please send me and Alexei a formal patch
through standard "git send-email" or just an email with
all necessary patch information, and I will apply
to llvm trunk.

Don't we need to cc llvm-commits?  I can also send via Phabricator if
that's fine with you.

You can also cc llvm-commits.


Yonghong


Regards,
Yutaro