Skip to content

Commit

Permalink
Merge tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel…
Browse files Browse the repository at this point in the history 
…/git/bpf/bpf

Daniel Borkmann says:

====================
pull-request: bpf 2023-08-31

We've added 15 non-merge commits during the last 3 day(s) which contain
a total of 17 files changed, 468 insertions(+), 97 deletions(-).

The main changes are:

1) BPF selftest fixes: one flake and one related to clang18 testing,
   from Yonghong Song.

2) Fix a d_path BPF selftest failure after fast-forward from Linus'
   tree, from Jiri Olsa.

3) Fix a preempt_rt splat in sockmap when using raw_spin_lock_t,
   from John Fastabend.

4) Fix a xsk_diag_fill use-after-free race during socket cleanup,
   from Magnus Karlsson.

5) Fix xsk_build_skb to address a buggy dereference of an ERR_PTR(),
   from Tirthendu Sarkar.

6) Fix a bpftool build warning when compiled with -Wtype-limits,
   from Yafang Shao.

7) Several misc fixes and cleanups in standardization docs,
   from David Vernet.

8) Fix BPF selftest install to consider no_alu32/cpuv4/bpf-gcc flavors,
   from Björn Töpel.

9) Annotate a data race in bpf_long_memcpy for KCSAN, from Daniel Borkmann.

10) Extend documentation with a description for CO-RE relocations,
    from Eduard Zingerman.

11) Fix several invalid escape sequence warnings in bpf_doc.py script,
    from Vishal Chourasia.

12) Fix the instruction set doc wrt offset of BPF-to-BPF call,
    from Will Hawkins.

* tag 'for-netdev' of https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
  selftests/bpf: Include build flavors for install target
  bpf: Annotate bpf_long_memcpy with data_race
  selftests/bpf: Fix d_path test
  bpf, docs: Fix invalid escape sequence warnings in bpf_doc.py
  xsk: Fix xsk_diag use-after-free error during socket cleanup
  bpf, docs: s/eBPF/BPF in standards documents
  bpf, docs: Add abi.rst document to standardization subdirectory
  bpf, docs: Move linux-notes.rst to root bpf docs tree
  bpf, sockmap: Fix preempt_rt splat when using raw_spin_lock_t
  docs/bpf: Add description for CO-RE relocations
  bpf, docs: Correct source of offset for program-local call
  selftests/bpf: Fix flaky cgroup_iter_sleepable subtest
  xsk: Fix xsk_build_skb() error: 'skb' dereferencing possible ERR_PTR()
  bpftool: Fix build warnings with -Wtype-limits
  bpf: Prevent inlining of bpf_fentry_test7()
====================

Link: https://lore.kernel.org/r/20230831210019.14417-1-daniel@iogearbox.net
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
  • Loading branch information
Jakub Kicinski committed Sep 1, 2023
2 parents 8aae762 + be8e754 commit ddaa935
Show file tree
Hide file tree
Showing 17 changed files with 468 additions and 97 deletions.
31 changes: 25 additions & 6 deletions Documentation/bpf/btf.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -726,8 +726,8 @@ same as the one describe in :ref:`BTF_Type_String`.
4.2 .BTF.ext section
--------------------

The .BTF.ext section encodes func_info and line_info which needs loader
manipulation before loading into the kernel.
The .BTF.ext section encodes func_info, line_info and CO-RE relocations
which needs loader manipulation before loading into the kernel.

The specification for .BTF.ext section is defined at ``tools/lib/bpf/btf.h``
and ``tools/lib/bpf/btf.c``.
Expand All @@ -745,15 +745,20 @@ The current header of .BTF.ext section::
__u32 func_info_len;
__u32 line_info_off;
__u32 line_info_len;

/* optional part of .BTF.ext header */
__u32 core_relo_off;
__u32 core_relo_len;
};

It is very similar to .BTF section. Instead of type/string section, it
contains func_info and line_info section. See :ref:`BPF_Prog_Load` for details
about func_info and line_info record format.
contains func_info, line_info and core_relo sub-sections.
See :ref:`BPF_Prog_Load` for details about func_info and line_info
record format.

The func_info is organized as below.::

func_info_rec_size
func_info_rec_size /* __u32 value */
btf_ext_info_sec for section #1 /* func_info for section #1 */
btf_ext_info_sec for section #2 /* func_info for section #2 */
...
Expand All @@ -773,7 +778,7 @@ Here, num_info must be greater than 0.

The line_info is organized as below.::

line_info_rec_size
line_info_rec_size /* __u32 value */
btf_ext_info_sec for section #1 /* line_info for section #1 */
btf_ext_info_sec for section #2 /* line_info for section #2 */
...
Expand All @@ -787,6 +792,20 @@ kernel API, the ``insn_off`` is the instruction offset in the unit of ``struct
bpf_insn``. For ELF API, the ``insn_off`` is the byte offset from the
beginning of section (``btf_ext_info_sec->sec_name_off``).

The core_relo is organized as below.::

core_relo_rec_size /* __u32 value */
btf_ext_info_sec for section #1 /* core_relo for section #1 */
btf_ext_info_sec for section #2 /* core_relo for section #2 */

``core_relo_rec_size`` specifies the size of ``bpf_core_relo``
structure when .BTF.ext is generated. All ``bpf_core_relo`` structures
within a single ``btf_ext_info_sec`` describe relocations applied to
section named by ``btf_ext_info_sec->sec_name_off``.

See :ref:`Documentation/bpf/llvm_reloc <btf-co-re-relocations>`
for more information on CO-RE relocations.

4.2 .BTF_ids section
--------------------

Expand Down
1 change: 1 addition & 0 deletions Documentation/bpf/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ that goes into great technical depth about the BPF Architecture.
bpf_licensing
test_debug
clang-notes
linux-notes
other
redirect

Expand Down
0 ...ation/bpf/standardization/linux-notes.rst → Documentation/bpf/linux-notes.rst
View file
File renamed without changes.
304 changes: 304 additions & 0 deletions Documentation/bpf/llvm_reloc.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -240,3 +240,307 @@ The .BTF/.BTF.ext sections has R_BPF_64_NODYLD32 relocations::
Offset Info Type Symbol's Value Symbol's Name
000000000000002c 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text
0000000000000040 0000000200000004 R_BPF_64_NODYLD32 0000000000000000 .text

.. _btf-co-re-relocations:

=================
CO-RE Relocations
=================

From object file point of view CO-RE mechanism is implemented as a set
of CO-RE specific relocation records. These relocation records are not
related to ELF relocations and are encoded in .BTF.ext section.
See :ref:`Documentation/bpf/btf <BTF_Ext_Section>` for more
information on .BTF.ext structure.

CO-RE relocations are applied to BPF instructions to update immediate
or offset fields of the instruction at load time with information
relevant for target kernel.

Field to patch is selected basing on the instruction class:

* For BPF_ALU, BPF_ALU64, BPF_LD `immediate` field is patched;
* For BPF_LDX, BPF_STX, BPF_ST `offset` field is patched;
* BPF_JMP, BPF_JMP32 instructions **should not** be patched.

Relocation kinds
================

There are several kinds of CO-RE relocations that could be split in
three groups:

* Field-based - patch instruction with field related information, e.g.
change offset field of the BPF_LDX instruction to reflect offset
of a specific structure field in the target kernel.

* Type-based - patch instruction with type related information, e.g.
change immediate field of the BPF_ALU move instruction to 0 or 1 to
reflect if specific type is present in the target kernel.

* Enum-based - patch instruction with enum related information, e.g.
change immediate field of the BPF_LD_IMM64 instruction to reflect
value of a specific enum literal in the target kernel.

The complete list of relocation kinds is represented by the following enum:

.. code-block:: c
enum bpf_core_relo_kind {
BPF_CORE_FIELD_BYTE_OFFSET = 0, /* field byte offset */
BPF_CORE_FIELD_BYTE_SIZE = 1, /* field size in bytes */
BPF_CORE_FIELD_EXISTS = 2, /* field existence in target kernel */
BPF_CORE_FIELD_SIGNED = 3, /* field signedness (0 - unsigned, 1 - signed) */
BPF_CORE_FIELD_LSHIFT_U64 = 4, /* bitfield-specific left bitshift */
BPF_CORE_FIELD_RSHIFT_U64 = 5, /* bitfield-specific right bitshift */
BPF_CORE_TYPE_ID_LOCAL = 6, /* type ID in local BPF object */
BPF_CORE_TYPE_ID_TARGET = 7, /* type ID in target kernel */
BPF_CORE_TYPE_EXISTS = 8, /* type existence in target kernel */
BPF_CORE_TYPE_SIZE = 9, /* type size in bytes */
BPF_CORE_ENUMVAL_EXISTS = 10, /* enum value existence in target kernel */
BPF_CORE_ENUMVAL_VALUE = 11, /* enum value integer value */
BPF_CORE_TYPE_MATCHES = 12, /* type match in target kernel */
};
Notes:

* ``BPF_CORE_FIELD_LSHIFT_U64`` and ``BPF_CORE_FIELD_RSHIFT_U64`` are
supposed to be used to read bitfield values using the following
algorithm:

.. code-block:: c
// To read bitfield ``f`` from ``struct s``
is_signed = relo(s->f, BPF_CORE_FIELD_SIGNED)
off = relo(s->f, BPF_CORE_FIELD_BYTE_OFFSET)
sz = relo(s->f, BPF_CORE_FIELD_BYTE_SIZE)
l = relo(s->f, BPF_CORE_FIELD_LSHIFT_U64)
r = relo(s->f, BPF_CORE_FIELD_RSHIFT_U64)
// define ``v`` as signed or unsigned integer of size ``sz``
v = *({s|u}<sz> *)((void *)s + off)
v <<= l
v >>= r
* The ``BPF_CORE_TYPE_MATCHES`` queries matching relation, defined as
follows:

* for integers: types match if size and signedness match;
* for arrays & pointers: target types are recursively matched;
* for structs & unions:

* local members need to exist in target with the same name;

* for each member we recursively check match unless it is already behind a
pointer, in which case we only check matching names and compatible kind;

* for enums:

* local variants have to have a match in target by symbolic name (but not
numeric value);

* size has to match (but enum may match enum64 and vice versa);

* for function pointers:

* number and position of arguments in local type has to match target;
* for each argument and the return value we recursively check match.

CO-RE Relocation Record
=======================

Relocation record is encoded as the following structure:

.. code-block:: c
struct bpf_core_relo {
__u32 insn_off;
__u32 type_id;
__u32 access_str_off;
enum bpf_core_relo_kind kind;
};
* ``insn_off`` - instruction offset (in bytes) within a code section
associated with this relocation;

* ``type_id`` - BTF type ID of the "root" (containing) entity of a
relocatable type or field;

* ``access_str_off`` - offset into corresponding .BTF string section.
String interpretation depends on specific relocation kind:

* for field-based relocations, string encodes an accessed field using
a sequence of field and array indices, separated by colon (:). It's
conceptually very close to LLVM's `getelementptr <GEP_>`_ instruction's
arguments for identifying offset to a field. For example, consider the
following C code:

.. code-block:: c
struct sample {
int a;
int b;
struct { int c[10]; };
} __attribute__((preserve_access_index));
struct sample *s;
* Access to ``s[0].a`` would be encoded as ``0:0``:

* ``0``: first element of ``s`` (as if ``s`` is an array);
* ``0``: index of field ``a`` in ``struct sample``.

* Access to ``s->a`` would be encoded as ``0:0`` as well.
* Access to ``s->b`` would be encoded as ``0:1``:

* ``0``: first element of ``s``;
* ``1``: index of field ``b`` in ``struct sample``.

* Access to ``s[1].c[5]`` would be encoded as ``1:2:0:5``:

* ``1``: second element of ``s``;
* ``2``: index of anonymous structure field in ``struct sample``;
* ``0``: index of field ``c`` in anonymous structure;
* ``5``: access to array element #5.

* for type-based relocations, string is expected to be just "0";

* for enum value-based relocations, string contains an index of enum
value within its enum type;

* ``kind`` - one of ``enum bpf_core_relo_kind``.

.. _GEP: https://llvm.org/docs/LangRef.html#getelementptr-instruction

.. _btf_co_re_relocation_examples:

CO-RE Relocation Examples
=========================

For the following C code:

.. code-block:: c
struct foo {
int a;
int b;
unsigned c:15;
} __attribute__((preserve_access_index));
enum bar { U, V };
With the following BTF definitions:

.. code-block::
...
[2] STRUCT 'foo' size=8 vlen=2
'a' type_id=3 bits_offset=0
'b' type_id=3 bits_offset=32
'c' type_id=4 bits_offset=64 bitfield_size=15
[3] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[4] INT 'unsigned int' size=4 bits_offset=0 nr_bits=32 encoding=(none)
...
[16] ENUM 'bar' encoding=UNSIGNED size=4 vlen=2
'U' val=0
'V' val=1
Field offset relocations are generated automatically when
``__attribute__((preserve_access_index))`` is used, for example:

.. code-block:: c
void alpha(struct foo *s, volatile unsigned long *g) {
*g = s->a;
s->a = 1;
}
00 <alpha>:
0: r3 = *(s32 *)(r1 + 0x0)
00: CO-RE <byte_off> [2] struct foo::a (0:0)
1: *(u64 *)(r2 + 0x0) = r3
2: *(u32 *)(r1 + 0x0) = 0x1
10: CO-RE <byte_off> [2] struct foo::a (0:0)
3: exit
All relocation kinds could be requested via built-in functions.
E.g. field-based relocations:

.. code-block:: c
void bravo(struct foo *s, volatile unsigned long *g) {
*g = __builtin_preserve_field_info(s->b, 0 /* field byte offset */);
*g = __builtin_preserve_field_info(s->b, 1 /* field byte size */);
*g = __builtin_preserve_field_info(s->b, 2 /* field existence */);
*g = __builtin_preserve_field_info(s->b, 3 /* field signedness */);
*g = __builtin_preserve_field_info(s->c, 4 /* bitfield left shift */);
*g = __builtin_preserve_field_info(s->c, 5 /* bitfield right shift */);
}
20 <bravo>:
4: r1 = 0x4
20: CO-RE <byte_off> [2] struct foo::b (0:1)
5: *(u64 *)(r2 + 0x0) = r1
6: r1 = 0x4
30: CO-RE <byte_sz> [2] struct foo::b (0:1)
7: *(u64 *)(r2 + 0x0) = r1
8: r1 = 0x1
40: CO-RE <field_exists> [2] struct foo::b (0:1)
9: *(u64 *)(r2 + 0x0) = r1
10: r1 = 0x1
50: CO-RE <signed> [2] struct foo::b (0:1)
11: *(u64 *)(r2 + 0x0) = r1
12: r1 = 0x31
60: CO-RE <lshift_u64> [2] struct foo::c (0:2)
13: *(u64 *)(r2 + 0x0) = r1
14: r1 = 0x31
70: CO-RE <rshift_u64> [2] struct foo::c (0:2)
15: *(u64 *)(r2 + 0x0) = r1
16: exit
Type-based relocations:

.. code-block:: c
void charlie(struct foo *s, volatile unsigned long *g) {
*g = __builtin_preserve_type_info(*s, 0 /* type existence */);
*g = __builtin_preserve_type_info(*s, 1 /* type size */);
*g = __builtin_preserve_type_info(*s, 2 /* type matches */);
*g = __builtin_btf_type_id(*s, 0 /* type id in this object file */);
*g = __builtin_btf_type_id(*s, 1 /* type id in target kernel */);
}
88 <charlie>:
17: r1 = 0x1
88: CO-RE <type_exists> [2] struct foo
18: *(u64 *)(r2 + 0x0) = r1
19: r1 = 0xc
98: CO-RE <type_size> [2] struct foo
20: *(u64 *)(r2 + 0x0) = r1
21: r1 = 0x1
a8: CO-RE <type_matches> [2] struct foo
22: *(u64 *)(r2 + 0x0) = r1
23: r1 = 0x2 ll
b8: CO-RE <local_type_id> [2] struct foo
25: *(u64 *)(r2 + 0x0) = r1
26: r1 = 0x2 ll
d0: CO-RE <target_type_id> [2] struct foo
28: *(u64 *)(r2 + 0x0) = r1
29: exit
Enum-based relocations:

.. code-block:: c
void delta(struct foo *s, volatile unsigned long *g) {
*g = __builtin_preserve_enum_value(*(enum bar *)U, 0 /* enum literal existence */);
*g = __builtin_preserve_enum_value(*(enum bar *)V, 1 /* enum literal value */);
}
f0 <delta>:
30: r1 = 0x1 ll
f0: CO-RE <enumval_exists> [16] enum bar::U = 0
32: *(u64 *)(r2 + 0x0) = r1
33: r1 = 0x1 ll
108: CO-RE <enumval_value> [16] enum bar::V = 1
35: *(u64 *)(r2 + 0x0) = r1
36: exit
Loading

0 comments on commit ddaa935

Please sign in to comment.