Skip to content

JIT shim object drops GNU property notes (CET/BTI/PAC) from output binaries #149697

Open

Description

@stratakis

Bug report

Bug description:

When building with --enable-experimental-jit the jit_shim.o object lacks GNU property notes for hardware security features. The linker removes all such properties from the output binaries when any input object lacks them.

The regression was introduced with 9633c52 which makes the shim being compiled as a regular object file.

You can reproduce it by using ./configure --enable-experimental-jit=yes-off CFLAGS="-fcf-protection" && make -j.

Before the aforementioned commit a readelf -n python | grep 'x86 feature' would show Properties: x86 feature: IBT, SHSTK, whereas now nothing.

This is the case as well for BTI/PAC/GCS for aarch64 when building with #130864 and -mbranch-protection=standard.

My testing included GCC15.2.1 for the interpreter and Clang 21.1.8 for the jit, on Fedora 43.

CPython versions tested on:

3.16, CPython main branch, 3.15

Operating systems tested on:

Linux

Activity

  1. stratakis commented on May 11, 2026

    @stratakis
    ContributorAuthor
  2. stratakis commented on May 12, 2026

    @stratakis
    ContributorAuthor

    Digging into it more there are a couple more layers here beyond the property notes.

    The stencils need landing pad instructions (endbr64 on x86_64 / BTI or aarch64) on their entry points because JIT code is entered via indirect branches. When CET/BTI is active the CPU will reject the branch if the target of an indirect branch lacks a landing pad.

    This doesn't crash now with -fcf-protection (endbr64) or -mbranch-protection(bti), as the missing property note stops the kernel from enabling the hardware protections (if they are there). That is for x86_64, for aarch64 is the same but won't manifest till #130864 is resolved as well.

    So just fixing the property note of the shim would make the JIT crash :(

    For x86_64 if I pass -fcf-protection via CFLAGS_JIT (e.g. from extracting it via CFLAGS), things work fine, the stenctils get the endbr64 instruction and the shim the correct property note, all tests pass as well, but there is a cost of 4 bytes per stencil and one extra cycle per stencil at runtime (only the first stencil actually needs the instruction since it goes there from an indirect branch, for the rest the instruction becomes NOP).

    Now for aarch64, passing -mbranch-protection=standard/bti from CFLAGS_JIT fails with KeyError: '_JIT_ENTRY', the stencil assembly optimizer doesn't handle properly the .note.gnu.property and the .Lsec_end0 label that is added to the assembly by clang. Same for the .ARM.attributes section type.

    In summary, the x86_64 fix would be simple, but there may be a performance tradeoff, no idea how much.

    The aarch64 fix would be a lot more involved.

  3. stratakis commented on May 12, 2026

    @stratakis
    ContributorAuthor

    Fed it into Claude, Opus 4.6 and this came out:

    diff
    diff --git a/Tools/jit/_optimizers.py b/Tools/jit/_optimizers.py
    index f192783a559..ff104b2cdd2 100644
    --- a/Tools/jit/_optimizers.py
    +++ b/Tools/jit/_optimizers.py
    @@ -573,6 +573,18 @@ class OptimizerAArch64(Optimizer):  # pylint: disable = too-few-public-methods
         # https://developer.arm.com/documentation/ddi0602/2025-09/Base-Instructions/RET--Return-from-subroutine-
         _re_return = re.compile(r"\s*ret\b")
     
    +    def _preprocess(self, text: str) -> str:
    +        text = super()._preprocess(text)
    +        # Strip .note.gnu.property section and aeabi build attributes emitted
    +        # by clang with -mbranch-protection.  The note section contains a
    +        # .Lsec_end0: label that breaks the block structure of the optimizer.
    +        text = re.sub(r"^\s*\.aeabi_\w+.*\n", "", text, flags=re.MULTILINE)
    +        text = re.sub(
    +            r"^\s*\.section\s+\.note\.gnu\.property.*?^\.Lsec_end\d+:\n",
    +            "", text, flags=re.MULTILINE | re.DOTALL,
    +        )
    +        return text
    +
         supports_small_constants = True
         _re_small_const_1 = re.compile(
             r"\s*(?P<instruction>adrp)\s+(?P<register>x\d\d?),.*(?P<value>_JIT_OP(ARG|ERAND(0|1))_(16|32)).*"
    diff --git a/Tools/jit/_targets.py b/Tools/jit/_targets.py
    index ceee383ea68..3d7b2b5110a 100644
    --- a/Tools/jit/_targets.py
    +++ b/Tools/jit/_targets.py
    @@ -569,6 +569,7 @@ def _handle_section(
                     "SHT_NULL",
                     "SHT_STRTAB",
                     "SHT_SYMTAB",
    +                "Unknown",
                 }, section_type
     
         def _handle_relocation(
    diff --git a/configure b/configure
    index ecdd6095669..7c6c8f066d8 100755
    --- a/configure
    +++ b/configure
    @@ -11311,6 +11311,19 @@ then :
     
     else case e in #(
       e) as_fn_append CFLAGS_NODIST " $jit_flags"
    +                                                       if ${CFLAGS_JIT:+false} :
    +then :
    +
    +             for _cflag in $CFLAGS $CFLAGS_NODIST; do
    +               case $_cflag in #(
    +  -mbranch-protection=*|-fcf-protection|-fcf-protection=*) :
    +    as_fn_append CFLAGS_JIT " $_cflag" ;; #(
    +  *) :
    +     ;;
    +esac
    +             done
    +
    +fi
                REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\" --llvm-version=\"$LLVM_VERSION\" --llvm-tools-install-dir=\"$LLVM_TOOLS_INSTALL_DIR\""
                if test "x$Py_DEBUG" = xtrue
     then :
    diff --git a/configure.ac b/configure.ac
    index 9863afe5bba..dad035e6387 100644
    --- a/configure.ac
    +++ b/configure.ac
    @@ -2893,6 +2893,17 @@ AS_VAR_IF([jit_flags],
               [],
               [],
               [AS_VAR_APPEND([CFLAGS_NODIST], [" $jit_flags"])
    +           dnl If CFLAGS_JIT was not explicitly set, populate it with flags from
    +           dnl CFLAGS/CFLAGS_NODIST that affect GNU property notes.  The linker
    +           dnl uses AND-merge for these properties, so every object in the link
    +           dnl must carry them or they are dropped from the output binary.
    +           AS_VAR_IF([CFLAGS_JIT], [], [
    +             for _cflag in $CFLAGS $CFLAGS_NODIST; do
    +               AS_CASE([$_cflag],
    +                 [-mbranch-protection=*|-fcf-protection|-fcf-protection=*],
    +                 [AS_VAR_APPEND([CFLAGS_JIT], [" $_cflag"])])
    +             done
    +           ])
                AS_VAR_SET([REGEN_JIT_COMMAND],
                           ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\" --llvm-version=\"$LLVM_VERSION\" --llvm-tools-install-dir=\"$LLVM_TOOLS_INSTALL_DIR\""])
                AS_VAR_IF([Py_DEBUG],

    I don't understand the JIT infra code well enough to be confident on this code but I tested it on both x86_64 and aarch64 on top of #130864 with enabling the respective hardware protections and the issue is resolved, JIT works, hardware protections are enabled and the final binary/libpython contain the proper notes.

  4. hroncok commented on May 14, 2026

    @hroncok
    Contributor

    If CFLAGS_JIT was not explicitly set, populate it with flags from CFLAGS/CFLAGS_NODIST that affect GNU property notes...

    I suppose it might break the approach for folks who build JIT with CFLAGS_JIT set. Should it append the flags if CFLAGS_JIT is set instead?

  5. stratakis commented on May 14, 2026

    @stratakis
    ContributorAuthor

    I suppose it might break the approach for folks who build JIT with CFLAGS_JIT set. Should it append the flags if CFLAGS_JIT is set instead?

    True, appending these flags unconditionally there is the solution (unconditionally as in whatever CFLAGS_JIT is set to, still the flags would be appended only if they are found).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-JITtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions