Disable AVX-optimized functions in glibc (LD_HWCAP_MASK, /etc/ld.so.nohwcap) for valgrind & gdb record

Question

Modern x86_64 linux with glibc will detect that CPU has support of AVX extension and will switch many string functions from generic implementation to AVX-optimized version (

Answer 1

It looks like there is a nice workaround for this implemented in recent versions of glibc: a "tunables" feature that guides selection of optimized string functions. You can find a general overview of this feature here and the relevant code inside glibc in ifunc-impl-list.c.

Here's how I figured it out. First, I took the address being complained about by gdb:

Process record does not support instruction 0xc5 at address 0x7ffff75c65d4.

I then looked it up in the table of shared libraries:

(gdb) info shared
From                To                  Syms Read   Shared Object Library
0x00007ffff7fd3090  0x00007ffff7ff3130  Yes         /lib64/ld-linux-x86-64.so.2
0x00007ffff76366b0  0x00007ffff766b52e  Yes         /usr/lib/x86_64-linux-gnu/libubsan.so.1
0x00007ffff746a320  0x00007ffff75d9cab  Yes         /lib/x86_64-linux-gnu/libc.so.6
...

You can see that this address is within glibc. But what function, specifically?

(gdb) disassemble 0x7ffff75c65d4
Dump of assembler code for function __strcmp_avx2:
   0x00007ffff75c65d0 <+0>:     mov    %edi,%eax
   0x00007ffff75c65d2 <+2>:     xor    %edx,%edx
=> 0x00007ffff75c65d4 <+4>:     vpxor  %ymm7,%ymm7,%ymm7

I can look in ifunc-impl-list.c to find the code that controls selecting the avx2 version:

  IFUNC_IMPL (i, name, strcmp,
          IFUNC_IMPL_ADD (array, i, strcmp,
                  HAS_ARCH_FEATURE (AVX2_Usable),
                  __strcmp_avx2)
          IFUNC_IMPL_ADD (array, i, strcmp, HAS_CPU_FEATURE (SSE4_2),
                  __strcmp_sse42)
          IFUNC_IMPL_ADD (array, i, strcmp, HAS_CPU_FEATURE (SSSE3),
                  __strcmp_ssse3)
          IFUNC_IMPL_ADD (array, i, strcmp, 1, __strcmp_sse2_unaligned)
          IFUNC_IMPL_ADD (array, i, strcmp, 1, __strcmp_sse2))

It looks like AVX2_Usable is the feature to disable. Let's rerun gdb accordingly:

GLIBC_TUNABLES=glibc.cpu.hwcaps=-AVX2_Usable gdb...

On this iteration it complained about __memmove_avx_unaligned_erms, which appeared to be enabled by AVX_Usable - but I found another path in ifunc-memmove.h enabled by AVX_Fast_Unaligned_Load. Back to the drawing board:

GLIBC_TUNABLES=glibc.cpu.hwcaps=-AVX2_Usable,-AVX_Fast_Unaligned_Load gdb ...

On this final round I discovered a rdtscp instruction in the ASAN shared library, so I recompiled without the address sanitizer and at last, it worked.

In summary: with some work it's possible to disable these instructions from the command line and use gdb's record feature without severe hacks.