I use LD_LIBRARY_PATH
to set the path of a certain user library for an application. But if I set capabilities on this application
sudo setcap CA
An alternative to consider is to "correct" a poorly compiled ELF shared library and/or executable using patchelf to set the rpath. https://nixos.org/patchelf.html
ld.so.conf is not always the sure bet. It will work if whatever you are running was compiled properly. In my case, with a particular specially packaged vendor's apache product, it was compiled so poorly: They did not even use unique .so filenames so they conflicted with .so filenames from RPMs in the base RHEL repositories that provided some pretty critical commonly used libraries. So this was the only option to isolate how they were used. Using ld.so.conf against those shared objects in the vendor's lib path would have blown away a lot of stuff, that included yum, along with glibc shared library failures, system-wide.