For some reason the symlink did not work with the provided target, however placing it in the tensorflow site-packages worked properly
sudo ln -s /usr/local/cuda-11.1/targets/x86_64-linux/lib/libcusolver.so.11 /home/YourUsernameHere/.local/lib/python3.8/site-packages/tensorflow/python/libcusolver.so.10
(Obviously replacing python3.8 with proper value as well as username)
Edit: For those finding their way here, I recommend seeing the extra details provided by hoefling below (#43947 (comment))