argument confuses the linker or resets the search path for libraries.
Instead, use AC_PATH_XTRA and $X_LIBS.