Setting the background to "@null"
will make the button have no effect when clicked. This will be a better choice.
style="?android:attr/borderlessButtonStyle"
Later I found that using
android:background="?android:attr/selectableItemBackground"
is also a good solution. And you can inherit this attribute in your own style.