For me VCS totally removed this way:
"You could disable all VCS-related plugins in Settings: File | Settings | Plugins."
(answer from this page: https://rider-support.jetbrains.com/hc/en-us/community/posts/115000709890-Turn-off-vcs-integration-by-default)