Got the credentials for Service Account from Developer Console
First, I converted p12 private key to PEM:
openssl pkcs12 -in
Try to use BASE64URL instead of BASE64. JWT/JWS/JWE specifications use BASE64URL.
MIME::Base64 ---> MIME::Base64::URLSafe encode_base64 ---> urlsafe_b64encode