I am pushing my Xcode project which written in Swift to GitHub. In this project I used GoogleMaps SDK for IOS and GoogleMaps framework integrated with few other frameworks which
GitHub has a limit of 100MB unless you pay for Git LFS. Sadly there is no way to get around this unless you ignore files using .gitignore. But those files will no longer be tracked by git.
EDIT
Forgot to mention for your specific case especially with a common framework such as Google Maps it is very easy to re download that at a later date using CocoaPods. Personally I would ignore that whole folder in .gitignore because after you pull down your repo you can always re download it easily.