Methods for using Git with Google Colab

Question

Are there any recommended methods to integrate git with colab?

For example, is it possible to work off code from google source repositories or the likes?

Nei

Answer 1

You can almost use this link: https://qiita.com/Rowing0914/items/51a770925653c7c528f9

as a summary of the above link you should do this steps:

1- connect your google colab runtime to your Google Drive using this commands:

from google.colab import drive
drive.mount('/content/drive')

It would need a authentication process. Do whatever it needs.

2- Set current directory the path you want to clone the Git project there:

in my example:

path_clone = "drive/My Drive/projects"
!cd path_clone

don't forget to use ! in the beginning of cd command.

3- Clone the Git project:

!git clone <Git project URL address>

now you would have the cloned Git project in projects folder in you Google Drive (which is also connected to your Google Colab runtime machine)

4- Go to your Google Drive (using browser or etc) and then go to the "projects" folder and open the .ipynb file that you want to use in Google Colab.

5- Now you have Google Colab runtime with the .ipynb that you wanted to use which is also connected to your Google Drive and all cloned git files are in the Colab runtime's storage.

Note:

1- Check that your Colab runtime is connected to Google Drive. If it's not connected, just repeat the step #1 above.

2- Double check by using "pwd" and "cd" commands that the current directory is related to the cloned git project in google Drive (step #2 above).

Answer 2

The solution https://stackoverflow.com/a/53094151/3924118 did not work for me because the expression {user} was not being converted to the actual username (I was getting a 400 bad request), so I slightly changed that solution to the following one.

from getpass import getpass
import os

os.environ['USER'] = input('Enter the username of your Github account: ')
os.environ['PASSWORD'] = getpass('Enter the password of your Github account: ')
os.environ['REPOSITORY'] = input('Enter the name of the Github repository: ')
os.environ['GITHUB_AUTH'] = os.environ['USER'] + ':' + os.environ['PASSWORD']

!rm -rf $REPOSITORY # To remove the previous clone of the Github repository
!git clone https://$GITHUB_AUTH@github.com/$USER/$REPOSITORY.git

os.environ['USER'] = os.environ['PASSWORD'] = os.environ['REPOSITORY'] = os.environ['GITHUB_AUTH'] = ""

If you are able to clone your-repo, you should not see any password in the output of this command. If you get an error, the password could be displayed to the output, so make sure you do not share your notebook whenever this command fails.

Answer 3

Mount the drive using:

from google.colab import drive
drive.mount('/content/drive/')

Then:

%cd /content/drive/

To clone the repo in your drive

!git clone <github repo url> 

Access other files from the repo(example: helper.py is another file in repo):

import imp 
helper = imp.new_module('helper')
exec(open("drive/path/to/helper.py").read(), helper.__dict__)

Answer 4

If you want to clone a private repository, the quickest way would be:

!git clone https://username:password@github.com/username/repository.git

Answer 5

In order to protect your account username and password, you can use getPass and concatenate them in the shell command:

from getpass import getpass
import os

user = getpass('BitBucket user')
password = getpass('BitBucket password')
os.environ['BITBUCKET_AUTH'] = user + ':' + password

!git clone https://$BITBUCKET_AUTH@bitbucket.org/{user}/repository.git

Answer 6

I tried some of the methods here and they all worked well, but an issue I faced was, it became difficult to handle all the git commands and other related commands, for example version control with DVC, within notebook cells. So, I turned to this nice solution, Kora. It is a terminal emulator that can be run with in colab. This gives the ease of usage very similar to a terminal in local machine. The notebook will be still alive and we can edit files and cells as usual. Since this console is temporary, no information is exposed. GitHub login and other commands can be run as usual.

Kora: https://pypi.org/project/kora/

Usage:

!pip install kora

from kora import console
console.start()