Is it possible to move/rename files in Git and maintain their history?

Question

I would like to rename/move a project subtree in Git moving it from

/project/xyz

to

/components/xyz

If I

Answer 1

Yes

1. You convert the commit history of files into email patches using git log --pretty=email
2. You reorganize these files in new directories and rename them
3. You convert back these files (emails) to Git commits to keep the history using git am.

Limitation

• Tags and branches are not kept
• History is cut on path file rename (directory rename)

---

Step by step explanation with examples

1. Extract history in email format

Example: Extract history of file3, file4 and file5

my_repo
├── dirA
│   ├── file1
│   └── file2
├── dirB            ^
│   ├── subdir      | To be moved
│   │   ├── file3   | with history
│   │   └── file4   | 
│   └── file5       v
└── dirC
    ├── file6
    └── file7

Set/clean the destination

export historydir=/tmp/mail/dir       # Absolute path
rm -rf "$historydir"    # Caution when cleaning the folder

Extract history of each file in email format

cd my_repo/dirB
find -name .git -prune -o -type d -o -exec bash -c 'mkdir -p "$historydir/${0%/*}" && git log --pretty=email -p --stat --reverse --full-index --binary -- "$0" > "$historydir/$0"' {} ';'

Unfortunately option --follow or --find-copies-harder cannot be combined with --reverse. This is why history is cut when file is renamed (or when a parent directory is renamed).

Temporary history in email format:

/tmp/mail/dir
    ├── subdir
    │   ├── file3
    │   └── file4
    └── file5

Dan Bonachea suggests to invert the loops of the git log generation command in this first step: rather than running git log once per file, run it exactly once with a list of files on the command line and generate a single unified log. This way commits that modify multiple files remain a single commit in the result, and all the new commits maintain their original relative order. Note this also requires changes in second step below when rewriting filenames in the (now unified) log.

---

2. Reorganize file tree and update filenames

Suppose you want to move these three files in this other repo (can be the same repo).

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB              # New tree
│   ├── dirB1         # from subdir
│   │   ├── file33    # from file3
│   │   └── file44    # from file4
│   └── dirB2         # new dir
│        └── file5    # from file5
└── dirH
    └── file77

Therefore reorganize your files:

cd /tmp/mail/dir
mkdir -p dirB/dirB1
mv subdir/file3 dirB/dirB1/file33
mv subdir/file4 dirB/dirB1/file44
mkdir -p dirB/dirB2
mv file5 dirB/dirB2

Your temporary history is now:

/tmp/mail/dir
    └── dirB
        ├── dirB1
        │   ├── file33
        │   └── file44
        └── dirB2
             └── file5

Change also filenames within the history:

cd "$historydir"
find * -type f -exec bash -c 'sed "/^diff --git a\|^--- a\|^+++ b/s:\( [ab]\)/[^ ]*:\1/$0:g" -i "$0"' {} ';'

---

3. Apply new history

Your other repo is:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
└── dirH
    └── file77

Apply commits from temporary history files:

cd my_other_repo
find "$historydir" -type f -exec cat {} + | git am --committer-date-is-author-date

--committer-date-is-author-date preserves the original commit time-stamps (Dan Bonachea's comment).

Your other repo is now:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB
│   ├── dirB1
│   │   ├── file33
│   │   └── file44
│   └── dirB2
│        └── file5
└── dirH
    └── file77

Use git status to see amount of commits ready to be pushed :-)

---

Extra trick: Check renamed/moved files within your repo

To list the files having been renamed:

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow {} ';' | grep '=>'

More customizations: You can complete the command git log using options --find-copies-harder or --reverse. You can also remove the first two columns using cut -f3- and grepping complete pattern '{.* => .*}'.

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow --find-copies-harder --reverse {} ';' | cut -f3- | grep '{.* => .*}'