I want to split a directory from a large Subversion repository to a repository of its own, and keep the history of the files in that directory.
I tried the regular way of doing it first
svnadmin dump /path/to/repo > largerepo.dump
cat largerepo.dump | svndumpfilter include my/directory >mydir.dump
but that does not work, since the directory has been moved and copied over the years and files have been moved into and out of it to other parts of the repository. The result is a lot of these:
svndumpfilter: Invalid copy source path '/some/old/path'
Next thing I tried is to include those /some/old/path as they appear and after a long, long list of files and directories included, the svndumpfilter completes, BUT importing the resulting dump isn't producing the same files as the current directory has.
So, how do I properly split the directory from that repository while keeping the history?
EDIT: I specifically want trunk/myproj to be the trunk in a new repository PLUS have the new repository include none of the other old stuff, ie. there should not be possibility for anyone to update to old revision before the split and get/see the files.
The svndumpfilter solution I tried would achieve exactly that, sadly its not doable since the path/files have been moved around. The solution by ng isn't accetable since its basically a clone+removal of extras which keeps ALL the history, not just relevant myproj history.
This problem occurs when one of the directories/files included by svndumpfilter originally was copied or moved from a section of the tree that is not being included.
To solve the problem use this script: svndumpfilter3
I had a similar problem splitting a repository ..
svndumpfilter: Invalid copy source path /dir/old_dir
What I did to get around the problem was to include the additional old directories that is was requesting, or that you know you moved. In my case I had moved 3 directories into another directory.
eg. Moved Folders A,B,C in to Folder D
cat project.dump | svndumpfilter include A B C D > new.dump
This seemed to solve my problem. I was able to separate Folder D from the rest of the Repo. On the flip-side, when excluding D I did not get the error, I would guess because removing D didn't require the links/history to A,B,C
I've tried at least 4 different applications to do that, the only that really worked was using svndumpfilterIN :
cd /usr/local/bin/
sudo wget --no-check-certificate https://raw.github.com/jasperlee108/svndumpfilterIN/master/svndumpfilter.py
sudo chmod +x svndumpfilter.py
# To be sure nothing will happened on the original repo :
cp -au /path/to/repo /tmp/largerepo.repo/
svnadmin dump /path/to/repo > /tmp/largerepo.dump
svndumpfilter.py /tmp/largerepo.dump --repo=/tmp/largerepo.repo --output-dump=/tmp/mydir.dump include my/directory
Here is what I tried and didn't worked :
- http://furius.ca/pubcode/pub/conf/bin/svndumpfilter3.html (from previous answer)
- Including the problematic path as said in another previous answer. Problem : I still have paths I do not want in the target dump
- svndumpsanitizer (I still have paths I do not want in the target dump also)
This could potentially help you: Quote from http://svnbook.red-bean.com/en/1.5/svn.reposadmin.maint.html#svn.reposadmin.maint.replication
In Subversion 1.5, svnsync grew the ability to also mirror a subset of a repository rather than the whole thing. The process of setting up and maintaining such a mirror is exactly the same as when mirroring a whole repository, except that instead of specifying the source repository's root URL when running svnsync init, you specify the URL of some subdirectory within that repository. Synchronization to that mirror will now copy only the bits that changed under that source repository subdirectory. There are some limitations to this support, though. First, you can't mirror multiple disjoint subdirectories of the source repository into a single mirror repository—you'd need to instead mirror some parent directory that is common to both. Second, the filtering logic is entirely path-based, so if the subdirectory you are mirroring was renamed at some point in the past, your mirror would contain only the revisions since the directory appeared at the URL you specified. And likewise, if the source subdirectory is renamed in the future, your synchronization processes will stop mirroring data at the point that the source URL you specified is no longer valid.
The Problem of course is losing the pre-rename history...
auriarte's link to svndumpfilter3 404's. Here's a working link (as of 2011-01-31), for anyone looking for it: http://furius.ca/pubcode/pub/conf/bin/svndumpfilter3.html
I've just successfully migrated a project from an existing combined repo (at Google Code) to its own repo. The posts here were very helpful.
This is what finally worked for me...
- Used svnsync to make a local mirror of my Google Code repo following the directions here.
svnadmin dump to foo-dumpfilecat dumpfile | ./svndumpfilter3 --untangle mymirrorrepo trunk/foo > foo-dumpfilesvnadmin create fooreposvnadmin load foorepo --ignore-uuid < foo-dumpfile
The --untangle option in step 3 managed to resolve all of the path problems that stumped svndumpfilter and svndumpfilter2.
Initially, at step 5 I was stuck on the error:
<<< Started new transaction, based on original revision 2
svnadmin: File not found: transaction '1-1', path 'trunk/foo'
But this post in Charles Calvert's blog explained that all that was required was to create the trunk dir in foorepo before doing the load.
I encountered this problem and ended up using svndumpfilter2.
Specifically, this command:
sudo svnadmin dump /home/setup/svn/repos/main_repl | sudo ./svndumpfilter2.py /home/setup/svn/repos/main_repl Development QA compliance > ~/main_repl_dump.trim
I did get the out of memory error mentioned, however, since I was running svn on a VM, I just bumped the memory up to 2G. While I realize that this may not be an option for everyone, I noticed that it ran much faster than it had with 512M. (2G probably wasn't necessary).
Currently, it is processing revision 18,631.
In case anyone wonders, the reason why I needed to break out part of the repo was because we were creating tags/copies for distribution to implementation of files in another path of the repo. For some reason, this process was causing the repo to balloon to huge proportions. (We're at 17G now.)
I'm doing this on a replication repo of SVN, version 1.5.6, on Debian Lenny, 5.0.4.
Why not replicate the entire repository, dump it in to a new one. Then branch out the trunk, delete the head and merge the portions you want back in to the trunk from the branch. Then you have kept the history and split out the parts you want to a new repository.
- Dump to /trunk
- Branch /trunk to /branches/trunk
- Delete /trunk
- Merge /branches/trunk/whatever back in to /trunk or /trunk/whatever
This way you have kept all the history, and selectively picked the parts you want.
I'm also looking for an answer on this question (having to deal with it myself). Based on Alex' answer, I found http://furius.ca/pubcode/pub/conf/common/bin/svndumpfilter3.html which claims to fix some of the svndumpfilter2 issues. I believe it is a partial solution.
The good:
A rewrite of Subversion's svndumpfilter in pure Python, that allows you to untangle move/copy operations between excluded and included sets of files/dirs, by converting them into additions. If you use this option, it fetches the original files from a given repository.
Concern:
Important
Some people have been reporting a bug with this script, that it will create an empty file on a large repository. It worked great for the split that I had to do on my repository, but I have no time to fix the problem that occurs for some other people's repositories
Stumbled onto this problem and found this tool svndumpsanitizer It seemed to work well I was able to import the file it created into a new repository.
The specific commands are as follows, I am going to assume the repository is hosted on a http(s):// server, although the same commands will work for svn:// or file://.
svnadmin dump /path/to/repository > dumpfile
svnadmin create /path/to/new_repository
svnadmin load /path/to/new_repository < dumpfile
svn co https://localhost/svn/new_repository_url new_repository_checkout
cd new_repository_checkout
svn move https://localhost/svn/new_repository_url/trunk https://localhost/svn/new_repository_url/branches/head -m "Moving HEAD to branches"
svn move https://localhost/svn/new_repository_url/branches/head/whatever https://localhost/svn/new_repository_url/trunk -m "Creating new trunk"
svn update
cd branches
svn remove head
svn commit
You should now have the part you want from the old repository as the trunk of the new one.
I see this is quite old now, but does adding "--skip-missing-merge-sources" help any? It seems like it might...
If you don't need the entire history you can pick it up from just after the error. If your error was at revision 412 then you can try picking it up right after with:
svnadmin dump /path/to/repo -r 413:HEAD > largerepo.dump
I realize this may not be a perfect solution either but it may be good enough in your case.
You may want to also just do this all in one step
svnadmin dump /path/to/repo -r 413:HEAD | svndumpfilter include my/directory > mydir.dump
Some more info about svndumpfilter and how to fix - http://blog.rlucas.net/uncategorized/some-gotchas-with-using-svndumpfilter/
Or you can try svndumpfilter replacement script, now called as svndumpfilter2 - http://cogo.wordpress.com/2009/03/10/problems-with-svndumpfilter/
I didn't tried that script, coz i need some time to make a repo backup, to test in on that (i have a backup dump to play with but on Windows, and it is a linux script).
just ran into this problem and wrote a little script to retry dumping until all invalid source paths are resolved.
#!/usr/bin/env ruby
require 'open3'
include Open3
paths = [ "/your/path" ]
command = ""
new_path = "xx"
while (! new_path.nil?)
lines = nil
popen3(" svndumpfilter include #{paths.join(' ')} > svn.result.dump < svn.original.dump") do |i, o, err|
i.close
puts "Processing, please wait ..."
lines = err.readlines
end
new_path = nil
lines.each do |line|
if line =~ /Invalid copy source path '(.*)'/
new_path = $1
end
end
puts "Adding #{new_path}"
paths << new_path
end
Based on the answer by ng., but with filtering and dropping empty revisions.
Step 1. Dump and filter:
svnadmin dump /path/to/repository > fulldumpfile
svndumpfilter include trunk/the/part/you/want --drop-empty-revs --renumber-revs < fulldumpfile > dumpfile
Step 2. Create new repo. (note that this can also be done e.g. with Tortoise SVN)
svnadmin create /path/to/new_repo
Remember to add whatever you need to be able to checkout (permissions and such).
Step 3. Checkout and add base folder (can also be done e.g. with Tortoise SVN)
svn checkout http://localhost/new_repo /some/checkout/path/newrepo
cd /some/checkout/path/newrepo
# to be able to create "trunk/the/part/you/want" you will need to add parent dir:
mkdir -p trunk/the/part/you
svn add trunk
svn commit -m "old base"
Step 4. Load filtered dump
svnadmin load /path/to/new_repo < dumpfile
Step 5. Move old root to new root (can also be done e.g. with Tortoise SVN)
cd /some/checkout/path/newrepo
svn update
svn move trunk/the/part/you/want/* trunk/
svn move tags/the/part/you/want/* tags/
svn move branches/the/part/you/want/* branches/
svn commit -m "re-structure base"
You should now have the part you want from the old repository as the trunk of the new one.
We developed Subdivision, a GUI tool designed to split svn repositories.
Subdivision analyzes the repository and calculates the history of the files as they are being copied and moved throughout the repository. Using this information, your selections are intelligently augmented to avoid all "Invalid copy source path" errors.
In addition to splitting a repository, Subdivision can be used to delete files from a repository as well as extract files and folders into a new repository.
Subdivision is free for small repositories.
来源：https://stackoverflow.com/questions/433276/how-to-split-an-svn-folder-into-its-own-repository-when-it-has-been-renamed