---
breaks: false
---
# Move packages while preserving history
## Goal
The goal is to move packages (directories) from one git repository to another,
while preserving the full git history.
This is done by filtering the commits in the source repository using
[git-filter-repo](https://github.com/newren/git-filter-repo).
Then, the new filtered history containing only the changes-to-be-moved is
transplanted on top of the desired destination repository branch (e.g. `master`)
using `git rebase --rebase-merges --root`.
As part of the filtering step, the commit messages are rewritten
(see screenshot below) to
1. append a link to the source commit message (e.g. `Moore@123456789`),
2. replace short references to issues and MRs such as `#12345` with `Moore#12345`.
Note that the sources are not modified in any way during the filtering.
There might be tweaks necessary on top of the automatic move in order
for the packages to work fine from within the destination project.
Also, any short references to issues or MRs will not make sense unless
one knows about the move.
Here is an example of how a moved commit looks like on GitLab:
## Prerequisites
### git
Tested with `git version 2.26.2`.
### git-filter-repo
```shell
url="https://github.com/newren/git-filter-repo/releases/download/v2.29.0/git-filter-repo-2.29.0.tar.xz"
checksum="eb269f6e9b91fcacf676f7d5b8174d962dab5facce2022cc59cb672cd33cd602"
archive=git-filter-repo-2.29.0.tar.xz
curl -s -L "$url" | tee $archive | sha256sum -c <(echo "$checksum -") || rm -f $archive
tar xf $archive
export PATH=$PATH:$(pwd)/git-filter-repo-2.29.0
```
## Filter source repository
```shell
setopt nobanghist 2> /dev/null || set +H # disalbe history expansion (zsh or bash)
SOURCE=Moore
FILTERS=(--path This/Package1 That/Package2)
git clone ssh://git@gitlab.cern.ch:7999/lhcb/$SOURCE.git
cd $SOURCE
git filter-repo "${FILTERS[@]}" --commit-callback "
import re
ORIGIN = '$SOURCE'
message = commit.message.decode()
message = re.sub(r'(\s)([#!][0-9]+)', r'\1{}\2'.format(ORIGIN), message)
message = message.rstrip('\n')
message += '\n\nMoved from {}@{}\n'.format(ORIGIN, commit.original_id.decode())
commit.message = message.encode()
" --dry-run
# Check the filtered and transformed commits
less .git/filter-repo/fast-export.filtered
# If happy, re-run without --dry-run
# ...
cd ..
```
## Add commits to the destination repository
```shell
DESTINATION=LHCb
git clone ssh://git@gitlab.cern.ch:7999/lhcb/$DESTINATION.git
cd $DESTINATION
git remote add src ../$SOURCE
git fetch src
git rebase --rebase-merges --root src/master --onto origin/master
```
### Dealing with conflicts
As the git manual for `--rebase-merges` [points out](https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---rebase-mergesrebase-cousinsno-rebase-cousins):
> Any resolved merge conflicts or manual amendments in these merge commits will have to be resolved/re-applied manually.
Whenever you get into a conflicted state, check the conflict carefully with `git diff`. It can be very useful to use the `diff3` conflict style (set with `git config --global merge.conflictstyle diff3`) and see [here](https://stackoverflow.com/questions/27417656/should-diff3-be-default-conflictstyle-on-git) for some details.
It is usually fine to automatically resolve the conflicts by taking the incomming change, which you can do with `git checkout --theirs path/to/file`.
### Verify the final package contents and push
To check that the final package contents are identical in the source and the destination repository, you can do the following
```shell
git diff src/master..HEAD -- This/Package1 That/Package2
```
If there are no differences, you can proceed to create a branch and push it
```shell
git checkout -b move-packages-XYX
git push -u origin move-packages-XYZ
```
If necessary, follow up with tweaks of the package contents in new commits.