Bug: Merge behavior is inconsistent when file has a rename in its history
(1) By anonymous on 2021-04-14 21:01:12 [source]
This issue stems from the analysis related to forumpost/28c0f56ee1 .
In short, the issue is as follows:
- File change is committed on trunk and then merged successfully on a branch.
- Next, another edit to the file is committed on trunk, yet the subsequent merge (on the branch) fails to pick up the change.
- The file in question has a rename in its prior history.
Below is a test-case which trips this issue. The test returns "Error: merge failed (2)!" and the 'fossil diff --from trunk' returns the edits missed from trunk.
As little as I could comprehend the merge code, the issue is somehow related to the processing done for "Add files found in P". In the second (failed) merge, an additional record is added, unlike in the preceding successful merge. This extra record interferes with the subsequent transformations and results in an empty merge set.
Hope this issue is serious enough to warrant a closer look.
Here's the test-case (tried with fossil 2.15.1).
#-------------
fossil init test.fossil
fossil open test.fossil -f
echo "file1" > file1
fossil add file1
fossil commit -m "added file1"
echo "file2" > file2
fossil add file2
fossil commit -m "added file2" --branch added
echo "edit file2 on added" >> file2
fossil commit -m "edited file2"
fossil mv --hard file2 file2.renamed
fossil commit -m "renamed file2" --branch renamed
fossil branch new branched current
echo "edit file2.renamed on renamed" >> file2.renamed
fossil commit -m "edited file2.renamed"
fossil update trunk
fossil merge renamed
fossil commit -m "merged from renamed"
echo "edit file2.renamed on trunk (1)" >> file2.renamed
fossil commit -m "edited file2.renamed on trunk (1)"
fossil update branched
fossil merge trunk
fossil diff --from trunk | grep -q "on trunk (1)" && echo "Error: merge failed (1)!" && exit 1
fossil commit -m "merged edit from trunk (1:this merge succeeded)"
fossil update trunk
echo "edit2 file2.renamed on trunk (2)" >> file2.renamed
fossil commit -m "edited file2.renamed on trunk (2)"
fossil update branched
fossil merge trunk
fossil diff --from trunk | grep -q "on trunk (2)" && echo "Error: merge failed (2)!" && exit 1
fossil commit -m "merged edit from trunk (2:this merge succeeded)"
fossil close
#-------------
(2) By Stephan Beal (stephan) on 2021-04-15 02:53:35 in reply to 1 [link] [source]
This issue stems from the analysis related to forumpost/28c0f56ee1.
...
Here's the test-case (tried with fossil 2.15.1).
Thank you for tracking that down! Though i don't understand the merge code well enough to resolve this, your test case and characterization of the underlying problem will definitely help in getting it resolved.
(3) By Richard Hipp (drh) on 2021-04-15 12:02:21 in reply to 2 [link] [source]
FWIW: I have the test case up on my screen. I'm trying to figure out what is going on. Dunno yet if I will be successful or not. It does not look like a trivial problem.
(4) By Andy Bradford (andybradford) on 2021-04-15 14:12:35 in reply to 3 [link] [source]
> I have the test case up on my screen. I'm trying to figure out what is > going on. I took a look at it a while ago and was also unable to dedicate enough time to track it down. The merge code is also the least familiar to me, which is why I posted my findings for the historical record until it could be looked at again. Andy
(5) By Richard Hipp (drh) on 2021-04-16 18:41:46 in reply to 1 [link] [source]
I think this is working now for check-in cb4f38ee6733dccc and later. Please verify and also try to break the newly enhanced merge logic.
(6) By anonymous on 2021-04-19 17:48:45 in reply to 5 [link] [source]
I think I have encountered a bug (maybe) that is caused by the checkin you mentioned.
SQLITE_CONSTRAINT(2067): abort at 64 in [UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=2172;]: UNIQUE constraint failed: fv.fn
SQL: UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=2172;
Database error: UNIQUE constraint failed: fv.fn: {UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=2172;}
I have read the comments in merge.c and there is no such case of file
swapping place and more than one rename of a file.
I can't merge this branch to trunk anymore because of this bug, although
I can still cherrypick single commits.
What could cause this?
(7) By Richard Hipp (drh) on 2021-04-19 18:28:14 in reply to 6 [link] [source]
Dunno. Can you share the repository so that I can analyze the problem?
(8) By anonymous on 2021-04-19 18:58:09 in reply to 7 [link] [source]
Could I send it to you in private? Maybe by email?
(9) By Richard Hipp (drh) on 2021-04-19 18:59:46 in reply to 8 [link] [source]
My email is drh at sqlite dot org.
(10) By anonymous on 2021-04-19 19:24:46 in reply to 5 [link] [source]
I think this is working now for check-in cb4f38ee6733dccc and later. Please verify and also try to break the newly enhanced merge logic.
OP here.
The fix you applied indeed appears to clear the test-case. Thank you!
However, the updated Fossil now fails the merge_renames.test:
----------------
tclsh ../../fossil/test/tester.tcl ../../fossil/fossil merge_renames
....
***** End of merge_renames: 8 errors so far ******
***** Final results: 8 errors out of 55 tests
***** Considered failures: merge_renames-6-1 merge_renames-6-2 merge_renames-6-3 merge_renames-6-4 merge_renames-13-6 merge_renames-13-8 merge_renames-13-9 merge_renames-13-11
***** Ignored results: 0 ignored errors out of 55 tests
----------------
Looking closer at the merge_renames-6, shows an error:
----------------
....
fossil merge trunk
ERROR (1): SQLITE_CONSTRAINT(2067): abort at 55 in [UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=11;]: UNIQUE constraint failed: fv.fn
SQL: UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=11;
Database error: UNIQUE constraint failed: fv.fn: {UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=11;}
Expected:
UPDATE f2
UPDATE f3
UPDATE f4
Got:
SQL: {UPDATE fv SET fn=vfile.pathname FROM vfile WHERE fn IS NULL AND vfile.pathname IN (fv.fnm,fv.fnp,fv.fnn) AND vfile.vid=11;}
test merge_renames-6-1 FAILED!
Expected:
f1.2
Got:
f1.1
test merge_renames-6-2 FAILED!
Expected:
f3.1
Got:
f4
test merge_renames-6-3 FAILED!
Expected:
f4.1
Got:
f3
test merge_renames-6-4 FAILED!
----------------
(11) By anonymous on 2021-04-22 23:53:41 in reply to 10 [link] [source]
The updated version dc850c9b3b2d7e9e does fix the SQL_CONTRAINT error mentioned above, however it still fails the 'merge_renames.test'.
../../fossil/fossil version
This is fossil version 2.15 [5df47469a1] 2021-04-22 01:40:52 UTC
tclsh ../../fossil/test/tester.tcl ../../fossil/fossil merge_renames
....
ERROR (1): UPDATE f2
MERGE f4
***** 1 merge conflict in f4
"fossil undo" is available to undo changes to the working checkout.
WARNING: 1 merge conflicts
Expected:
UPDATE f2
UPDATE f3
UPDATE f4
Got:
WARNING: {1 merge conflicts}
UPDATE f2
MERGE f4
test merge_renames-6-1 FAILED!
.....
***** End of merge_renames: 7 errors so far ******
***** Final results: 7 errors out of 55 tests
***** Considered failures: merge_renames-6-1 merge_renames-6-3 merge_renames-6-4 merge_renames-13-6 merge_renames-13-8 merge_renames-13-9 merge_renames-13-11
***** Ignored results: 0 ignored errors out of 55 tests