Fossil Forum

git import preserving moves?

git import preserving moves?

git import preserving moves?

(1) By James Cook (falsifian) on 2023-06-13 02:22:00 [link] [source]

Summary: Is fossil import supposed to be able to import file renames detected using git fast-import -M or is that just not implemented?


I've been experimenting with importing a git repo into an (existing) Fossil repo.

I am passing the -M and -C options to git-fast-export, which is supposed to detect moves and copies. Glancing at the output, I think it's working (at least for moves), because although I don't know the git-fast-export file format, lines like the following are quite suggestive:

R beancount/from_gnucash beancount/from_gnucash.beancount

However, when I import with fossil import and examine cases where git apparently detected a rename, in the fossil history it appears as a delete + create.

Is importing renames from the git history simply not supported?

Also, does fossil have a notion of "copying" a file? If not, I suppose I shouldn't bother with the -C option to git fast-export.

The command I'm using, with some parts abbreviated, is: fossil import --git -i --rename-branch money_tracking-% --export-marks .../mt_import/fossil-import_marks --attribute ' falsifian' (more --attribute args) -- test.fossil .../mt_import/git_export

(2) By mark on 2023-06-16 08:10:04 in reply to 1 [link] [source]

I spent some time in the git import code not too long ago and IIRC we indeed do have the plumbing for renames (which, you are correct, are represented by the exemplar line you cite). If they are not being properly imported, this is not the intended behaviour. However, in Fossil, renames can be shown as deletions and additions. For example, see the (DELETED) skins/default/js.txt to src/hbmenu.js (ADDED) rename with the diff command:

fossil diff --verbose --from 18e2f5337f --to 9cd74289c0

This makes it practically impossible to see the changes from this view as you are shown a file of deletions and a file of additions.

But if you look at the same commit in the web ui, it is shown as a rename, albeit without any real clue to the rename (we're only given the new filename) without drilling down further:

(3) By James Cook (falsifian) on 2023-06-16 15:25:30 in reply to 2 [link] [source]

I don't see the rename in the web UI. It appears as a delete and an add.

Here's a script to reproduce:

set -e

mkdir example; cd example

fossil import --git test.fossil <<EOF
mark :1
data 13
Hello world!

reset refs/heads/main
commit refs/heads/main
mark :2
author James Cook <> 1686928435 +0000
committer James Cook <> 1686928435 +0000
data 15
Create a file.
M 100644 :1 a_file

commit refs/heads/main
mark :3
author James Cook <> 1686928445 +0000
committer James Cook <> 1686928445 +0000
data 17
Rename the file.
from :2
R a_file a_renamed_file


After running that script, if I run fossil ui example/test.fossil and view the "Rename a file" commit in the timeline view, it's presented as "Deleted a_file version ..." and "Added a_renamed_file version ...". By contrast, if I do the rename in fossil using fossil mv --hard, then the rename is shown nicely in the ui, except that if I also made changes, the old name is not shown as you point out.

(A bit of a tangent, but would it make sense to file a bug saying the web UI doesn't show that a file was renamed if it was also changed? The closest existing ticket I could find is this one.)

(4) By mark on 2023-06-17 03:38:03 in reply to 3 [link] [source]

Thanks for the repro. That is indeed not the intended behaviour. Could you please let me know if the below diff fixes this for you?

wrt the rename+change eliding the rename in the web ui, don't worry about filing a bug--I'll raise it in /chat.

Index: src/import.c
hash - ac87067f1476aebfb9d2f81f8a1849fc7489e2ef4d12db747428f6430ce7de79
hash + 26bf6db4a5a11bbec1ddde3be392cade9aecdf3a8fe7d2ef39cc54f755debe7b
--- src/import.c
+++ src/import.c
@@ -298,12 +298,16 @@ static void finish_commit(void){
     if( zUuid==0 ) continue;
     blob_appendf(&record, "F %F %s", gg.aFile[i].zName, zUuid);
     if( gg.aFile[i].isExe ){
-      blob_append(&record, " x\n", 3);
+      blob_append(&record, " x", 2);
     }else if( gg.aFile[i].isLink ){
-      blob_append(&record, " l\n", 3);
+      blob_append(&record, " l", 2);
-      blob_append(&record, "\n", 1);
+      blob_append(&record, " w", 2);
+    }
+    if( gg.aFile[i].zPrior!=0 ){
+      blob_appendf(&record, " %F", gg.aFile[i].zPrior);
+    blob_append(&record, "\n", 1);
   if( gg.zFrom ){
     blob_appendf(&record, "P %s", gg.zFrom);

(5.1) By mark on 2023-06-17 14:15:43 edited from 5.0 in reply to 3 [link] [source]

ETA: a slight tweak on this diff has been committed. Thanks for the report, James!

After discussing with larrybr@ in /chat about how we might improve renamed and changed files eliding the renamed detail in the /info view, does the below diff net an improvement in such cases?

Index: src/info.c
hash - adbfea143b8317be6db627b933269980400952bafafab60413d89bd78ce5b12b
hash + 4555594037a8d6cd1efa7f1058a450d70cc74035fde0daec70207eb23093e2d7
--- src/info.c
+++ src/info.c
@@ -399,9 +399,19 @@ static void append_file_change_line(
     if( zOld && zNew ){
       if( fossil_strcmp(zOld, zNew)!=0 ){
-        @ Modified %z(href("%R/finfo?name=%T&m=%!S&ci=%!S",zName,zNew,zCkin))\
-        @ %h(zName)</a>
-        @ from %z(href("%R/artifact/%!S",zOld))[%S(zOld)]</a>
+        if( zOldName!=0 && fossil_strcmp(zName,zOldName)!=0 ){
+          @ Renamed and modified
+          @ %z(href("%R/finfo?name=%T&m=%!S&ci=%!S",zName,zNew,zCkin))\
+          @ %h(zName)</a>
+          @ from
+          @ %z(href("%R/finfo?name=%T&m=%!S&ci=%!S",zOldName,zOld,zCkin))\
+          @ %h(zOldName)</a>
+        }else{
+          @ Modified %z(href("%R/finfo?name=%T&m=%!S&ci=%!S",zName,zNew,zCkin))\
+          @ %h(zName)</a>
+          @ from
+        }
+        @ %z(href("%R/artifact/%!S",zOld))[%S(zOld)]</a>
         @ to %z(href("%R/artifact/%!S",zNew))[%S(zNew)]</a>.
       }else if( zOldName!=0 && fossil_strcmp(zName,zOldName)!=0 ){
         @ Name change

(6) By James Cook (falsifian) on 2023-06-18 01:27:03 in reply to 5.1 [source]

Thank you, the committed version works for me.

I found one or two other places where there's a similar issue:

  • In the output of fossil changes and similar places (fossil stat, initial editor contents for commit message), I only see EDITED for a file that has been both renamed and edited.
  • In the code preceding your change, in the if( !g.perm.Hyperlink ) case, it looks like a rename+edit will only show as a rename? I couldn't figure out how to trigger that code path so I'm not sure. (I tried revoking the hyperlink permission from the anonymous user but I still see hyperlinks.)

I will try your src/import.c patch next.

(7) By James Cook (falsifian) on 2023-06-18 01:44:52 in reply to 4 [link] [source]

Thanks. I tried your patch with the git repo I'm trying to import, and moves do seem to be properly detected. (And it works with the reproducing script I posted too.)

(8) By mark on 2023-06-18 03:15:38 in reply to 7 [link] [source]

Thanks for the report and testing the fix! This has been committed.