fossil git autopush fails for git mirroring
(1) By abhijit on 2020-02-06 05:20:13 [link] [source]
O/S - Windows 10 Fossil version - 2.10 Git version - 2.20
I'm trying to create a git mirror of a fossil repository.
I tried the following from my fossil working copy folder
fossil git export < git repo folder > --autopush <git url>
I got the following output -
git fast-import --export-marks=.mirror_state/marks.txt --quiet --done
fatal: Branch name doesn't conform to GIT standards: refs/heads/master
fast-import: dumping crash report to .git/fast_import_crash_3736
11 check-ins added to the < fossil working copy folder>
git update-ref "refs/heads/master"
usage: git update-ref [<options>] -d <refname> [<old-val>]
or: git update-ref [<options>] <refname> <new-val> [<old-val>]
or: git update-ref [<options>] --stdin [-z]
-m <reason> reason of the update
-d delete the reference
--no-deref update <refname> not the one it points to
-z stdin has NUL-terminated arguments
--stdin read updates from stdin
--create-reflog create a reflog
git push --mirror < git url>
No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as 'master'.
Everything up-to-date
Essentially, it doesn't seem to work on a bare repository. I tried this both on github as well as self-hosted git repository.
The other commands mentioned in the Fossil - Import And Export seems to work just fine.
The crash report output is -
fast-import crash report:
fast-import process: 3736
parent process : 1
at 2020-02-06 05:04:09 +0000
fatal: Branch name doesn't conform to GIT standards: refs/heads/master
Most Recent Commands Before Crash
---------------------------------
* commit refs/heads/master
Active Branch LRU
-----------------
active_branches = 0 cur, 5 max
pos clock name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Inactive Branches
-----------------
Marks
-----
exported to .mirror_state/marks.txt
-------------------
END OF CRASH REPORT
(2) By Warren Young (wyetr) on 2020-02-06 06:47:14 in reply to 1 [link] [source]
O/S - Windows 10 Fossil version - 2.10 Git version - 2.20
This appears to be a Windows Git problem, probably not with Git itself but with the way the native Fossil binary interacts with the necessarily POSIX based Git distribution you're using. It's an inherent semantic mismatch.
The simple fact is that Git isn't native on Windows. You have to run it under some kind of POSIX emulation environment — typically either MinGW, Cygwin, or WSL — in order to get it to run at all. This means there can be semantic mismatches when calling across from one to the other, as fossil git export
does.
I've come to this diagnosis because I can reproduce your symptom using the native Windows Fossil 2.10 executable (the official build) with the Cygwin build of Git using these commands:
$ mkdir y
$ fossil-native-2.10 init x.fossil
$ fossil-native-2.10 git export -R x.fossil y
However, when I switch the second two commands to just "fossil
", giving the Cygwin build of 2.10, the symptom goes away.
Switching to WSL on the same system also works, since in that case you're using Linux ELF executables for Fossil and Git, so the translation layer is thinner than in the Cygwin case.
Personally, I'd just switch to WSL or Cygwin for this step at least. Not only will the problem go away, you'll get a far superior shell, as we'll see below.
fossil git export < git repo folder > --autopush <git url>
Please don't use <
and >
in command examples. That's a meaningful character to the shell on almost every OS in common use. In my initial reading of your post, I thought you were trying to redirect I/O to/from Git there until I figured out you were eliding a local path name.
Instead, provide a vague equivalent like ..\path\to\git\repo
, something that could work, but which just happens not to be the actual path on your system.
Branch name doesn't conform to GIT standards: refs/heads/master
Git is babbling here, since that's a legal ref name according to Git 2.18 on CentOS 8 and Git 2.21.1 on macOS. Test case:
$ git check-ref-format refs/heads/master && echo yes
That command won't work on native Windows. This program produces no text output: it only gives a zero or nonzero exit status, and I don't know the "&& echo yes
" equivalent for Windows. Once again I suggest using Cygwin or WSL instead.
(3) By Warren Young (wyetr) on 2020-02-06 06:53:17 in reply to 2 [link] [source]
Branch name doesn't conform to GIT standards: refs/heads/master
Git is babbling here, since that's a legal ref name
Here's a thought: it could be a CRLF vs LF thing, since rule 4 according to man git-check-ref-format
is:
They cannot have ASCII control characters (i.e. bytes whose values are lower than 040, or 177 DEL), space, tilde ~, caret ^, or colon : anywhere.
This would explain the error, since if it's getting something like "refs/heads/master\n
", the real problem wouldn't be visible in the output, but it'd still cause a problem.
Too bad Git doesn't say which of its ten (10!) ref naming rules is being violated here.
And seriously, Git, octal? In 2020? Sheesh.
(4) By Florian Balmer (florian.balmer) on 2020-02-06 14:20:09 in reply to 2 [link] [source]
This is a duplicate of [692cf62eb3] "git export problem". I forgot to take a note about the first post, and hence forgot to reply.
(This is the downside of the Forum compared to Tickets: once Forum posts with bug reports have vanished from the "Most recent threads" page of the Forum, the bug reports have also vanished from collective memory, somehow -- unless somebody has jumped in with a quick fix. Without consulting my bookmarks, I can recall at least 3 or 4 more bug reports to the Forum, left unaddressed to date.)
This is not a Windows Git problem, nor a problem of the Windows Git environment (such as the underlying shell).
The problem is that the MSVC runtime library stream I/O functions convert LF to CR+LF for text mode streams. The fix is simple (credits to Warren, you already got it ☺):
Patch to Fossil [5a7cda57cb]
Index: src/export.c
==================================================================
--- src/export.c
+++ src/export.c
@@ -1385,11 +1385,11 @@
}else{
zCmd = mprintf("git fast-import"
" --export-marks=.mirror_state/marks.txt"
" --quiet --done");
gitmirror_message(VERB_NORMAL, "%s\n", zCmd);
- xCmd = popen(zCmd, "w");
+ xCmd = popen(zCmd, "wb");
if( zCmd==0 ){
fossil_fatal("cannot start the \"git fast-import\" command");
}
fossil_free(zCmd);
}
I don't know the "&& echo yes" equivalent for Windows.
The command separators / chaining operators &
, &&
and ¦¦
work fine with CMD.EXE on Windows!
(5) By Richard Hipp (drh) on 2020-02-06 14:35:12 in reply to 4 [link] [source]
The patch segfaults on Linux since popen() there does not understand "wb".
How about this one instead:
Index: src/export.c
==================================================================
--- src/export.c
+++ src/export.c
@@ -1385,11 +1385,15 @@
}else{
zCmd = mprintf("git fast-import"
" --export-marks=.mirror_state/marks.txt"
" --quiet --done");
gitmirror_message(VERB_NORMAL, "%s\n", zCmd);
+#ifdef _WIN32
+ xCmd = popen(zCmd, "wb");
+#else
xCmd = popen(zCmd, "w");
+#endif
if( zCmd==0 ){
fossil_fatal("cannot start the \"git fast-import\" command");
}
fossil_free(zCmd);
}
Can somebody with a Windows system that has Git installed please try this patch and report back? Please try using both MSVC and Mingw32 builds.
(6) By Florian Balmer (florian.balmer) on 2020-02-06 14:46:15 in reply to 5 [link] [source]
Can somebody with a Windows system that has Git installed please try this patch and report back? Please try using both MSVC and Mingw32 builds.
Leaving this to the people who reported the problem, since I don't have Mingw32 to test.
(There's a light-weight MinGW distribution I meant to try, but haven't so far.)
(7) By Florian Balmer (florian.balmer) on 2020-02-06 14:56:30 in reply to 5 [link] [source]
On the other hand, since the Mingw32 builds are reported to work, why not apply the fix to MSVC only:
#ifdef _MSC_VER
...
#endif
(8) By Richard Hipp (drh) on 2020-02-06 15:07:43 in reply to 5 [source]
I installed Git on my Win10 laptop and was able to test the revised patch. It has now been checked in.