Fossil

Check-in [a69f71a2]
Login

Check-in [a69f71a2]

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Fix how Fossil handles 'import --git' into a repository that already exists when neither the '--increment' nor '--force' options are passed, as reported by Warren in forum post 29e358909c. The proposed fix required duplicating a file descriptor for stdin to accept keyboard input from the user and piped input from git fast-import, which necessitated machine-dependent ifdefs that have not yet been tested on Windows platforms. The fix revealed another bug that this commit also solves by avoiding duplicate artifacts being inserted into the blob table during fast_insert_content(), which can arise when importing into a repository that has used the recently implemented '--attribute' feature if manifests are generated from the same content but with different U cards. Due to a bug in Git's output, this required an additional temp table for tracking tag artifacts (see comments). Due to the complexity of this commit, and having not been tested yet on Windows platforms, further testing is needed.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | import-git-dev
Files: files | file ages | folders
SHA3-256: a69f71a2758ad66daf82d8bb71afa202bb3514e63ee2fccbb44def0012c6c143
User & Date: jamsek 2020-11-28 13:36:08
Original Comment: Fix how Fossil handles 'import --git' into a repository that already exists when neither the '--increment' nor '--force' options are passed, as reported by Warren in [forum post 29e358909c|forum:/forumpost/29e358909c]. The proposed fix required duplicating a file descriptor for stdin to accept keyboard input from the user and piped input from git fast-import, which necessitated machine-dependent ifdefs that have not yet been tested on Windows platforms. The fix revealed another bug that this commit also solves by avoiding duplicate artifacts being inserted into the blob table during fast_insert_content(), which can arise when importing into a repository that has used the recently implemented '--attribute' feature if manifests are generated from the same content but with different U cards. Due to a bug in Git's output, this required an additional temp table for tracking tag artifacts (see comments). Due to the complexity of this commit, and having not been tested yet on Windows platforms, further testing is needed.
Context
2020-11-28
13:36
Fix how Fossil handles 'import --git' into a repository that already exists when neither the '--increment' nor '--force' options are passed, as reported by Warren in forum post 29e358909c. The proposed fix required duplicating a file descriptor for stdin to accept keyboard input from the user and piped input from git fast-import, which necessitated machine-dependent ifdefs that have not yet been tested on Windows platforms. The fix revealed another bug that this commit also solves by avoiding duplicate artifacts being inserted into the blob table during fast_insert_content(), which can arise when importing into a repository that has used the recently implemented '--attribute' feature if manifests are generated from the same content but with different U cards. Due to a bug in Git's output, this required an additional temp table for tracking tag artifacts (see comments). Due to the complexity of this commit, and having not been tested yet on Windows platforms, further testing is needed. ... (Closed-Leaf check-in: a69f71a2 user: jamsek tags: import-git-dev)
2020-11-27
13:08
Set an appropriate base URL for the /file page when it is using name= and ci= query parameters. ... (check-in: a7343c6a user: drh tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to src/import.c.

32
33
34
35
36
37
38












39
40
41
42
43
44
45
  char *zPrior;          /* Prior name if the name was changed */
  char isFrom;           /* True if obtained from the parent */
  char isExe;            /* True if executable */
  char isLink;           /* True if symlink */
};
#endif













/*
** State information common to all import types.
*/
static struct {
  const char *zTrunkName;     /* Name of trunk branch */
  const char *zBranchPre;     /* Prepended to non-trunk branch names */
  const char *zBranchSuf;     /* Appended to non-trunk branch names */







>
>
>
>
>
>
>
>
>
>
>
>







32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
  char *zPrior;          /* Prior name if the name was changed */
  char isFrom;           /* True if obtained from the parent */
  char isExe;            /* True if executable */
  char isLink;           /* True if symlink */
};
#endif

/*
 * Flags to indicate whether the import is to an existing or new repository;
 * or using the --attribute option and in the process of importing a commit or
 * tag (to determine the artifact type during fast_insert_content()).
 */
enum import_mode {
  OLD_REPO,
  NEW_REPO,
  COMMIT_ATTR,
  TAG_ATTR
};

/*
** State information common to all import types.
*/
static struct {
  const char *zTrunkName;     /* Name of trunk branch */
  const char *zBranchPre;     /* Prepended to non-trunk branch names */
  const char *zBranchSuf;     /* Appended to non-trunk branch names */
56
57
58
59
60
61
62

63
64
65
66
67
68
69
  char *zTag;                 /* Name of a tag */
  char *zBranch;              /* Name of a branch for a commit */
  char *zPrevBranch;          /* The branch of the previous check-in */
  char *aData;                /* Data content */
  char *zMark;                /* The current mark */
  char *zDate;                /* Date/time stamp */
  char *zUser;                /* User name */

  char *zComment;             /* Comment of a commit */
  char *zFrom;                /* from value as a hash */
  char *zPrevCheckin;         /* Name of the previous check-in */
  char *zFromMark;            /* The mark of the "from" field */
  int nMerge;                 /* Number of merge values */
  int nMergeAlloc;            /* Number of slots in azMerge[] */
  char **azMerge;             /* Merge values */







>







68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
  char *zTag;                 /* Name of a tag */
  char *zBranch;              /* Name of a branch for a commit */
  char *zPrevBranch;          /* The branch of the previous check-in */
  char *aData;                /* Data content */
  char *zMark;                /* The current mark */
  char *zDate;                /* Date/time stamp */
  char *zUser;                /* User name */
  char *zEmail;               /* Email from Git committer string */
  char *zComment;             /* Comment of a commit */
  char *zFrom;                /* from value as a hash */
  char *zPrevCheckin;         /* Name of the previous check-in */
  char *zFromMark;            /* The mark of the "from" field */
  int nMerge;                 /* Number of merge values */
  int nMergeAlloc;            /* Number of slots in azMerge[] */
  char **azMerge;             /* Merge values */
113
114
115
116
117
118
119

120
121
122
123
124
125
126
  gg.xFinish = 0;
  fossil_free(gg.zTag); gg.zTag = 0;
  fossil_free(gg.zBranch); gg.zBranch = 0;
  fossil_free(gg.aData); gg.aData = 0;
  fossil_free(gg.zMark); gg.zMark = 0;
  fossil_free(gg.zDate); gg.zDate = 0;
  fossil_free(gg.zUser); gg.zUser = 0;

  fossil_free(gg.zComment); gg.zComment = 0;
  fossil_free(gg.zFrom); gg.zFrom = 0;
  fossil_free(gg.zFromMark); gg.zFromMark = 0;
  for(i=0; i<gg.nMerge; i++){
    fossil_free(gg.azMerge[i]); gg.azMerge[i] = 0;
  }
  gg.nMerge = 0;







>







126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
  gg.xFinish = 0;
  fossil_free(gg.zTag); gg.zTag = 0;
  fossil_free(gg.zBranch); gg.zBranch = 0;
  fossil_free(gg.aData); gg.aData = 0;
  fossil_free(gg.zMark); gg.zMark = 0;
  fossil_free(gg.zDate); gg.zDate = 0;
  fossil_free(gg.zUser); gg.zUser = 0;
  fossil_free(gg.zEmail); gg.zEmail = 0;
  fossil_free(gg.zComment); gg.zComment = 0;
  fossil_free(gg.zFrom); gg.zFrom = 0;
  fossil_free(gg.zFromMark); gg.zFromMark = 0;
  for(i=0; i<gg.nMerge; i++){
    fossil_free(gg.azMerge[i]); gg.azMerge[i] = 0;
  }
  gg.nMerge = 0;
137
138
139
140
141
142
143













144
145
146
147
148
149
150
    fossil_free(gg.azMerge);
    fossil_free(gg.aFile);
    memset(&gg, 0, sizeof(gg));
  }
  gg.xFinish = finish_noop;
}














/*
** Insert an artifact into the BLOB table if it isn't there already.
** If zMark is not zero, create a cross-reference from that mark back
** to the newly inserted artifact.
**
** If saveHash is true, then pContent is a commit record.  Record its
** artifact hash in gg.zPrevCheckin.







>
>
>
>
>
>
>
>
>
>
>
>
>







151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
    fossil_free(gg.azMerge);
    fossil_free(gg.aFile);
    memset(&gg, 0, sizeof(gg));
  }
  gg.xFinish = finish_noop;
}

static struct{
  const char *zMasterName;   /* Name of master branch */
  int authorFlag;            /* Use author as checkin committer */
  Blob altRecord;            /* Record to cross-check --attribute'd commits */
  uint8_t commitType:2;      /* To indicate --attribute'd manifest or tag */
  uint8_t importType:2;      /* To indicate import into new or existing repo */
  int nGitAttr;              /* Number of Git --attribute entries */
  struct {                   /* Git --attribute details */
    char *zUser;
    char *zEmail;
  } *gitUserInfo;
} ggit;

/*
** Insert an artifact into the BLOB table if it isn't there already.
** If zMark is not zero, create a cross-reference from that mark back
** to the newly inserted artifact.
**
** If saveHash is true, then pContent is a commit record.  Record its
** artifact hash in gg.zPrevCheckin.
158
159
160
161
162
163
164

























165
166
167
168
169
170
171
){
  Blob hash;
  Blob cmpr;
  int rid;

  hname_hash(pContent, 0, &hash);
  rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);

























  if( rid==0 ){
    static Stmt ins;
    assert( g.rcvid>0 );
    db_static_prepare(&ins,
        "INSERT INTO blob(uuid, size, rcvid, content)"
        "VALUES(:uuid, :size, %d, :content)", g.rcvid
    );







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
){
  Blob hash;
  Blob cmpr;
  int rid;

  hname_hash(pContent, 0, &hash);
  rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);
  /*
   * If this repo has had --attribute'd commits, we need to check the blob
   * table for manifests with U cards constructed from both the username and
   * email address to ensure no duplicate entries are attempted.
   */
  if (ggit.commitType == COMMIT_ATTR && rid == 0) {
    blob_reset(&hash);
    hname_hash(&ggit.altRecord, 0, &hash);
    rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);
  }
  /*
   * Likewise, for --attribute'd tags pop the tag artifact with the alternate
   * U card from the temp xtag2 table; both these tags and the above commit
   * artifacts are the same as pContent albeit with the {user|email} U card.
   */
  if (ggit.commitType == TAG_ATTR && rid == 0) {
    db_blob(&ggit.altRecord,
        "SELECT tcontent FROM xtag2 ORDER BY ROWID ASC LIMIT 1");
    db_multi_exec(
        "DELETE FROM xtag2 WHERE tcontent=%Q", blob_str(&ggit.altRecord)
    );
    blob_reset(&hash);
    hname_hash(&ggit.altRecord, 0, &hash);
    rid = db_int(0, "SELECT rid FROM blob WHERE uuid=%B", &hash);
  }
  if( rid==0 ){
    static Stmt ins;
    assert( g.rcvid>0 );
    db_static_prepare(&ins,
        "INSERT INTO blob(uuid, size, rcvid, content)"
        "VALUES(:uuid, :size, %d, :content)", g.rcvid
    );
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
** manifest artifact to the BLOB table.
*/
static void finish_commit(void){
  int i;
  char *zFromBranch;
  char *aTCard[4];                /* Array of T cards for manifest */
  int nTCard = 0;                 /* Entries used in aTCard[] */
  Blob record, cksum;

  import_prior_files();
  qsort(gg.aFile, gg.nFile, sizeof(gg.aFile[0]), mfile_cmp);
  blob_zero(&record);
  blob_appendf(&record, "C %F\n", gg.zComment);
  blob_appendf(&record, "D %s\n", gg.zDate);
  if( !g.fQuiet ){







|







330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
** manifest artifact to the BLOB table.
*/
static void finish_commit(void){
  int i;
  char *zFromBranch;
  char *aTCard[4];                /* Array of T cards for manifest */
  int nTCard = 0;                 /* Entries used in aTCard[] */
  Blob record, cksum, altCksum;

  import_prior_files();
  qsort(gg.aFile, gg.nFile, sizeof(gg.aFile[0]), mfile_cmp);
  blob_zero(&record);
  blob_appendf(&record, "C %F\n", gg.zComment);
  blob_appendf(&record, "D %s\n", gg.zDate);
  if( !g.fQuiet ){
340
341
342
343
344
345
346














347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363

364
365
366
367
368























369

370
371
372
373
374
375
376
    }
  }
  for(i=0; i<nTCard; i++) free(aTCard[i]);

  free(zFromBranch);
  db_multi_exec("INSERT INTO xbranch(tname, brnm) VALUES(%Q,%Q)",
                gg.zMark, gg.zBranch);














  blob_appendf(&record, "U %F\n", gg.zUser);
  md5sum_blob(&record, &cksum);
  blob_appendf(&record, "Z %b\n", &cksum);
  fast_insert_content(&record, gg.zMark, 0, 1, 1);
  blob_reset(&cksum);

  /* The "git fast-export" command might output multiple "commit" lines
  ** that reference a tag using "refs/tags/TAGNAME".  The tag should only
  ** be applied to the last commit that is output.  The problem is we do not
  ** know at this time if the current commit is the last one to hold this
  ** tag or not.  So make an entry in the XTAG table to record this tag
  ** but overwrite that entry if a later instance of the same tag appears.
  **
  ** This behavior seems like a bug in git-fast-export, but it is easier
  ** to work around the problem than to fix git-fast-export.
  */
  if( gg.tagCommit && gg.zDate && gg.zUser && gg.zFrom ){

    record.nUsed = 0
      /*in case fast_insert_comment() did not indirectly blob_reset() it */;
    blob_appendf(&record, "D %s\n", gg.zDate);
    blob_appendf(&record, "T +sym-%F%F%F %s\n", gimport.zBranchPre, gg.zBranch,
        gimport.zBranchSuf, gg.zPrevCheckin);























    blob_appendf(&record, "U %F\n", gg.zUser);

    md5sum_blob(&record, &cksum);
    blob_appendf(&record, "Z %b\n", &cksum);
    db_multi_exec(
       "INSERT OR REPLACE INTO xtag(tname, tcontent)"
       " VALUES(%Q,%Q)", gg.zBranch, blob_str(&record)
    );
    blob_reset(&cksum);







>
>
>
>
>
>
>
>
>
>
>
>
>
>

















>





>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>







392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
    }
  }
  for(i=0; i<nTCard; i++) free(aTCard[i]);

  free(zFromBranch);
  db_multi_exec("INSERT INTO xbranch(tname, brnm) VALUES(%Q,%Q)",
                gg.zMark, gg.zBranch);
  /*
   * The fx_git table indicates this repo has --attribute'd commits; therefore,
   * blob entries may exist with U cards generated from either a username or
   * email address. Create both for cross-checking to avoid adding duplicates.
   */
  if (db_table_exists("repository", "fx_git") &&
   db_text(0, "SELECT user FROM fx_git WHERE email=%Q", gg.zEmail)) {
    blob_copy(&ggit.altRecord, &record);
    blob_appendf(&ggit.altRecord, "U %F\n", gg.zEmail);
    md5sum_blob(&ggit.altRecord, &altCksum);
    blob_appendf(&ggit.altRecord, "Z %b\n", &altCksum);
    blob_reset(&altCksum);
    ggit.commitType = COMMIT_ATTR;
  }
  blob_appendf(&record, "U %F\n", gg.zUser);
  md5sum_blob(&record, &cksum);
  blob_appendf(&record, "Z %b\n", &cksum);
  fast_insert_content(&record, gg.zMark, 0, 1, 1);
  blob_reset(&cksum);

  /* The "git fast-export" command might output multiple "commit" lines
  ** that reference a tag using "refs/tags/TAGNAME".  The tag should only
  ** be applied to the last commit that is output.  The problem is we do not
  ** know at this time if the current commit is the last one to hold this
  ** tag or not.  So make an entry in the XTAG table to record this tag
  ** but overwrite that entry if a later instance of the same tag appears.
  **
  ** This behavior seems like a bug in git-fast-export, but it is easier
  ** to work around the problem than to fix git-fast-export.
  */
  if( gg.tagCommit && gg.zDate && gg.zUser && gg.zFrom ){
    Blob altTag;
    record.nUsed = 0
      /*in case fast_insert_comment() did not indirectly blob_reset() it */;
    blob_appendf(&record, "D %s\n", gg.zDate);
    blob_appendf(&record, "T +sym-%F%F%F %s\n", gimport.zBranchPre, gg.zBranch,
        gimport.zBranchSuf, gg.zPrevCheckin);
    /*
     * If --attribute'd commits are present, we also need tag artifacts of the
     * other potential U card value to be cross-checked against the blob table.
     * Due to the abovementioned Git bug, store in a different table: xtag2.
     */
    if (ggit.commitType == COMMIT_ATTR) {
      blob_copy(&altTag, &record);
      blob_appendf(&altTag, "U %F\n", gg.zEmail);
      md5sum_blob(&altTag, &altCksum);
      blob_appendf(&altTag, "Z %b\n", &altCksum);
      db_multi_exec(
         "INSERT OR REPLACE INTO xtag2(tname, tcontent)"
         " VALUES(%Q,%Q)", gg.zBranch, blob_str(&altTag)
      );
      blob_reset(&altCksum);
      blob_reset(&altTag);
      ggit.commitType = TAG_ATTR;
    }
    /*
     * IF this is a brand new repo, ONLY one type of tag artifact can exist in
     * the blob table, which are those generated in THIS session---and these
     * will only contain the U card created with the gg.zUser value.
     */
    blob_appendf(&record, "U %F\n", ggit.importType == NEW_REPO ?
     gg.zUser : gg.zEmail);
    md5sum_blob(&record, &cksum);
    blob_appendf(&record, "Z %b\n", &cksum);
    db_multi_exec(
       "INSERT OR REPLACE INTO xtag(tname, tcontent)"
       " VALUES(%Q,%Q)", gg.zBranch, blob_str(&record)
    );
    blob_reset(&cksum);
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
    }
    zName[i++] = c;
  }
  zName[i] = 0;
}


static struct{
  const char *zMasterName;    /* Name of master branch */
  int authorFlag;             /* Use author as checkin committer */
  int nGitAttr;               /* Number of Git --attribute entries */
  struct {                    /* Git --attribute details */
    char *zUser;
    char *zEmail;
  } *gitUserInfo;
} ggit;

/*
** Read the git-fast-import format from pIn and insert the corresponding
** content into the database.
*/
static void git_fast_import(FILE *pIn){
  ImportFile *pFile, *pNew;
  int i, mx;







<
<
<
<
<
<
<
<
<
<







629
630
631
632
633
634
635










636
637
638
639
640
641
642
    }
    zName[i++] = c;
  }
  zName[i] = 0;
}












/*
** Read the git-fast-import format from pIn and insert the corresponding
** content into the database.
*/
static void git_fast_import(FILE *pIn){
  ImportFile *pFile, *pNew;
  int i, mx;
689
690
691
692
693
694
695

696
697
698
699
700
701
702
        *(zTo-1) = '\0';
        gg.zUser = fossil_strdup(z);
      }
      if (ggit.nGitAttr > 0 || db_table_exists("repository", "fx_git")) {
        gg.zUser = db_text(gg.zUser,
         "SELECT user FROM fx_git WHERE email=%Q", z);
      }

      secSince1970 = 0;
      for(zTo++; fossil_isdigit(*zTo); zTo++){
        secSince1970 = secSince1970*10 + *zTo - '0';
      }
      fossil_free(gg.zDate);
      gg.zDate = db_text(0, "SELECT datetime(%lld, 'unixepoch')",secSince1970);
      gg.zDate[10] = 'T';







>







770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
        *(zTo-1) = '\0';
        gg.zUser = fossil_strdup(z);
      }
      if (ggit.nGitAttr > 0 || db_table_exists("repository", "fx_git")) {
        gg.zUser = db_text(gg.zUser,
         "SELECT user FROM fx_git WHERE email=%Q", z);
      }
      gg.zEmail = fossil_strdup(z); /* Keep email for blob table cross-check */
      secSince1970 = 0;
      for(zTo++; fossil_isdigit(*zTo); zTo++){
        secSince1970 = secSince1970*10 + *zTo - '0';
      }
      fossil_free(gg.zDate);
      gg.zDate = db_text(0, "SELECT datetime(%lld, 'unixepoch')",secSince1970);
      gg.zDate[10] = 'T';
1721
1722
1723
1724
1725
1726
1727

1728
1729
1730
1731
1732
1733
1734
**
** See also: export
*/
void import_cmd(void){
  char *zPassword;
  FILE *pIn;
  Stmt q;

  int forceFlag = find_option("force", "f", 0)!=0;
  int svnFlag = find_option("svn", 0, 0)!=0;
  int gitFlag = find_option("git", 0, 0)!=0;
  int omitRebuild = find_option("no-rebuild",0,0)!=0;
  int omitVacuum = find_option("no-vacuum",0,0)!=0;
  const char *zDefaultUser = find_option("admin-user","A",1);








>







1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
**
** See also: export
*/
void import_cmd(void){
  char *zPassword;
  FILE *pIn;
  Stmt q;
  int fd;  /* To duplicate stdin file descriptor. */
  int forceFlag = find_option("force", "f", 0)!=0;
  int svnFlag = find_option("svn", 0, 0)!=0;
  int gitFlag = find_option("git", 0, 0)!=0;
  int omitRebuild = find_option("no-rebuild",0,0)!=0;
  int omitVacuum = find_option("no-vacuum",0,0)!=0;
  const char *zDefaultUser = find_option("admin-user","A",1);

1824
1825
1826
1827
1828
1829
1830









1831

1832
1833




1834
1835

1836














1837
1838
1839
1840
1841
1842
1843
  if( g.argc!=3 && g.argc!=4 ){
    usage("--git|--svn ?OPTIONS? NEW-REPOSITORY ?INPUT-FILE?");
  }
  if( g.argc==4 ){
    pIn = fossil_fopen(g.argv[3], "rb");
    if( pIn==0 ) fossil_fatal("cannot open input file \"%s\"", g.argv[3]);
  }else{









    pIn = stdin;

    fossil_binary_mode(pIn);
  }




  if( !incrFlag ){
    if( forceFlag ) file_delete(g.argv[2]);

    db_create_repository(g.argv[2]);














  }
  db_open_repository(g.argv[2]);
  db_open_config(0, 0);
  db_unprotect(PROTECT_ALL);

  db_begin_transaction();
  if( !incrFlag ){







>
>
>
>
>
>
>
>
>
|
>


>
>
>
>
|
|
>

>
>
>
>
>
>
>
>
>
>
>
>
>
>







1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
  if( g.argc!=3 && g.argc!=4 ){
    usage("--git|--svn ?OPTIONS? NEW-REPOSITORY ?INPUT-FILE?");
  }
  if( g.argc==4 ){
    pIn = fossil_fopen(g.argv[3], "rb");
    if( pIn==0 ) fossil_fatal("cannot open input file \"%s\"", g.argv[3]);
  }else{
    /*
     * If piping from stdin with git fast-export, we need to duplicate a fd
     * so we can also accept keyboard input from the user (see: block at 1931).
     */
#if defined(_WIN32) || defined(_WIN64)
    fd = _dup(fileno(stdin));
#else  /* if UNIX */
    fd = dup(fileno(stdin));
#endif  /* dup() hack */
    pIn = fdopen(fd, "r");
    (void) freopen("/dev/tty", "r", stdin);
    fossil_binary_mode(pIn);
  }
  /*
   * If neither --incremental nor --force has been passed but the repository
   * file exists, prompt the user to continue with an incremental import.
   */
  if (forceFlag && file_size(g.argv[2], ExtFILE) != -1)
    file_delete(g.argv[2]);
  if (!incrFlag && file_size(g.argv[2], ExtFILE) == -1) {
    db_create_repository(g.argv[2]);
    ggit.importType = NEW_REPO;
  } else if (!incrFlag && file_size(g.argv[2], ExtFILE) != -1) {
    Blob x;
    char c;
    fossil_print( "[!] Repository file exists: <%s>\n", g.argv[2]);
    prompt_user(">>> Proceed with incremental import [Y/n]? ", &x);
    c = blob_str(&x)[0];
    blob_reset(&x);
    incrFlag = (c != 'n' && c != 'N' );
    if (!incrFlag) {
      fossil_print("Please either provide an alternative filename, delete "
       "the\nfile, or use --force to overwrite the existing repository.\n");
      exit(1);
    }
  }
  db_open_repository(g.argv[2]);
  db_open_config(0, 0);
  db_unprotect(PROTECT_ALL);

  db_begin_transaction();
  if( !incrFlag ){
1918
1919
1920
1921
1922
1923
1924
1925



1926
1927
1928
1929
1930
1931

1932
1933
1934
1935
1936
1937
1938
    ** check-in then xbranch.brnm is the branch that check-in is part of.
    **
    ** The XTAG table records information about tags that need to be applied
    ** to various branches after the import finishes.  The xtag.tcontent field
    ** contains the text of an artifact that will add a tag to a check-in.
    ** The git-fast-export file format might specify the same tag multiple
    ** times but only the last tag should be used.  And we do not know which
    ** occurrence of the tag is the last until the import finishes.



    */
    db_multi_exec(
       "CREATE TEMP TABLE xmark(tname TEXT UNIQUE, trid INT, tuuid TEXT);"
       "CREATE INDEX temp.i_xmark ON xmark(trid);"
       "CREATE TEMP TABLE xbranch(tname TEXT UNIQUE, brnm TEXT);"
       "CREATE TEMP TABLE xtag(tname TEXT UNIQUE, tcontent TEXT);"

    );

    if( markfile_in ){
      FILE *f = fossil_fopen(markfile_in, "r");
      if( !f ){
        fossil_fatal("cannot open %s for reading", markfile_in);
      }







|
>
>
>






>







2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
    ** check-in then xbranch.brnm is the branch that check-in is part of.
    **
    ** The XTAG table records information about tags that need to be applied
    ** to various branches after the import finishes.  The xtag.tcontent field
    ** contains the text of an artifact that will add a tag to a check-in.
    ** The git-fast-export file format might specify the same tag multiple
    ** times but only the last tag should be used.  And we do not know which
    ** occurrence of the tag is the last until the import finishes.  Also,
    ** depending on whether --attribute'd, artifacts may contain U cards made
    ** with either username or emailaddr, so store the last seen version of
    ** each in the xtag and xtag2 tables, respectively.
    */
    db_multi_exec(
       "CREATE TEMP TABLE xmark(tname TEXT UNIQUE, trid INT, tuuid TEXT);"
       "CREATE INDEX temp.i_xmark ON xmark(trid);"
       "CREATE TEMP TABLE xbranch(tname TEXT UNIQUE, brnm TEXT);"
       "CREATE TEMP TABLE xtag(tname TEXT UNIQUE, tcontent TEXT);"
       "CREATE TEMP TABLE xtag2(tname TEXT UNIQUE, tcontent TEXT);"
    );

    if( markfile_in ){
      FILE *f = fossil_fopen(markfile_in, "r");
      if( !f ){
        fossil_fatal("cannot open %s for reading", markfile_in);
      }
1947
1948
1949
1950
1951
1952
1953

1954
1955
1956

1957
1958
1959
1960
1961
1962
1963
1964
1965


1966
1967
1968
1969
1970
1971
1972
    ** The following 'fx_' table is used to hold information needed for
    ** importing and exporting to attribute Fossil check-ins or Git commits
    ** to either a desired username or full contact information string.
    */
    if(ggit.nGitAttr > 0) {
      int idx;
      db_unprotect(PROTECT_ALL);

      db_multi_exec(
        "CREATE TABLE fx_git(user TEXT, email TEXT UNIQUE);"
      );

      for(idx = 0; idx < ggit.nGitAttr; ++idx ){
        db_multi_exec(
            "INSERT OR IGNORE INTO fx_git(user, email) VALUES(%Q, %Q)",
            ggit.gitUserInfo[idx].zUser, ggit.gitUserInfo[idx].zEmail
        );
      }
      db_protect_pop();
    }
    git_fast_import(pIn);


    db_prepare(&q, "SELECT tcontent FROM xtag");
    while( db_step(&q)==SQLITE_ROW ){
      Blob record;
      db_ephemeral_blob(&q, 0, &record);
      fast_insert_content(&record, 0, 0, 0, 1);
      import_reset(0);
    }







>
|
|
|
>









>
>







2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
    ** The following 'fx_' table is used to hold information needed for
    ** importing and exporting to attribute Fossil check-ins or Git commits
    ** to either a desired username or full contact information string.
    */
    if(ggit.nGitAttr > 0) {
      int idx;
      db_unprotect(PROTECT_ALL);
      if (!db_table_exists("repository", "fx_git")) {
        db_multi_exec(
            "CREATE TABLE fx_git(user TEXT, email TEXT UNIQUE);"
            );
      }
      for(idx = 0; idx < ggit.nGitAttr; ++idx ){
        db_multi_exec(
            "INSERT OR IGNORE INTO fx_git(user, email) VALUES(%Q, %Q)",
            ggit.gitUserInfo[idx].zUser, ggit.gitUserInfo[idx].zEmail
        );
      }
      db_protect_pop();
    }
    git_fast_import(pIn);
    if (ggit.commitType == COMMIT_ATTR)
      ggit.commitType = TAG_ATTR;
    db_prepare(&q, "SELECT tcontent FROM xtag");
    while( db_step(&q)==SQLITE_ROW ){
      Blob record;
      db_ephemeral_blob(&q, 0, &record);
      fast_insert_content(&record, 0, 0, 0, 1);
      import_reset(0);
    }