Fossil

Check-in [8bdd0abc]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Improve comments
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | invalid_utf8_improvements
Files: files | file ages | folders
SHA1:8bdd0abc7a4e95728c12103e3bbec38c911157af
User & Date: jan.nijtmans 2016-06-26 17:04:26
Context
2016-06-26
17:05
micro-optimizing invalid_utf8 function, should be as fast as possible now check-in: 7c08a685 user: jan.nijtmans tags: trunk
17:04
Improve comments Closed-Leaf check-in: 8bdd0abc user: jan.nijtmans tags: invalid_utf8_improvements
2016-06-24
03:36
If the FOSSIL_SECURITY_LEVEL environment variable is 2 or more, then present a simple substitution matrix when entering passwords, as a defense against key loggers. For FOSSIL_SECURITY_LEVEL of 1 or more, do not remember the remote-url password. check-in: e1034c4c user: drh tags: trunk
2016-06-18
16:50
If the table is encoded as start-value/size, a variable and a comparison can be saved. Should be even faster .... check-in: 758e3d31 user: jan.nijtmans tags: invalid_utf8_improvements
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to src/allrepo.c.

136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
**    unset       conjunction with the "max-loadavg" setting which cannot
**                otherwise be set globally.
**
** In addition, the following maintenance operations are supported:
**
**    add         Add all the repositories named to the set of repositories
**                tracked by Fossil.  Normally Fossil is able to keep up with
**                this list by itself, but sometime it can benefit from this
**                hint if you rename repositories.
**
**    ignore      Arguments are repositories that should be ignored by
**                subsequent clean, extras, list, pull, push, rebuild, and
**                sync operations.  The -c|--ckout option causes the listed
**                local checkouts to be ignored instead.
**







|







136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
**    unset       conjunction with the "max-loadavg" setting which cannot
**                otherwise be set globally.
**
** In addition, the following maintenance operations are supported:
**
**    add         Add all the repositories named to the set of repositories
**                tracked by Fossil.  Normally Fossil is able to keep up with
**                this list by itself, but sometimes it can benefit from this
**                hint if you rename repositories.
**
**    ignore      Arguments are repositories that should be ignored by
**                subsequent clean, extras, list, pull, push, rebuild, and
**                sync operations.  The -c|--ckout option causes the listed
**                local checkouts to be ignored instead.
**

Changes to src/bundle.c.

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
...
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
/*
** SQL code used to initialize the schema of a bundle.
**
** The bblob.delta field can be an integer, a text string, or NULL.
** If an integer, then the corresponding blobid is the delta basis.
** If a text string, then that string is a SHA1 hash for the delta
** basis, which is presumably in the master repository.  If NULL, then
** data contains contain without delta compression.
*/
static const char zBundleInit[] =
@ CREATE TABLE IF NOT EXISTS "%w".bconfig(
@   bcname TEXT,
@   bcvalue ANY
@ );
@ CREATE TABLE IF NOT EXISTS "%w".bblob(
................................................................................
        deltaFrom = db_int(0,
           "SELECT max(fid) FROM mlink"
           " WHERE fnid=(SELECT fnid FROM mlink WHERE fid=%d)"
           "   AND fid<%d", rid, mnToBundle);
      }
    }

    /* Try to insert the insert the artifact as a delta
    */
    if( deltaFrom ){
      Blob basis, delta;
      content_get(deltaFrom, &basis);
      blob_delta_create(&basis, &content, &delta);
      if( blob_size(&delta)>0.9*blob_size(&content) ){
        deltaFrom = 0;







|







 







|







24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
...
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
/*
** SQL code used to initialize the schema of a bundle.
**
** The bblob.delta field can be an integer, a text string, or NULL.
** If an integer, then the corresponding blobid is the delta basis.
** If a text string, then that string is a SHA1 hash for the delta
** basis, which is presumably in the master repository.  If NULL, then
** data contains content without delta compression.
*/
static const char zBundleInit[] =
@ CREATE TABLE IF NOT EXISTS "%w".bconfig(
@   bcname TEXT,
@   bcvalue ANY
@ );
@ CREATE TABLE IF NOT EXISTS "%w".bblob(
................................................................................
        deltaFrom = db_int(0,
           "SELECT max(fid) FROM mlink"
           " WHERE fnid=(SELECT fnid FROM mlink WHERE fid=%d)"
           "   AND fid<%d", rid, mnToBundle);
      }
    }

    /* Try to insert the artifact as a delta
    */
    if( deltaFrom ){
      Blob basis, delta;
      content_get(deltaFrom, &basis);
      blob_delta_create(&basis, &content, &delta);
      if( blob_size(&delta)>0.9*blob_size(&content) ){
        deltaFrom = 0;

Changes to src/fusefs.c.

49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
  ManifestFile *pFile;      /* Name of a cached file */
  Blob content;             /* Content of the cached file */
  /* Parsed path */
  char *az[3];              /* 0=type, 1=id, 2=path */
} fusefs;

/*
** Clear the fusefs.sz[] array.
*/
static void fusefs_clear_path(void){
  int i;
  for(i=0; i<count(fusefs.az); i++){
    fossil_free(fusefs.az[i]);
    fusefs.az[i] = 0;
  }







|







49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
  ManifestFile *pFile;      /* Name of a cached file */
  Blob content;             /* Content of the cached file */
  /* Parsed path */
  char *az[3];              /* 0=type, 1=id, 2=path */
} fusefs;

/*
** Clear the fusefs.az[] array.
*/
static void fusefs_clear_path(void){
  int i;
  for(i=0; i<count(fusefs.az); i++){
    fossil_free(fusefs.az[i]);
    fusefs.az[i] = 0;
  }

Changes to src/lookslike.c.

48
49
50
51
52
53
54































55
56
57
58
59
60
61
...
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
#define LOOK_ODD     ((int)0x00000080) /* An odd number of bytes was found. */
#define LOOK_SHORT   ((int)0x00000100) /* Unable to perform full check. */
#define LOOK_INVALID ((int)0x00000200) /* Invalid sequence was found. */
#define LOOK_BINARY  (LOOK_NUL | LOOK_LONG | LOOK_SHORT) /* May be binary. */
#define LOOK_EOL     (LOOK_LONE_CR | LOOK_LONE_LF | LOOK_CRLF) /* Line seps. */
#endif /* INTERFACE */

































/*
** This function attempts to scan each logical line within the blob to
** determine the type of content it appears to contain.  The return value
** is a combination of one or more of the LOOK_XXX flags (see above):
**
** !LOOK_BINARY -- The content appears to consist entirely of text; however,
................................................................................
  }
  return flags;
}

/*
** Checks for proper UTF-8. It uses the method described in:
**   http://en.wikipedia.org/wiki/UTF-8#Invalid_byte_sequences
** except for the "overlong form" of \u0000 which is not considered invalid
** here: Some languages like Java and Tcl use it. This function also
** considers valid the derivatives CESU-8 & WTF-8 (as described in the
** same wikipedia article referenced previously). For UTF-8 characters
** > 7f, the variable 'c2' not necessary means the previous character.
** It's number of higher 1-bits indicate the number of continuation bytes
** that are expected to be followed. E.g. when 'c2' has a value in the range
** 0xc0..0xdf it means that 'c' is expected to contain the last continuation
** byte of a UTF-8 character. A value 0xe0..0xef means that after 'c' one
** more continuation byte is expected.
*/

/* definitions for various UTF-8 sequence lengths, encoded as start value
 * and size of each valid range belonging to some lead byte*/
#define US2A  0x80, 0x01 /* for lead byte 0xC0 */
#define US2B  0x80, 0x40 /* for lead bytes 0xC2-0xDF */
#define US3A  0xA0, 0x20 /* for lead byte 0xE0 */
#define US3B  0x80, 0x40 /* for lead bytes 0xE1-0xEF */
#define US4A  0x90, 0x30 /* for lead byte 0xF0 */
#define US4B  0x80, 0x40 /* for lead bytes 0xF1-0xF3 */
#define US4C  0x80, 0x10 /* for lead byte 0xF4 */
#define US0A  0xFF, 0x00 /* for any other lead byte */

/* a table used for quick lookup of the definition that goes with a
 * particular lead byte */
static const unsigned char lb_tab[] = {
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US2A, US0A, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US3A, US3B, US3B, US3B, US3B, US3B, US3B, US3B,
  US3B, US3B, US3B, US3B, US3B, US3B, US3B, US3B,
  US4A, US4B, US4B, US4B, US4C, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A
};

int invalid_utf8(
  const Blob *pContent
){
  const unsigned char *z = (unsigned char *) blob_buffer(pContent);
  unsigned int n = blob_size(pContent);
  unsigned char c; /* lead byte to be handled. */







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
|
|
|
|
|
|
|
|
|

<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
...
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
































182
183
184
185
186
187
188
#define LOOK_ODD     ((int)0x00000080) /* An odd number of bytes was found. */
#define LOOK_SHORT   ((int)0x00000100) /* Unable to perform full check. */
#define LOOK_INVALID ((int)0x00000200) /* Invalid sequence was found. */
#define LOOK_BINARY  (LOOK_NUL | LOOK_LONG | LOOK_SHORT) /* May be binary. */
#define LOOK_EOL     (LOOK_LONE_CR | LOOK_LONE_LF | LOOK_CRLF) /* Line seps. */
#endif /* INTERFACE */

/* definitions for various UTF-8 sequence lengths, encoded as start value
 * and size of each valid range belonging to some lead byte*/
#define US2A  0x80, 0x01 /* for lead byte 0xC0 */
#define US2B  0x80, 0x40 /* for lead bytes 0xC2-0xDF */
#define US3A  0xA0, 0x20 /* for lead byte 0xE0 */
#define US3B  0x80, 0x40 /* for lead bytes 0xE1-0xEF */
#define US4A  0x90, 0x30 /* for lead byte 0xF0 */
#define US4B  0x80, 0x40 /* for lead bytes 0xF1-0xF3 */
#define US4C  0x80, 0x10 /* for lead byte 0xF4 */
#define US0A  0x00, 0x00 /* for any other lead byte */

/* a table used for quick lookup of the definition that goes with a
 * particular lead byte */
static const unsigned char lb_tab[] = {
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A,
  US2A, US0A, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US2B, US2B, US2B, US2B, US2B, US2B, US2B, US2B,
  US3A, US3B, US3B, US3B, US3B, US3B, US3B, US3B,
  US3B, US3B, US3B, US3B, US3B, US3B, US3B, US3B,
  US4A, US4B, US4B, US4B, US4C, US0A, US0A, US0A,
  US0A, US0A, US0A, US0A, US0A, US0A, US0A, US0A
};

/*
** This function attempts to scan each logical line within the blob to
** determine the type of content it appears to contain.  The return value
** is a combination of one or more of the LOOK_XXX flags (see above):
**
** !LOOK_BINARY -- The content appears to consist entirely of text; however,
................................................................................
  }
  return flags;
}

/*
** Checks for proper UTF-8. It uses the method described in:
**   http://en.wikipedia.org/wiki/UTF-8#Invalid_byte_sequences
** except for the "overlong form" of \u0000 which is not considered
** invalid here: Some languages like Java and Tcl use it. This function
** also considers valid the derivatives CESU-8 & WTF-8 (as described in
** the same wikipedia article referenced previously). For UTF-8 characters
** > 0x7f, the variable 'c' not necessary means the real lead byte.
** It's number of higher 1-bits indicate the number of continuation
** bytes that are expected to be followed. E.g. when 'c' has a value
** in the range 0xc0..0xdf it means that after 'c' a single continuation
** byte is expected. A value 0xe0..0xef means that after 'c' two more
** continuation bytes are expected.
*/

































int invalid_utf8(
  const Blob *pContent
){
  const unsigned char *z = (unsigned char *) blob_buffer(pContent);
  unsigned int n = blob_size(pContent);
  unsigned char c; /* lead byte to be handled. */

Changes to src/markdown.c.

33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
...
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
...
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
...
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
...
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
...
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
...
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
....
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
....
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
....
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
....
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
....
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
....
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
....
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
....
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
 ********************/

#if INTERFACE

/* mkd_autolink -- type of autolink */
enum mkd_autolink {
  MKDA_NOT_AUTOLINK,    /* used internally when it is not an autolink*/
  MKDA_NORMAL,          /* normal http/http/ftp/etc link */
  MKDA_EXPLICIT_EMAIL,  /* e-mail link with explit mailto: */
  MKDA_IMPLICIT_EMAIL   /* e-mail link without mailto: */
};

/* mkd_renderer -- functions for rendering parsed data */
struct mkd_renderer {
  /* document level callbacks */
  void (*prolog)(struct Blob *ob, void *opaque);
................................................................................
    i++;
  }
  if( i>=size || data[i]!='>' || nb!=1 ) return 0;
  return i+1;
}


/* tag_length -- returns the length of the given tag, or 0 is it's not valid */
static size_t tag_length(char *data, size_t size, enum mkd_autolink *autolink){
  size_t i, j;

  /* a valid tag can't be shorter than 3 chars */
  if( size<3 ) return 0;

  /* begins with a '<' optionally followed by '/', followed by letter */
................................................................................
    /* one of the forbidden chars has been found */
    *autolink = MKDA_NOT_AUTOLINK;
  }else if( (j = is_mail_autolink(data+i, size-i))!=0 ){
    *autolink = (i==8) ? MKDA_EXPLICIT_EMAIL : MKDA_IMPLICIT_EMAIL;
    return i+j;
  }

  /* looking for sometinhg looking like a tag end */
  while( i<size && data[i]!='>' ){ i++; }
  if( i>=size ) return 0;
  return i+1;
}


/* parse_inline -- parses inline markdown elements */
................................................................................
      i++;
    }
  }
  return 0;
}


/* parse_emph1 -- parsing single emphase */
/* closed by a symbol not preceded by whitespace and not followed by symbol */
static size_t parse_emph1(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
................................................................................
      return r ? i+1 : 0;
    }
  }
  return 0;
}


/* parse_emph2 -- parsing single emphase */
static size_t parse_emph2(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
){
................................................................................
    }
    i++;
  }
  return 0;
}


/* parse_emph3 -- parsing single emphase */
/* finds the first closing tag, and delegates to the other emph */
static size_t parse_emph3(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
................................................................................
    }
  }
  return 2;
}


/* char_entity -- '&' escaped when it doesn't belong to an entity */
/* valid entities are assumed to be anything mathing &#?[A-Za-z0-9]+; */
static size_t char_entity(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t offset,
  size_t size
){
................................................................................
    if( id_end>=size ) goto char_link_cleanup;

    if( i+1==id_end ){
      /* implicit id - use the contents */
      id_data = data+1;
      id_size = txt_e-1;
    }else{
      /* explici id - between brackets */
      id_data = data+i+1;
      id_size = id_end-(i+1);
    }

    if( get_link_ref(rndr, link, title, id_data, id_size)<0 ){
      goto char_link_cleanup;
    }
................................................................................
    return (i>=size || data[i]=='\n') ? 2 : 0;
  }

  return 0;
}


/* is_table_sep -- returns wether there is a table separator at the given pos */
static int is_table_sep(char *data, size_t pos){
  return data[pos]=='|' && (pos==0 || data[pos-1]!='\\');
}


/* is_tableline -- returns the number of column tables in the given line */
static int is_tableline(char *data, size_t size){
................................................................................
    }
  }else{
    return 0;
  }
}


/* prefix_code -- returns prefix length for block code*/
static size_t prefix_code(char *data, size_t size){
  if( size>0 && data[0]=='\t' ) return 1;
  if( size>3 && data[0]==' ' && data[1]==' ' && data[2]==' ' && data[3]==' ' ){
    return 4;
  }
  return 0;
}
................................................................................
static void parse_block(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size);


/* parse_blockquote -- hanldes parsing of a blockquote fragment */
static size_t parse_blockquote(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t beg, end = 0, pre, work_size = 0;
................................................................................
    rndr->make.blockquote(ob, out ? out : &fallback, rndr->make.opaque);
  }
  release_work_buffer(rndr, out);
  return end;
}


/* parse_blockquote -- hanldes parsing of a regular paragraph */
static size_t parse_paragraph(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t i = 0, end = 0;
................................................................................
      release_work_buffer(rndr, span);
    }
  }
  return end;
}


/* parse_blockquote -- hanldes parsing of a block-level code fragment */
static size_t parse_blockcode(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t beg, end, pre;
................................................................................
  int flags               /* table flags */
){
  size_t i = 0, col = 0;
  size_t beg, end, total = 0;
  struct Blob *cells = new_work_buffer(rndr);
  int align;

  /* skip leading blanks and sperator */
  while( i<size && (data[i]==' ' || data[i]=='\t') ){ i++; }
  if( i<size && data[i]=='|' ) i++;

  /* go over all the cells */
  while( i<size && total==0 ){
    /* check optional left/center align marker */
    align = 0;
................................................................................

/* is_ref -- returns whether a line is a reference or not */
static int is_ref(
  char *data,         /* input text */
  size_t beg,         /* offset of the beginning of the line */
  size_t end,         /* offset of the end of the text */
  size_t *last,       /* last character of the link */
  struct Blob *refs   /* arry of link references */
){
  size_t i = 0;
  size_t id_offset, id_end;
  size_t link_offset, link_end;
  size_t title_offset, title_end;
  size_t line_end;
  struct link_ref lr = {







|
|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







 







|







33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
...
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
...
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
...
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
...
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
...
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
...
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
....
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
....
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
....
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
....
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
....
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
....
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
....
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
....
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
 ********************/

#if INTERFACE

/* mkd_autolink -- type of autolink */
enum mkd_autolink {
  MKDA_NOT_AUTOLINK,    /* used internally when it is not an autolink*/
  MKDA_NORMAL,          /* normal http/http/ftp link */
  MKDA_EXPLICIT_EMAIL,  /* e-mail link with explicit mailto: */
  MKDA_IMPLICIT_EMAIL   /* e-mail link without mailto: */
};

/* mkd_renderer -- functions for rendering parsed data */
struct mkd_renderer {
  /* document level callbacks */
  void (*prolog)(struct Blob *ob, void *opaque);
................................................................................
    i++;
  }
  if( i>=size || data[i]!='>' || nb!=1 ) return 0;
  return i+1;
}


/* tag_length -- returns the length of the given tag, or 0 if it's not valid */
static size_t tag_length(char *data, size_t size, enum mkd_autolink *autolink){
  size_t i, j;

  /* a valid tag can't be shorter than 3 chars */
  if( size<3 ) return 0;

  /* begins with a '<' optionally followed by '/', followed by letter */
................................................................................
    /* one of the forbidden chars has been found */
    *autolink = MKDA_NOT_AUTOLINK;
  }else if( (j = is_mail_autolink(data+i, size-i))!=0 ){
    *autolink = (i==8) ? MKDA_EXPLICIT_EMAIL : MKDA_IMPLICIT_EMAIL;
    return i+j;
  }

  /* looking for something looking like a tag end */
  while( i<size && data[i]!='>' ){ i++; }
  if( i>=size ) return 0;
  return i+1;
}


/* parse_inline -- parses inline markdown elements */
................................................................................
      i++;
    }
  }
  return 0;
}


/* parse_emph1 -- parsing single emphasis */
/* closed by a symbol not preceded by whitespace and not followed by symbol */
static size_t parse_emph1(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
................................................................................
      return r ? i+1 : 0;
    }
  }
  return 0;
}


/* parse_emph2 -- parsing single emphasis */
static size_t parse_emph2(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
){
................................................................................
    }
    i++;
  }
  return 0;
}


/* parse_emph3 -- parsing single emphasis */
/* finds the first closing tag, and delegates to the other emph */
static size_t parse_emph3(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size,
  char c
................................................................................
    }
  }
  return 2;
}


/* char_entity -- '&' escaped when it doesn't belong to an entity */
/* valid entities are assumed to be anything matching &#?[A-Za-z0-9]+; */
static size_t char_entity(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t offset,
  size_t size
){
................................................................................
    if( id_end>=size ) goto char_link_cleanup;

    if( i+1==id_end ){
      /* implicit id - use the contents */
      id_data = data+1;
      id_size = txt_e-1;
    }else{
      /* explicit id - between brackets */
      id_data = data+i+1;
      id_size = id_end-(i+1);
    }

    if( get_link_ref(rndr, link, title, id_data, id_size)<0 ){
      goto char_link_cleanup;
    }
................................................................................
    return (i>=size || data[i]=='\n') ? 2 : 0;
  }

  return 0;
}


/* is_table_sep -- returns whether there is a table separator at pos */
static int is_table_sep(char *data, size_t pos){
  return data[pos]=='|' && (pos==0 || data[pos-1]!='\\');
}


/* is_tableline -- returns the number of column tables in the given line */
static int is_tableline(char *data, size_t size){
................................................................................
    }
  }else{
    return 0;
  }
}


/* prefix_code -- returns prefix length for block code */
static size_t prefix_code(char *data, size_t size){
  if( size>0 && data[0]=='\t' ) return 1;
  if( size>3 && data[0]==' ' && data[1]==' ' && data[2]==' ' && data[3]==' ' ){
    return 4;
  }
  return 0;
}
................................................................................
static void parse_block(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size);


/* parse_blockquote -- handles parsing of a blockquote fragment */
static size_t parse_blockquote(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t beg, end = 0, pre, work_size = 0;
................................................................................
    rndr->make.blockquote(ob, out ? out : &fallback, rndr->make.opaque);
  }
  release_work_buffer(rndr, out);
  return end;
}


/* parse_paragraph -- handles parsing of a regular paragraph */
static size_t parse_paragraph(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t i = 0, end = 0;
................................................................................
      release_work_buffer(rndr, span);
    }
  }
  return end;
}


/* parse_blockcode -- handles parsing of a block-level code fragment */
static size_t parse_blockcode(
  struct Blob *ob,
  struct render *rndr,
  char *data,
  size_t size
){
  size_t beg, end, pre;
................................................................................
  int flags               /* table flags */
){
  size_t i = 0, col = 0;
  size_t beg, end, total = 0;
  struct Blob *cells = new_work_buffer(rndr);
  int align;

  /* skip leading blanks and separator */
  while( i<size && (data[i]==' ' || data[i]=='\t') ){ i++; }
  if( i<size && data[i]=='|' ) i++;

  /* go over all the cells */
  while( i<size && total==0 ){
    /* check optional left/center align marker */
    align = 0;
................................................................................

/* is_ref -- returns whether a line is a reference or not */
static int is_ref(
  char *data,         /* input text */
  size_t beg,         /* offset of the beginning of the line */
  size_t end,         /* offset of the end of the text */
  size_t *last,       /* last character of the link */
  struct Blob *refs   /* array of link references */
){
  size_t i = 0;
  size_t id_offset, id_end;
  size_t link_offset, link_end;
  size_t title_offset, title_end;
  size_t line_end;
  struct link_ref lr = {

Changes to src/markdown_html.c.

355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
  struct Blob *link,
  struct Blob *title,
  struct Blob *content,
  void *opaque
){
  char *zLink = blob_buffer(link);
  BLOB_APPEND_LITERAL(ob, "<a href=\"");
  if( zLink[0]=='/' ){
    /* For any hyperlink that begins with "/", make it refer to the root
    ** of the Fossil repository */
    blob_append(ob, g.zTop, -1);
  }
  html_escape(ob, blob_buffer(link), blob_size(link));
  if( title && blob_size(title)>0 ){
    BLOB_APPEND_LITERAL(ob, "\" title=\"");







|







355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
  struct Blob *link,
  struct Blob *title,
  struct Blob *content,
  void *opaque
){
  char *zLink = blob_buffer(link);
  BLOB_APPEND_LITERAL(ob, "<a href=\"");
  if( zLink && zLink[0]=='/' ){
    /* For any hyperlink that begins with "/", make it refer to the root
    ** of the Fossil repository */
    blob_append(ob, g.zTop, -1);
  }
  html_escape(ob, blob_buffer(link), blob_size(link));
  if( title && blob_size(title)>0 ){
    BLOB_APPEND_LITERAL(ob, "\" title=\"");

Changes to src/timeline.c.

1181
1182
1183
1184
1185
1186
1187


























1188
1189
1190
1191
1192
1193
1194
....
1207
1208
1209
1210
1211
1212
1213


1214
1215
1216
1217
1218
1219
1220
....
1243
1244
1245
1246
1247
1248
1249

1250
1251
1252
1253
1254
1255
1256
....
1435
1436
1437
1438
1439
1440
1441

1442
1443


1444
1445
1446
1447
1448

1449
1450
1451
1452
1453
1454
1455
....
1518
1519
1520
1521
1522
1523
1524

1525
1526
1527
1528
1529
1530
1531
....
1728
1729
1730
1731
1732
1733
1734

1735
1736
1737
1738
1739
1740
1741
    }
    assert( i<=ArraySize(az) );
  }
  if( i>2 ){
    style_submenu_multichoice("y", i/2, az, isDisabled);
  }
}



























/*
** WEBPAGE: timeline
**
** Query parameters:
**
**    a=TIMEORTAG    after this event
................................................................................
**    nd             Do not highlight the focus check-in
**    v              Show details of files changed
**    f=CHECKIN      Show family (immediate parents and children) of CHECKIN
**    from=CHECKIN   Path from...
**    to=CHECKIN       ... to this
**    shortest         ... show only the shortest path
**    uf=FILE_SHA1   Show only check-ins that contain the given file version


**    brbg           Background color from branch name
**    ubg            Background color from user
**    namechng       Show only check-ins that have filename changes
**    forks          Show only forks and their children
**    ym=YYYY-MM     Show only events for the given year/month.
**    yw=YYYY-WW     Show only events for the given week of the given year
**    ymd=YYYY-MM-DD Show only events on the given day
................................................................................
  const char *zTagName = P("t");     /* Show events with this tag */
  const char *zBrName = P("r");      /* Show events related to this tag */
  const char *zSearch = P("s");      /* Search string */
  const char *zUses = P("uf");       /* Only show check-ins hold this file */
  const char *zYearMonth = P("ym");  /* Show check-ins for the given YYYY-MM */
  const char *zYearWeek = P("yw");   /* Check-ins for YYYY-WW (week-of-year) */
  const char *zDay = P("ymd");       /* Check-ins for the day YYYY-MM-DD */

  int useDividers = P("nd")==0;      /* Show dividers if "nd" is missing */
  int renameOnly = P("namechng")!=0; /* Show only check-ins that rename files */
  int forkOnly = PB("forks");        /* Show only forks and their children */
  int bisectOnly = PB("bisect");     /* Show the check-ins of the bisect */
  int tagid;                         /* Tag ID */
  int tmFlags = 0;                   /* Timeline flags */
  const char *zThisTag = 0;          /* Suppress links to this tag */
................................................................................
    blob_append(&sql, " AND event.objid IN (0", -1);
    while( p ){
      blob_append_sql(&sql, ",%d", p->rid);
      p = p->u.pTo;
    }
    blob_append(&sql, ")", -1);
    path_reset();

    tmFlags |= TIMELINE_DISJOINT;
    db_multi_exec("%s", blob_sql_text(&sql));


    blob_appendf(&desc, "%d check-ins going from ",
                 db_int(0, "SELECT count(*) FROM timeline"));
    blob_appendf(&desc, "%z[%h]</a>", href("%R/info/%h", zFrom), zFrom);
    blob_append(&desc, " to ", -1);
    blob_appendf(&desc, "%z[%h]</a>", href("%R/info/%h",zTo), zTo);

  }else if( (p_rid || d_rid) && g.perm.Read ){
    /* If p= or d= is present, ignore all other parameters other than n= */
    char *zUuid;
    int np, nd;

    tmFlags |= TIMELINE_DISJOINT;
    if( p_rid && d_rid ){
................................................................................
  }else{
    /* Otherwise, a timeline based on a span of time */
    int n;
    const char *zEType = "timeline item";
    char *zDate;
    Blob cond;
    blob_zero(&cond);

    if( zUses ){
      blob_append_sql(&cond, " AND event.objid IN usesfile ");
    }
    if( renameOnly ){
      blob_append_sql(&cond, " AND event.objid IN rnfile ");
    }
    if( forkOnly ){
................................................................................
    if( zTagName ){
      blob_appendf(&desc, " tagged with \"%h\"", zTagName);
      tmFlags |= TIMELINE_DISJOINT;
    }else if( zBrName ){
      blob_appendf(&desc, " related to \"%h\"", zBrName);
      tmFlags |= TIMELINE_DISJOINT;
    }

    if( rAfter>0.0 ){
      if( rBefore>0.0 ){
        blob_appendf(&desc, " occurring between %h and %h.<br />",
                     zAfter, zBefore);
      }else{
        blob_appendf(&desc, " occurring on or after %h.<br />", zAfter);
      }







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>







 







>







 







>


>
>





>







 







>







 







>







1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
....
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
....
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
....
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
....
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
....
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
    }
    assert( i<=ArraySize(az) );
  }
  if( i>2 ){
    style_submenu_multichoice("y", i/2, az, isDisabled);
  }
}

/*
** If the zChng string is not NULL, then it should be a comma-separated
** list of glob patterns for filenames.  Add an term to the WHERE clause
** for the SQL statement under construction that excludes any check-in that
** does not modify one or more files matching the globs.
*/
static void addFileGlobExclusion(
  const char *zChng,        /* The filename GLOB list */
  Blob *pSql                /* The SELECT statement under construction */
){
  if( zChng==0 || zChng[0]==0 ) return;
  blob_append_sql(pSql," AND event.objid IN ("
      "SELECT mlink.mid FROM mlink, filename"
      " WHERE mlink.fnid=filename.fnid AND %s)",
      glob_expr("filename.name", zChng));
}
static void addFileGlobDescription(
  const char *zChng,        /* The filename GLOB list */
  Blob *pDescription        /* Result description */
){
  if( zChng==0 || zChng[0]==0 ) return;
  blob_appendf(pDescription, " that include changes to files matching %Q",
               zChng);
}


/*
** WEBPAGE: timeline
**
** Query parameters:
**
**    a=TIMEORTAG    after this event
................................................................................
**    nd             Do not highlight the focus check-in
**    v              Show details of files changed
**    f=CHECKIN      Show family (immediate parents and children) of CHECKIN
**    from=CHECKIN   Path from...
**    to=CHECKIN       ... to this
**    shortest         ... show only the shortest path
**    uf=FILE_SHA1   Show only check-ins that contain the given file version
**    chng=GLOBLIST  Show only check-ins that involve changes to a file whose
**                     name matches one of the comma-separate GLOBLIST.
**    brbg           Background color from branch name
**    ubg            Background color from user
**    namechng       Show only check-ins that have filename changes
**    forks          Show only forks and their children
**    ym=YYYY-MM     Show only events for the given year/month.
**    yw=YYYY-WW     Show only events for the given week of the given year
**    ymd=YYYY-MM-DD Show only events on the given day
................................................................................
  const char *zTagName = P("t");     /* Show events with this tag */
  const char *zBrName = P("r");      /* Show events related to this tag */
  const char *zSearch = P("s");      /* Search string */
  const char *zUses = P("uf");       /* Only show check-ins hold this file */
  const char *zYearMonth = P("ym");  /* Show check-ins for the given YYYY-MM */
  const char *zYearWeek = P("yw");   /* Check-ins for YYYY-WW (week-of-year) */
  const char *zDay = P("ymd");       /* Check-ins for the day YYYY-MM-DD */
  const char *zChng = P("chng");     /* List of GLOBs for files that changed */
  int useDividers = P("nd")==0;      /* Show dividers if "nd" is missing */
  int renameOnly = P("namechng")!=0; /* Show only check-ins that rename files */
  int forkOnly = PB("forks");        /* Show only forks and their children */
  int bisectOnly = PB("bisect");     /* Show the check-ins of the bisect */
  int tagid;                         /* Tag ID */
  int tmFlags = 0;                   /* Timeline flags */
  const char *zThisTag = 0;          /* Suppress links to this tag */
................................................................................
    blob_append(&sql, " AND event.objid IN (0", -1);
    while( p ){
      blob_append_sql(&sql, ",%d", p->rid);
      p = p->u.pTo;
    }
    blob_append(&sql, ")", -1);
    path_reset();
    addFileGlobExclusion(zChng, &sql);
    tmFlags |= TIMELINE_DISJOINT;
    db_multi_exec("%s", blob_sql_text(&sql));
    style_submenu_binary("v","With Files","Without Files",
                         zType[0]!='a' && zType[0]!='c');
    blob_appendf(&desc, "%d check-ins going from ",
                 db_int(0, "SELECT count(*) FROM timeline"));
    blob_appendf(&desc, "%z[%h]</a>", href("%R/info/%h", zFrom), zFrom);
    blob_append(&desc, " to ", -1);
    blob_appendf(&desc, "%z[%h]</a>", href("%R/info/%h",zTo), zTo);
    addFileGlobDescription(zChng, &desc);
  }else if( (p_rid || d_rid) && g.perm.Read ){
    /* If p= or d= is present, ignore all other parameters other than n= */
    char *zUuid;
    int np, nd;

    tmFlags |= TIMELINE_DISJOINT;
    if( p_rid && d_rid ){
................................................................................
  }else{
    /* Otherwise, a timeline based on a span of time */
    int n;
    const char *zEType = "timeline item";
    char *zDate;
    Blob cond;
    blob_zero(&cond);
    addFileGlobExclusion(zChng, &cond);
    if( zUses ){
      blob_append_sql(&cond, " AND event.objid IN usesfile ");
    }
    if( renameOnly ){
      blob_append_sql(&cond, " AND event.objid IN rnfile ");
    }
    if( forkOnly ){
................................................................................
    if( zTagName ){
      blob_appendf(&desc, " tagged with \"%h\"", zTagName);
      tmFlags |= TIMELINE_DISJOINT;
    }else if( zBrName ){
      blob_appendf(&desc, " related to \"%h\"", zBrName);
      tmFlags |= TIMELINE_DISJOINT;
    }
    addFileGlobDescription(zChng, &desc);
    if( rAfter>0.0 ){
      if( rBefore>0.0 ){
        blob_appendf(&desc, " occurring between %h and %h.<br />",
                     zAfter, zBefore);
      }else{
        blob_appendf(&desc, " occurring on or after %h.<br />", zAfter);
      }

Changes to src/user.c.

95
96
97
98
99
100
101





































































102
103
104
105
106
107
108
...
115
116
117
118
119
120
121

122
123
124
125
126
127
128
129
130
131
132
133






134
135
136
137
138
139
140
...
178
179
180
181
182
183
184

185
186
187
188
189
190
191
...
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
...
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
# include <io.h>
# include <fcntl.h>
# undef popen
# define popen _popen
# undef pclose
# define pclose _pclose
#endif






































































/*
** Do a single prompt for a passphrase.  Store the results in the blob.
**
** If the FOSSIL_PWREADER environment variable is set, then it will
** be the name of a program that prompts the user for their password/
** passphrase in a secure manner.  The program should take one or more
................................................................................
**
** The return value is a pointer to a static buffer that is overwritten
** on subsequent calls to this same routine.
*/
static void prompt_for_passphrase(const char *zPrompt, Blob *pPassphrase){
  char *z;
  const char *zProg = fossil_getenv("FOSSIL_PWREADER");

  if( zProg && zProg[0] ){
    static char zPass[100];
    Blob cmd;
    FILE *in;
    blob_zero(&cmd);
    blob_appendf(&cmd, "%s \"Fossil Passphrase\" \"%s\"", zProg, zPrompt);
    zPass[0] = 0;
    in = popen(blob_str(&cmd), "r");
    fgets(zPass, sizeof(zPass), in);
    pclose(in);
    blob_reset(&cmd);
    z = zPass;






  }else{
    z = getpass(zPrompt);
  }
  strip_string(pPassphrase, z);
}

/*
................................................................................
int save_password_prompt(const char *passwd){
  Blob x;
  char c;
  const char *old = db_get("last-sync-pw", 0);
  if( (old!=0) && fossil_strcmp(unobscure(old), passwd)==0 ){
     return 0;
  }

  prompt_user("remember password (Y/n)? ", &x);
  c = blob_str(&x)[0];
  blob_reset(&x);
  return ( c!='n' && c!='N' );
}

/*
................................................................................
  blob_append_sql(&sql,"  ORDER BY rowid DESC LIMIT %d OFFSET %d", n+1, skip);
  if( skip ){
    style_submenu_element("Newer", "Newer entries",
              "%s/access_log?o=%d&n=%d&y=%d", g.zTop, skip>=n ? skip-n : 0,
              n, y);
  }
  rc = db_prepare_ignore_error(&q, "%s", blob_sql_text(&sql));
  @ <center>
  fLogEnabled = db_get_boolean("access-log", 0);
  @ <div>Access logging is %s(fLogEnabled?"on":"off").
  @ (Change this on the <a href="setup_settings">settings</a> page.)</div>
  @ <table border="1" cellpadding="5" id='logtable'>
  @ <thead><tr><th width="33%%">Date</th><th width="34%%">User</th>
  @ <th width="33%%">IP Address</th></tr></thead><tbody>
  while( rc==SQLITE_OK && db_step(&q)==SQLITE_ROW ){
    const char *zName = db_column_text(&q, 0);
    const char *zIP = db_column_text(&q, 1);
    const char *zDate = db_column_text(&q, 2);
    int bSuccess = db_column_int(&q, 3);
................................................................................
    }
    @ <td>%s(zDate)</td><td>%h(zName)</td><td>%h(zIP)</td></tr>
  }
  if( skip>0 || cnt>n ){
    style_submenu_element("All", "All entries",
          "%s/access_log?n=10000000", g.zTop);
  }
  @ </tbody></table></center>
  db_finalize(&q);
  @ <hr />
  @ <form method="post" action="%s(g.zTop)/access_log">
  @ <label><input type="checkbox" name="delold">
  @ Delete all but the most recent 200 entries</input></label>
  @ <input type="submit" name="deloldbtn" value="Delete"></input>
  @ </form>







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>












>
>
>
>
>
>







 







>







 







<

|

|







 







|







95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
...
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
...
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
...
615
616
617
618
619
620
621

622
623
624
625
626
627
628
629
630
631
632
...
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
# include <io.h>
# include <fcntl.h>
# undef popen
# define popen _popen
# undef pclose
# define pclose _pclose
#endif

/*
** Scramble substitution matrix:
*/
static char aSubst[256];

/*
** Descramble the password
*/
static void userDescramble(char *z){
  int i;
  for(i=0; z[i]; i++) z[i] = aSubst[(unsigned char)z[i]];
}

/* Print a string in 5-letter groups */
static void printFive(const unsigned char *z){
  int i;
  for(i=0; z[i]; i++){
    if( i>0 && (i%5)==0 ) putchar(' ');
    putchar(z[i]);
  }
  putchar('\n');
}

/* Return a pseudo-random integer between 0 and N-1 */
static int randint(int N){
  unsigned char x;
  assert( N<256 );
  sqlite3_randomness(1, &x);
  return x % N;
}

/*
** Generate and print a random scrambling of letters a through z (omitting x)
** and set up the aSubst[] matrix to descramble.
*/
static void userGenerateScrambleCode(void){
  unsigned char zOrig[30];
  unsigned char zA[30];
  unsigned char zB[30];
  int nA = 25;
  int nB = 0;
  int i;
  memcpy(zOrig, "abcdefghijklmnopqrstuvwyz", nA+1);
  memcpy(zA, zOrig, nA+1);
  assert( nA==(int)strlen((char*)zA) );
  for(i=0; i<sizeof(aSubst); i++) aSubst[i] = i;
  printFive(zA);
  while( nA>0 ){
    int x = randint(nA);
    zB[nB++] = zA[x];
    zA[x] = zA[--nA];
  }
  assert( nB==25 );
  zB[nB] = 0;
  printFive(zB);
  for(i=0; i<nB; i++) aSubst[zB[i]] = zOrig[i];
}

/*
** Return the value of the FOSSIL_SECURITY_LEVEL environment variable.
** Or return 0 if that variable does not exist.
*/
int fossil_security_level(void){
  const char *zLevel = fossil_getenv("FOSSIL_SECURITY_LEVEL");
  if( zLevel==0 ) return 0;
  return atoi(zLevel);
}


/*
** Do a single prompt for a passphrase.  Store the results in the blob.
**
** If the FOSSIL_PWREADER environment variable is set, then it will
** be the name of a program that prompts the user for their password/
** passphrase in a secure manner.  The program should take one or more
................................................................................
**
** The return value is a pointer to a static buffer that is overwritten
** on subsequent calls to this same routine.
*/
static void prompt_for_passphrase(const char *zPrompt, Blob *pPassphrase){
  char *z;
  const char *zProg = fossil_getenv("FOSSIL_PWREADER");
  const char *zSecure;
  if( zProg && zProg[0] ){
    static char zPass[100];
    Blob cmd;
    FILE *in;
    blob_zero(&cmd);
    blob_appendf(&cmd, "%s \"Fossil Passphrase\" \"%s\"", zProg, zPrompt);
    zPass[0] = 0;
    in = popen(blob_str(&cmd), "r");
    fgets(zPass, sizeof(zPass), in);
    pclose(in);
    blob_reset(&cmd);
    z = zPass;
  }else if( fossil_security_level()>=2 ){
    userGenerateScrambleCode();
    z = getpass(zPrompt);
    if( z ) userDescramble(z);
    printf("\033[3A\033[J");  /* Erase previous three lines */
    fflush(stdout);
  }else{
    z = getpass(zPrompt);
  }
  strip_string(pPassphrase, z);
}

/*
................................................................................
int save_password_prompt(const char *passwd){
  Blob x;
  char c;
  const char *old = db_get("last-sync-pw", 0);
  if( (old!=0) && fossil_strcmp(unobscure(old), passwd)==0 ){
     return 0;
  }
  if( fossil_security_level()>=1 ) return 0;
  prompt_user("remember password (Y/n)? ", &x);
  c = blob_str(&x)[0];
  blob_reset(&x);
  return ( c!='n' && c!='N' );
}

/*
................................................................................
  blob_append_sql(&sql,"  ORDER BY rowid DESC LIMIT %d OFFSET %d", n+1, skip);
  if( skip ){
    style_submenu_element("Newer", "Newer entries",
              "%s/access_log?o=%d&n=%d&y=%d", g.zTop, skip>=n ? skip-n : 0,
              n, y);
  }
  rc = db_prepare_ignore_error(&q, "%s", blob_sql_text(&sql));

  fLogEnabled = db_get_boolean("access-log", 0);
  @ <div align="center">Access logging is %s(fLogEnabled?"on":"off").
  @ (Change this on the <a href="setup_settings">settings</a> page.)</div>
  @ <table border="1" cellpadding="5" id="logtable" align="center">
  @ <thead><tr><th width="33%%">Date</th><th width="34%%">User</th>
  @ <th width="33%%">IP Address</th></tr></thead><tbody>
  while( rc==SQLITE_OK && db_step(&q)==SQLITE_ROW ){
    const char *zName = db_column_text(&q, 0);
    const char *zIP = db_column_text(&q, 1);
    const char *zDate = db_column_text(&q, 2);
    int bSuccess = db_column_int(&q, 3);
................................................................................
    }
    @ <td>%s(zDate)</td><td>%h(zName)</td><td>%h(zIP)</td></tr>
  }
  if( skip>0 || cnt>n ){
    style_submenu_element("All", "All entries",
          "%s/access_log?n=10000000", g.zTop);
  }
  @ </tbody></table>
  db_finalize(&q);
  @ <hr />
  @ <form method="post" action="%s(g.zTop)/access_log">
  @ <label><input type="checkbox" name="delold">
  @ Delete all but the most recent 200 entries</input></label>
  @ <input type="submit" name="deloldbtn" value="Delete"></input>
  @ </form>

Changes to www/branching.wiki.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
..
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
..
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
...
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
...
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
<title>Branching, Forking, Merging, and Tagging</title>
<h2>Background</h2>

In a simple and perfect world, the development of a project would proceed
linearly, as shown in figure 1.

<center><table border=1 cellpadding=10 hspace=10 vspace=10>
<tr><td align="center">
<img src="branch01.gif" width=280 height=68><br>
Figure 1
</td></tr></table></center>

Each circle represents a check-in.  For the sake of clarity, the check-ins
are given small consecutive numbers.  In a real system, of course, the
check-in numbers would be 40-character SHA1 hashes since it is not possible
to allocate collision-free sequential numbers in a distributed system.
But as sequential numbers are easier to read, we will substitute them for
the 40-character SHA1 hashes in this document.
................................................................................
it has no descendants.  (We will give a more precise definition later of
"leaf.")

Alas, reality often interferes with the simple linear development of a
project.  Suppose two programmers make independent modifications to check-in 2.
After both changes are committed, the check-in graph looks like figure 2:

<center><table border=1 cellpadding=10 hspace=10 vspace=10>
<tr><td align="center">
<img src="branch02.gif" width=210 height=140><br>
Figure 2
</td></tr></table></center>

The graph in figure 2 has two leaves: check-ins 3 and 4.  Check-in 2 has
two children, check-ins 3 and 4.  We call this state a <i>fork</i>.

Fossil tries to prevent forks. Suppose two programmers named Alice and
Bob are each editing check-in 2 separately. Alice finishes her edits
first and commits her changes, resulting in check-in 3. Later, when Bob
................................................................................
graphs with a single leaf.

To resolve this situation, Alice can use the fossil <b>merge</b> command
to merge in Bob's changes in her local copy of check-in 3.  Then she
can commit the results as check-in 5.  This results in a DAG as shown
in figure 3.

<center><table border=1 cellpadding=10 hspace=10 vspace=10>
<tr><td align="center">
<img src="branch03.gif" width=282 height=152><br>
Figure 3
</td></tr></table></center>

Check-in 5 is a child of check-in 3 because it was created by editing
check-in 3.  But check-in 5 also inherits the changes from check-in 4 by
virtue of the merge.  So we say that check-in 5 is a <i>merge child</i>
of check-in 4 and that it is a <i>direct child</i> of check-in 3.
The graph is now back to a single leaf (check-in 5).

................................................................................
development and another leaf that is the latest version that has been
tested.
When multiple leaves are desirable, we call this <i>branching</i>
instead of <i>forking</i>.
Figure 4 shows an example of a project where there are two branches, one
for development work and another for testing.

<center><table border=1 cellpadding=10 hspace=10 vspace=10>
<tr><td align="center">
<img src="branch04.gif" width=426 height=123><br>
Figure 4
</td></tr></table></center>

The hypothetical scenario of figure 4 is this:  The project starts and
progresses to a point where (at check-in 2)
it is ready to enter testing for its first release.
In a real project, of course, there might be hundreds or thousands of
check-ins before a project reaches this point, but for simplicity of
presentation we will say that the project is ready after check-in 2.
................................................................................
<a name="tags"></a>
<h2>Tags And Properties</h2>

Tags and properties are used in fossil to help express the intent, and
thus to distinguish between forks and branches.  Figure 5 shows the
same scenario as figure 4 but with tags and properties added:

<center><table border=1 cellpadding=10 hspace=10 vspace=10>
<tr><td align="center">
<img src="branch05.gif" width=485 height=177><br>
Figure 5
</td></tr></table></center>

A <i>tag</i> is a name that is attached to a check-in.  A
<i>property</i> is a name/value pair.  Internally, fossil implements
tags as properties with a NULL value.  So, tags and properties really
are much the same thing, and henceforth we will use the word "tag"
to mean either a tag or a property.







|



|







 







|



|







 







|



|







 







|



|







 







|



|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
..
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
..
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
...
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
...
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
<title>Branching, Forking, Merging, and Tagging</title>
<h2>Background</h2>

In a simple and perfect world, the development of a project would proceed
linearly, as shown in figure 1.

<table border=1 cellpadding=10 hspace=10 vspace=10 align="center">
<tr><td align="center">
<img src="branch01.gif" width=280 height=68><br>
Figure 1
</td></tr></table>

Each circle represents a check-in.  For the sake of clarity, the check-ins
are given small consecutive numbers.  In a real system, of course, the
check-in numbers would be 40-character SHA1 hashes since it is not possible
to allocate collision-free sequential numbers in a distributed system.
But as sequential numbers are easier to read, we will substitute them for
the 40-character SHA1 hashes in this document.
................................................................................
it has no descendants.  (We will give a more precise definition later of
"leaf.")

Alas, reality often interferes with the simple linear development of a
project.  Suppose two programmers make independent modifications to check-in 2.
After both changes are committed, the check-in graph looks like figure 2:

<table border=1 cellpadding=10 hspace=10 vspace=10 align="center">
<tr><td align="center">
<img src="branch02.gif" width=210 height=140><br>
Figure 2
</td></tr></table>

The graph in figure 2 has two leaves: check-ins 3 and 4.  Check-in 2 has
two children, check-ins 3 and 4.  We call this state a <i>fork</i>.

Fossil tries to prevent forks. Suppose two programmers named Alice and
Bob are each editing check-in 2 separately. Alice finishes her edits
first and commits her changes, resulting in check-in 3. Later, when Bob
................................................................................
graphs with a single leaf.

To resolve this situation, Alice can use the fossil <b>merge</b> command
to merge in Bob's changes in her local copy of check-in 3.  Then she
can commit the results as check-in 5.  This results in a DAG as shown
in figure 3.

<table border=1 cellpadding=10 hspace=10 vspace=10 align="center">
<tr><td align="center">
<img src="branch03.gif" width=282 height=152><br>
Figure 3
</td></tr></table>

Check-in 5 is a child of check-in 3 because it was created by editing
check-in 3.  But check-in 5 also inherits the changes from check-in 4 by
virtue of the merge.  So we say that check-in 5 is a <i>merge child</i>
of check-in 4 and that it is a <i>direct child</i> of check-in 3.
The graph is now back to a single leaf (check-in 5).

................................................................................
development and another leaf that is the latest version that has been
tested.
When multiple leaves are desirable, we call this <i>branching</i>
instead of <i>forking</i>.
Figure 4 shows an example of a project where there are two branches, one
for development work and another for testing.

<table border=1 cellpadding=10 hspace=10 vspace=10 align="center">
<tr><td align="center">
<img src="branch04.gif" width=426 height=123><br>
Figure 4
</td></tr></table>

The hypothetical scenario of figure 4 is this:  The project starts and
progresses to a point where (at check-in 2)
it is ready to enter testing for its first release.
In a real project, of course, there might be hundreds or thousands of
check-ins before a project reaches this point, but for simplicity of
presentation we will say that the project is ready after check-in 2.
................................................................................
<a name="tags"></a>
<h2>Tags And Properties</h2>

Tags and properties are used in fossil to help express the intent, and
thus to distinguish between forks and branches.  Figure 5 shows the
same scenario as figure 4 but with tags and properties added:

<table border=1 cellpadding=10 hspace=10 vspace=10 align="center">
<tr><td align="center">
<img src="branch05.gif" width=485 height=177><br>
Figure 5
</td></tr></table>

A <i>tag</i> is a name that is attached to a check-in.  A
<i>property</i> is a name/value pair.  Internally, fossil implements
tags as properties with a NULL value.  So, tags and properties really
are much the same thing, and henceforth we will use the word "tag"
to mean either a tag or a property.

Changes to www/copyright-release.html.

72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
..
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
</ul>
</ol>

<p>By filling in the following information and signing your name,
you agree to be bound by all of the terms
set forth in this agreement.  Please print clearly.</p>

<center>
<p><table width="80%" border="1" cellpadding="0" cellspacing="0">
<tr><td width="20%" valign="top">Your name &amp email:</td><td width="80%">

    <!-- Replace this line with your name and email --> &nbsp;<p>&nbsp;

</td></tr>
<tr><td valign="top">Company name:<br>(if applicable)</td><td>

................................................................................
    <!-- Replace this line and the next line with your postal address -->
    <p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p>

</td></tr>
<tr><td valign="top">Signature:</td><td>&nbsp;<p>&nbsp;</td></tr>
<tr><td valign="top">Date:</td><td>&nbsp;<p>&nbsp;</td></tr>
</table>
</center>

<p>Send completed forms to:
<blockquote>
Hipp, Wyrick &amp; Company, Inc.<br>
6200 Maple Cove Lane<br>
Charlotte, NC 28269-1086<br>
USA
</p>







<
|







 







<








72
73
74
75
76
77
78

79
80
81
82
83
84
85
86
..
92
93
94
95
96
97
98

99
100
101
102
103
104
105
106
</ul>
</ol>

<p>By filling in the following information and signing your name,
you agree to be bound by all of the terms
set forth in this agreement.  Please print clearly.</p>


<p><table width="80%" border="1" cellpadding="0" cellspacing="0" align="center">
<tr><td width="20%" valign="top">Your name &amp email:</td><td width="80%">

    <!-- Replace this line with your name and email --> &nbsp;<p>&nbsp;

</td></tr>
<tr><td valign="top">Company name:<br>(if applicable)</td><td>

................................................................................
    <!-- Replace this line and the next line with your postal address -->
    <p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p>

</td></tr>
<tr><td valign="top">Signature:</td><td>&nbsp;<p>&nbsp;</td></tr>
<tr><td valign="top">Date:</td><td>&nbsp;<p>&nbsp;</td></tr>
</table>


<p>Send completed forms to:
<blockquote>
Hipp, Wyrick &amp; Company, Inc.<br>
6200 Maple Cove Lane<br>
Charlotte, NC 28269-1086<br>
USA
</p>

Changes to www/fossil-v-git.wiki.

17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
..
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
..
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
..
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
...
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
...
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
...
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
...
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288

&#185;<small><i>Git does not support
wiki, tickets, or tech-notes, so those elements will not transfer when
exporting from Fossil to Git.</i></small>

<h2>2.0 Executive Summary:</h2>

<blockquote><center><table border=1 cellpadding=5>
<tr><th width="50%">GIT</th><th width="50%">FOSSIL</th></tr>
<tr><td>File versioning only</td>
    <td>Versioning, Tickets, Wiki, and Technotes</td></tr>
<tr><td>Ad-hoc, pile-of-files key/value database</td>
    <td>Relational SQL database</td></tr>
<tr><td>Bazaar-style development</td><td>Cathedral-style development</td></tr>
<tr><td>Designed for Linux development</td>
................................................................................
    <td>Designed for SQLite development</td></tr>
<tr><td>Lots of little tools</td><td>Stand-alone executable</td></tr>
<tr><td>One check-out per repository</td>
    <td>Many check-outs per repository</td></tr>
<tr><td>Remembers what you should have done</td>
    <td>Remembers what you actually did</td></tr>
<tr><td>GPL</td><td>BSD</td></tr>
</table></center></blockquote>

<h2>3.0 Discussion</h2>

<h3>3.1 Feature Set</h3>

Git provides file versioning services only, whereas Fossil adds
integrated [./wikitheory.wiki | wiki],
................................................................................
If you clone Fossil's self-hosting repository, you get the entire
Fossil website - source code, documentation, ticket history, and so forth.

For developers who choose to self-host projects (rather than using a
3rd-party service such as GitHub) Fossil is much easier to set up, since
the stand-alone Fossil executable together with a 2-line CGI script
suffice to instantiate a full-featured developer website.  To accomplish
the same using Git requires locating, installing, configuring, integrating, 
and managing a wide assortment of separate tools.  Standing up a developer
website using Fossil can be done in minutes, whereas doing the same using
Git requires hours or days.

<h3>3.2 Database</h3>

The baseline data structures for Fossil and Git are the same (modulo
formatting details).  Both systems store check-ins as immutable
objects referencing their immediate ancestors and named by their SHA1 hash.

The difference is that Git stores its objects as individual files 
in the ".git" folder or compressed into
bespoke "pack-files", whereas Fossil stores its objects in a 
relational ([https://www.sqlite.org/|SQLite]) database file.  To put it
another way, Git uses an ad-hoc pile-of-files key/value database whereas
Fossil uses a proven, general-purpose SQL database.  This
difference is more than an implementation detail.  It
has important consequences.

With Git, one can easily locate the ancestors of a particular check-in
................................................................................
difficult to go the other direction and locate the descendants of a
check-in.  It is so difficult, in fact, that neither native Git nor
GitHub provide this capability.  With Git, if you are looking at some
historical check-in then you cannot ask
"what came next" or "what are the children of this check-in".

Fossil, on the other hand, parses essential information about check-ins
(parents, children, committers, comments, files changed, etc.) 
into a relational database that can be easily 
queried using concise SQL statements to find both ancestors and 
descendents of a check-in.

Leaf check-ins in Git that lack a "ref" become "detached", making them
difficult to locate and subject to garbage collection.  This
"detached head" problem has caused untold grief for countless
Git users.  With Fossil, all check-ins are easily located using
a variety of attributes (parents, children, committer, date, full-text
search of the check-in comment) and so detached heads are simply not possible.

The ease with which check-ins can be located and queried in Fossil
has resulted in a huge variety of reports and status screens 
([./webpage-ex.md|examples]) that show project state
in ways that help developers
maintain enhanced awareness and comprehension
and avoid errors.

<h3>3.3 Cathedral vs. Bazaar</h3>

Fossil and Git promote different development styles.  Git promotes a
"bazaar" development style in which numerous anonymous developers make
small and sometimes haphazard contributions.  Fossil
promotes a "cathedral" development model in which the project is
closely supervised by an highly engaged architect and implemented by 
a clique of developers.

Nota Bene:  This is not to say that Git cannot be used for cathedral-style
development or that Fossil cannot be used for bazaar-style development.
They can be.  But those modes are not their design intent nor the their
low-friction path.

................................................................................
<h3>3.5 Lots of little tools vs. Self-contained system</h3>

Git consists of many small tools, each doing one small part of the job,
which can be recombined (by experts) to perform powerful operations.
Git has a lot of complexity and many dependencies and requires an "installer"
script or program to get it running.

Fossil is a single self-contained stand-alone executable with hardly 
any dependencies.  Fossil can be (and often is) run inside a 
minimally configured chroot jail.  To install Fossil,
one merely puts the executable on $PATH.

The designer of Git says that the unix philosophy is to have lots of 
small tools that collaborate to get the job done.  The designer of 
Fossil says that the unix philosophy is "it just works".  Both 
individuals have written their DVCSes to reflect their own view 
of the "unix philosophy".

<h3>3.6 One vs. Many Check-outs per Repository</h3>

A "repository" in Git is a pile-of-files in the ".git" subdirectory
of a single check-out.  The check-out and the repository are inseperable.

With Fossil, a "repository" is a single SQLite database file 	
that can be stored anywhere.  There
can be multiple active check-outs from the same repository, perhaps
open on different branches or on different snapshots of the same branch.
Long-running tests or builds can be running in one check-out while
changes are being committed in another.

<h3>3.7 What you should have done vs. What you actually did</h3>
................................................................................
the development of a project should have looked like had there been no
mistakes.

Fossil, in contrast, puts more emphasis on recording exactly what happened,
including all of the messy errors, dead-ends, experimental branches, and
so forth.  One might argue that this
makes the history of a Fossil project "messy".  But another point of view
is that this makes the history "accurate".  In actual practice, the 
superior reporting tools available in Fossil mean that the added "mess"
is not a factor.

One commentator has mused that Git records history according to
the victors, whereas Fossil records history as it actually happened.

<h3>3.8 GPL vs. BSD</h3>

Git is covered by the GPL license whereas Fossil is covered by 
a two-clause BSD license.

Consider the difference between GPL and BSD licenses:  GPL is designed
to make writing easier at the expense of making reading harder.  BSD is
designed to make reading easier and the expense of making writing harder.

To a first approximation, the GPL license grants the right to read 
source code to anyone who promises to give back enhancements.  In other
words, the act of reading GPL source code (a prerequiste for making changes)
implies acceptance of the license which requires updates to be contributed
back under the same license.  (The details are more complex, but the
foregoing captures the essence of the idea.)  A big advantage of the GPL
is that anybody can contribute to the code without having to sign additional
legal documentation because they have implied their acceptance of the GPL
................................................................................
cliquish, cathedral-style approach more typical of BSD-licensed projects.

<h2>4.0 Missing Features</h2>

Most of the capabilities found in Git are also available in Fossil and
the other way around. For example, both systems have local check-outs,
remote repositories, push/pull/sync, bisect capabilities, and a "stash".
Both systems store project history as a directed acyclic graph (DAG) 
of immutable check-in objects.

But there are a few capabilities in one system that are missing from the
other.

<h3>4.1 Features found in Fossil but missing from Git</h3>

................................................................................
  *  <b>Wiki, Embedded documentation, Trouble-tickets, and Tech-Notes</b>

   Git only provides versioning of source code.  Fossil strives to provide
   other related configuration management services as well.

  *  <b>Named branches</b>

   Branches in Fossil have persistent names that are propagated 
   to collaborators via [/help?cmd=push|push] and [/help?cmd=pull|pull].
   All developers see the same name on the same branch.  Git, in contrast,
   uses only local branch names, so developers working on the
   same project can (and frequently do) use a different name for the
   same branch.  

  *  <b>The [/help?cmd=all|fossil all] command</b>

   Fossil keeps track of all repositories and check-outs and allows
   operations over all of them with a single command.  For example, in
   Fossil is possible to request a pull of all repositories on a laptop
   from their respective servers, prior to taking the laptop off network.







|







 







|







 







|










|

|







 







|
|
|










|











|







 







|
|



|
|
|
|







|







 







|








|






|







 







|







 







|




|







17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
..
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
..
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
..
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
...
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
...
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
...
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
...
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288

&#185;<small><i>Git does not support
wiki, tickets, or tech-notes, so those elements will not transfer when
exporting from Fossil to Git.</i></small>

<h2>2.0 Executive Summary:</h2>

<blockquote><table border=1 cellpadding=5 align=center>
<tr><th width="50%">GIT</th><th width="50%">FOSSIL</th></tr>
<tr><td>File versioning only</td>
    <td>Versioning, Tickets, Wiki, and Technotes</td></tr>
<tr><td>Ad-hoc, pile-of-files key/value database</td>
    <td>Relational SQL database</td></tr>
<tr><td>Bazaar-style development</td><td>Cathedral-style development</td></tr>
<tr><td>Designed for Linux development</td>
................................................................................
    <td>Designed for SQLite development</td></tr>
<tr><td>Lots of little tools</td><td>Stand-alone executable</td></tr>
<tr><td>One check-out per repository</td>
    <td>Many check-outs per repository</td></tr>
<tr><td>Remembers what you should have done</td>
    <td>Remembers what you actually did</td></tr>
<tr><td>GPL</td><td>BSD</td></tr>
</table></blockquote>

<h2>3.0 Discussion</h2>

<h3>3.1 Feature Set</h3>

Git provides file versioning services only, whereas Fossil adds
integrated [./wikitheory.wiki | wiki],
................................................................................
If you clone Fossil's self-hosting repository, you get the entire
Fossil website - source code, documentation, ticket history, and so forth.

For developers who choose to self-host projects (rather than using a
3rd-party service such as GitHub) Fossil is much easier to set up, since
the stand-alone Fossil executable together with a 2-line CGI script
suffice to instantiate a full-featured developer website.  To accomplish
the same using Git requires locating, installing, configuring, integrating,
and managing a wide assortment of separate tools.  Standing up a developer
website using Fossil can be done in minutes, whereas doing the same using
Git requires hours or days.

<h3>3.2 Database</h3>

The baseline data structures for Fossil and Git are the same (modulo
formatting details).  Both systems store check-ins as immutable
objects referencing their immediate ancestors and named by their SHA1 hash.

The difference is that Git stores its objects as individual files
in the ".git" folder or compressed into
bespoke "pack-files", whereas Fossil stores its objects in a
relational ([https://www.sqlite.org/|SQLite]) database file.  To put it
another way, Git uses an ad-hoc pile-of-files key/value database whereas
Fossil uses a proven, general-purpose SQL database.  This
difference is more than an implementation detail.  It
has important consequences.

With Git, one can easily locate the ancestors of a particular check-in
................................................................................
difficult to go the other direction and locate the descendants of a
check-in.  It is so difficult, in fact, that neither native Git nor
GitHub provide this capability.  With Git, if you are looking at some
historical check-in then you cannot ask
"what came next" or "what are the children of this check-in".

Fossil, on the other hand, parses essential information about check-ins
(parents, children, committers, comments, files changed, etc.)
into a relational database that can be easily
queried using concise SQL statements to find both ancestors and
descendents of a check-in.

Leaf check-ins in Git that lack a "ref" become "detached", making them
difficult to locate and subject to garbage collection.  This
"detached head" problem has caused untold grief for countless
Git users.  With Fossil, all check-ins are easily located using
a variety of attributes (parents, children, committer, date, full-text
search of the check-in comment) and so detached heads are simply not possible.

The ease with which check-ins can be located and queried in Fossil
has resulted in a huge variety of reports and status screens
([./webpage-ex.md|examples]) that show project state
in ways that help developers
maintain enhanced awareness and comprehension
and avoid errors.

<h3>3.3 Cathedral vs. Bazaar</h3>

Fossil and Git promote different development styles.  Git promotes a
"bazaar" development style in which numerous anonymous developers make
small and sometimes haphazard contributions.  Fossil
promotes a "cathedral" development model in which the project is
closely supervised by an highly engaged architect and implemented by
a clique of developers.

Nota Bene:  This is not to say that Git cannot be used for cathedral-style
development or that Fossil cannot be used for bazaar-style development.
They can be.  But those modes are not their design intent nor the their
low-friction path.

................................................................................
<h3>3.5 Lots of little tools vs. Self-contained system</h3>

Git consists of many small tools, each doing one small part of the job,
which can be recombined (by experts) to perform powerful operations.
Git has a lot of complexity and many dependencies and requires an "installer"
script or program to get it running.

Fossil is a single self-contained stand-alone executable with hardly
any dependencies.  Fossil can be (and often is) run inside a
minimally configured chroot jail.  To install Fossil,
one merely puts the executable on $PATH.

The designer of Git says that the unix philosophy is to have lots of
small tools that collaborate to get the job done.  The designer of
Fossil says that the unix philosophy is "it just works".  Both
individuals have written their DVCSes to reflect their own view
of the "unix philosophy".

<h3>3.6 One vs. Many Check-outs per Repository</h3>

A "repository" in Git is a pile-of-files in the ".git" subdirectory
of a single check-out.  The check-out and the repository are inseperable.

With Fossil, a "repository" is a single SQLite database file
that can be stored anywhere.  There
can be multiple active check-outs from the same repository, perhaps
open on different branches or on different snapshots of the same branch.
Long-running tests or builds can be running in one check-out while
changes are being committed in another.

<h3>3.7 What you should have done vs. What you actually did</h3>
................................................................................
the development of a project should have looked like had there been no
mistakes.

Fossil, in contrast, puts more emphasis on recording exactly what happened,
including all of the messy errors, dead-ends, experimental branches, and
so forth.  One might argue that this
makes the history of a Fossil project "messy".  But another point of view
is that this makes the history "accurate".  In actual practice, the
superior reporting tools available in Fossil mean that the added "mess"
is not a factor.

One commentator has mused that Git records history according to
the victors, whereas Fossil records history as it actually happened.

<h3>3.8 GPL vs. BSD</h3>

Git is covered by the GPL license whereas Fossil is covered by
a two-clause BSD license.

Consider the difference between GPL and BSD licenses:  GPL is designed
to make writing easier at the expense of making reading harder.  BSD is
designed to make reading easier and the expense of making writing harder.

To a first approximation, the GPL license grants the right to read
source code to anyone who promises to give back enhancements.  In other
words, the act of reading GPL source code (a prerequiste for making changes)
implies acceptance of the license which requires updates to be contributed
back under the same license.  (The details are more complex, but the
foregoing captures the essence of the idea.)  A big advantage of the GPL
is that anybody can contribute to the code without having to sign additional
legal documentation because they have implied their acceptance of the GPL
................................................................................
cliquish, cathedral-style approach more typical of BSD-licensed projects.

<h2>4.0 Missing Features</h2>

Most of the capabilities found in Git are also available in Fossil and
the other way around. For example, both systems have local check-outs,
remote repositories, push/pull/sync, bisect capabilities, and a "stash".
Both systems store project history as a directed acyclic graph (DAG)
of immutable check-in objects.

But there are a few capabilities in one system that are missing from the
other.

<h3>4.1 Features found in Fossil but missing from Git</h3>

................................................................................
  *  <b>Wiki, Embedded documentation, Trouble-tickets, and Tech-Notes</b>

   Git only provides versioning of source code.  Fossil strives to provide
   other related configuration management services as well.

  *  <b>Named branches</b>

   Branches in Fossil have persistent names that are propagated
   to collaborators via [/help?cmd=push|push] and [/help?cmd=pull|pull].
   All developers see the same name on the same branch.  Git, in contrast,
   uses only local branch names, so developers working on the
   same project can (and frequently do) use a different name for the
   same branch.

  *  <b>The [/help?cmd=all|fossil all] command</b>

   Fossil keeps track of all repositories and check-outs and allows
   operations over all of them with a single command.  For example, in
   Fossil is possible to request a pull of all repositories on a laptop
   from their respective servers, prior to taking the laptop off network.

Changes to www/index.wiki.

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<li> [http://www.fossil-scm.org/schimpf-book/home | Jim Schimpf's book]
<li> Mailing list
     <ul>
     <li> [http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/fossil-users | sign-up]
     <li> [http://www.mail-archive.com/fossil-users@lists.fossil-scm.org | archives]
     </ul>
</ul>
<center><img src="fossil3.gif"></center>
</div>

<p>Fossil is a simple, high-reliability, distributed software configuration
management system with these advanced features:

  1.  <b>Integrated Bug Tracking, Wiki, and Technotes</b> -
      In addition to doing [./concepts.wiki | distributed version control]







|







16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<li> [http://www.fossil-scm.org/schimpf-book/home | Jim Schimpf's book]
<li> Mailing list
     <ul>
     <li> [http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/fossil-users | sign-up]
     <li> [http://www.mail-archive.com/fossil-users@lists.fossil-scm.org | archives]
     </ul>
</ul>
<img src="fossil3.gif" align="center">
</div>

<p>Fossil is a simple, high-reliability, distributed software configuration
management system with these advanced features:

  1.  <b>Integrated Bug Tracking, Wiki, and Technotes</b> -
      In addition to doing [./concepts.wiki | distributed version control]

Changes to www/sync.wiki.

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
...
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
...
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
...
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
...
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
...
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
...
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
collection of artifacts.  Each artifact is identified by its SHA1 hash
expressed as a 40-character lower-case hexadecimal string.
Synchronization is simply the process of sharing artifacts between
servers so that all servers have copies of all artifacts.  Because
artifacts are unordered, the order in which artifacts are received
at a server is inconsequential.  It is assumed that the SHA1 hashes
of artifacts are unique - that every artifact has a different SHA1 hash.
To a first approximation, synchronization proceeds by sharing lists 
SHA1 hashes of available artifacts, then sharing those artifacts that
are not found on one side or the other of the connection.  In practice,
a repository might contain millions of artifacts.  The list of
SHA1 hashes for this many artifacts can be large.  So optimizations are
employed that usually reduce the number of SHA1 hashes that need to be
shared to a few hundred.</p>

................................................................................
from the server.  The nonce is the SHA1 hash of the remainder of
the message - all text that follows the newline character that
terminates the login card.  The signature is the SHA1 hash of
the concatenation of the nonce and the users password.</p>

<p>For each login card, the server looks up the user and verifies
that the nonce matches the SHA1 hash of the remainder of the
message.  It then checks the signature hash to make sure the 
signature matches.  If everything
checks out, then the client is granted all privileges of the
specified user.</p>

<p>Privileges are cumulative.  There can be multiple successful
login cards.  The session privileges are the bit-wise OR of the
privileges of each individual login.</p>

<h3>3.3 File and Compressed File Cards</h3>

<p>Artifacts are transferred using either "file" cards, or "cfile" cards.
(The name "file" card comes from the fact that most artifacts correspond to 
files, and "cfile" is just a compressed file.)
</p>

<h4>3.3.1 File Cards</h4>

<p>For sync protocols, artifacts are transferred using "file"
cards.  File cards come in two different formats depending
................................................................................
that immediately follow the cfile card.  If the cfile card has only
three arguments, that means the payload is the complete content of the
artifact.  If the cfile card has four arguments, then the payload is a
delta and the second argument is the ID of another artifact that is the
source of the delta and the third argument is the original size of the
delta artifact.</p>

<p>Unlike file cards, cfile cards are only sent in one direction during a 
clone from server to client for clone protocol version "3" or greater.</p>

<h3>3.4 Push and Pull Cards</h3>

<p>Among the first cards in a client-to-server message are
the push and pull cards.  The push card tells the server that
the client is pushing content.  The pull card tells the server
................................................................................
<blockquote>
<b>reqconfig</b> <i>configuration-name</i>
</blockquote>

<p>As of [/timeline?r=trunk&c=2015-03-19+03%3A57%3A46&n=20|2015-03-19], the configuration-name must be one of the
following values:

<center><table border=0>
<tr><td valign="top">
<ul>
<li> css
<li> header
<li> footer
<li> logo-mimetype
<li> logo-image
................................................................................
<li> ticket-title-expr
<li> ticket-closed-expr
<li> @reportfmt
<li> @user
<li> @concealed
<li> @shun
</ul></td></tr>
</table></center>

<p>New configuration-names are likely to be added in future releases of
Fossil.  If the server receives a configuration-name that it does not
understand, the entire reqconfig card is silently ignored.  The reqconfig
card might also be ignored if the user lacks sufficient privilege to
access the requested information.

................................................................................
the "logo-image" configuration item refers to the project logo image
that is configured on the Admin page of the [./webui.wiki | web-interface].
The value of the configuration item is returned to the client using a
"config" card.

<p>If the configuration-name begins with "@", that refers to a class of
values instead of a single value.  The content of these configuration items
is returned in a "config" card that contains pure SQL text that is 
intended to be evaluated by the client.

<p>The @user and @concealed configuration items contain sensitive information
and are ignored for clients without sufficient privilege.

<h3>3.10 Configuration Cards</h3>

................................................................................
<blockquote>
<b>error</b> <i>error-message</i>
</blockquote>

<p>The error message is English text that is encoded in order to
be a single token.
A space (ASCII 0x20) is represented as "\s" (ASCII 0x5C, 0x73).  A
newline (ASCII 0x0a) is "\n" (ASCII 0x6C, x6E).  A backslash 
(ASCII 0x5C) is represented as two backslashes "\\".  Apart from
space and newline, no other whitespace characters nor any
unprintable characters are allowed in
the error message.</p>

<h3>3.12 Comment Cards</h3>

................................................................................
<p>Here are the key points of the synchronization protocol:</p>

<ol>
<li>The client sends one or more PUSH HTTP requests to the server.
    The request and reply content type is "application/x-fossil".
<li>HTTP request content is compressed using zlib.
<li>The content of request and reply consists of cards with one
    card per line.  
<li>Card formats are:
    <ul>
    <li> <b>login</b> <i>userid nonce signature</i>
    <li> <b>push</b> <i>servercode projectcode</i>
    <li> <b>pull</b> <i>servercode projectcode</i>
    <li> <b>clone</b>
    <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>







|







 







|











|







 







|







 







|







 







|







 







|







 







|







 







|







13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
...
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
...
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
...
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
...
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
...
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
...
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
collection of artifacts.  Each artifact is identified by its SHA1 hash
expressed as a 40-character lower-case hexadecimal string.
Synchronization is simply the process of sharing artifacts between
servers so that all servers have copies of all artifacts.  Because
artifacts are unordered, the order in which artifacts are received
at a server is inconsequential.  It is assumed that the SHA1 hashes
of artifacts are unique - that every artifact has a different SHA1 hash.
To a first approximation, synchronization proceeds by sharing lists
SHA1 hashes of available artifacts, then sharing those artifacts that
are not found on one side or the other of the connection.  In practice,
a repository might contain millions of artifacts.  The list of
SHA1 hashes for this many artifacts can be large.  So optimizations are
employed that usually reduce the number of SHA1 hashes that need to be
shared to a few hundred.</p>

................................................................................
from the server.  The nonce is the SHA1 hash of the remainder of
the message - all text that follows the newline character that
terminates the login card.  The signature is the SHA1 hash of
the concatenation of the nonce and the users password.</p>

<p>For each login card, the server looks up the user and verifies
that the nonce matches the SHA1 hash of the remainder of the
message.  It then checks the signature hash to make sure the
signature matches.  If everything
checks out, then the client is granted all privileges of the
specified user.</p>

<p>Privileges are cumulative.  There can be multiple successful
login cards.  The session privileges are the bit-wise OR of the
privileges of each individual login.</p>

<h3>3.3 File and Compressed File Cards</h3>

<p>Artifacts are transferred using either "file" cards, or "cfile" cards.
(The name "file" card comes from the fact that most artifacts correspond to
files, and "cfile" is just a compressed file.)
</p>

<h4>3.3.1 File Cards</h4>

<p>For sync protocols, artifacts are transferred using "file"
cards.  File cards come in two different formats depending
................................................................................
that immediately follow the cfile card.  If the cfile card has only
three arguments, that means the payload is the complete content of the
artifact.  If the cfile card has four arguments, then the payload is a
delta and the second argument is the ID of another artifact that is the
source of the delta and the third argument is the original size of the
delta artifact.</p>

<p>Unlike file cards, cfile cards are only sent in one direction during a
clone from server to client for clone protocol version "3" or greater.</p>

<h3>3.4 Push and Pull Cards</h3>

<p>Among the first cards in a client-to-server message are
the push and pull cards.  The push card tells the server that
the client is pushing content.  The pull card tells the server
................................................................................
<blockquote>
<b>reqconfig</b> <i>configuration-name</i>
</blockquote>

<p>As of [/timeline?r=trunk&c=2015-03-19+03%3A57%3A46&n=20|2015-03-19], the configuration-name must be one of the
following values:

<table border=0 align="center">
<tr><td valign="top">
<ul>
<li> css
<li> header
<li> footer
<li> logo-mimetype
<li> logo-image
................................................................................
<li> ticket-title-expr
<li> ticket-closed-expr
<li> @reportfmt
<li> @user
<li> @concealed
<li> @shun
</ul></td></tr>
</table>

<p>New configuration-names are likely to be added in future releases of
Fossil.  If the server receives a configuration-name that it does not
understand, the entire reqconfig card is silently ignored.  The reqconfig
card might also be ignored if the user lacks sufficient privilege to
access the requested information.

................................................................................
the "logo-image" configuration item refers to the project logo image
that is configured on the Admin page of the [./webui.wiki | web-interface].
The value of the configuration item is returned to the client using a
"config" card.

<p>If the configuration-name begins with "@", that refers to a class of
values instead of a single value.  The content of these configuration items
is returned in a "config" card that contains pure SQL text that is
intended to be evaluated by the client.

<p>The @user and @concealed configuration items contain sensitive information
and are ignored for clients without sufficient privilege.

<h3>3.10 Configuration Cards</h3>

................................................................................
<blockquote>
<b>error</b> <i>error-message</i>
</blockquote>

<p>The error message is English text that is encoded in order to
be a single token.
A space (ASCII 0x20) is represented as "\s" (ASCII 0x5C, 0x73).  A
newline (ASCII 0x0a) is "\n" (ASCII 0x6C, x6E).  A backslash
(ASCII 0x5C) is represented as two backslashes "\\".  Apart from
space and newline, no other whitespace characters nor any
unprintable characters are allowed in
the error message.</p>

<h3>3.12 Comment Cards</h3>

................................................................................
<p>Here are the key points of the synchronization protocol:</p>

<ol>
<li>The client sends one or more PUSH HTTP requests to the server.
    The request and reply content type is "application/x-fossil".
<li>HTTP request content is compressed using zlib.
<li>The content of request and reply consists of cards with one
    card per line.
<li>Card formats are:
    <ul>
    <li> <b>login</b> <i>userid nonce signature</i>
    <li> <b>push</b> <i>servercode projectcode</i>
    <li> <b>pull</b> <i>servercode projectcode</i>
    <li> <b>clone</b>
    <li> <b>file</b> <i>artifact-id size</i> <b>\n</b> <i>content</i>

Changes to www/tech_overview.wiki.

3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
..
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
..
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
...
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
...
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
...
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
...
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
...
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
A Technical Overview<br>Of The Design And Implementation<br>Of Fossil
</h2>

<h2>1.0 Introduction</h2>

At its lowest level, a Fossil repository consists of an unordered set
of immutable "artifacts".  You might think of these artifacts as "files",
since in many cases the artifacts are exactly that.  
But other "control artifacts" 
are also included in the mix.  These control artifacts define the relationships
between artifacts - which files go together to form a particular
version of the project, who checked in that version and when, what was
the check-in comment, what wiki pages are included with the project, what
are the edit histories of each wiki page, what bug reports or tickets are
included, who contributed to the evolution of each ticket, and so forth.
This low-level file format is called the "global state" of
the repository, since this is the information that is synced to peer
repositories using push and pull operations.   The low-level file format
is also called "enduring" since it is intended to last for many years.
The details of the low-level, enduring, global file format 
are [./fileformat.wiki | described separately].

This article is about how Fossil is currently implemented.  Instead of
dealing with vague abstractions of "enduring file formats" as the
[./fileformat.wiki | other document] does, this article provides
some detail on how Fossil actually stores information on disk.  

<h2>2.0 Three Databases</h2>

Fossil stores state information in 
[http://www.sqlite.org/ | SQLite] database files.
SQLite keeps an entire relational database, including multiple tables and
indices, in a single disk file.  The SQLite library allows the database
files to be efficiently queried and updated using the industry-standard
SQL language.  SQLite updates are atomic, so even in the event of
a system crashes or power failure the repository content is protected.

................................................................................
<li>Repository databases
<li>Checkout databases
</ol>

The configuration database is a one-per-user database that holds
global configuration information used by Fossil.  There is one
repository database per project.  The repository database is the
file that people are normally referring to when they say 
"a Fossil repository".  The checkout database is found in the working
checkout for a project and contains state information that is unique
to that working checkout.

Fossil does not always use all three database files.  The web interface,
for example, typically only uses the repository database.  And the
[/help/all | fossil setting] command only opens the configuration database
................................................................................
database.  Whenever multiple databases are used at the same time,
they are all opened on the same SQLite database connection using
SQLite's [http://www.sqlite.org/lang_attach.html | ATTACH] command.

The chart below provides a quick summary of how each of these
database files are used by Fossil, with detailed discussion following.

<center><table border="1" width="80%" cellpadding="0">
<tr>
<td width="33%" valign="top">
<h3 align="center">Configuration Database<br>"~/.fossil"</h3>
<ul>
<li>Global [/help/setting |settings]
<li>List of active repositories used by the [/help/all | all] command
</ul>
................................................................................
     local edits
<li>The "[/help/stash | stash]"
<li>Information needed to "[/help/undo|undo]" or "[/help/redo|redo]"
</ul>
</td>
</tr>
</table>
</center>

<h3>2.1 The Configuration Database</h3>

The configuration database holds cross-repository preferences and a list of all
repositories for a single user.

The [/help/setting | fossil setting] command can be used to specify various
................................................................................
LOCALAPPDATA, APPDATA, or HOMEPATH environment variables, in that order.

You can override this default location by defining the environment
variable FOSSIL_HOME pointing to an appropriate (writable) directory.

<h3>2.2 Repository Databases</h3>

The repository database is the file that is commonly referred to as 
"the repository".  This is because the repository database contains,
among other things, the complete revision, ticket, and wiki history for
a project.  It is customary to name the repository database after then
name of the project, with a ".fossil" suffix.  For example, the repository
database for the self-hosting Fossil repository is called "fossil.fossil"
and the repository database for SQLite is called "sqlite.fossil".

<h4>2.2.1 Global Project State</h4>

The bulk of the repository database (typically 75 to 85%) consists
of the artifacts that comprise the 
[./fileformat.wiki | enduring, global, shared state] of the project.
The artifacts are stored as BLOBs, compressed using
[http://www.zlib.net/ | zlib compression] and, where applicable,
using [./delta_encoder_algorithm.wiki | delta compression].
The combination of zlib and delta compression results in a considerable
space savings.  For the SQLite project, at the time of this writing,
the total size of all artifacts is over 2.0 GB but thanks to the
combined zlib and delta compression, that content only takes up
32 MB of space in the repository database, for a compression ratio
of about 64:1.  The average size of a content BLOB in the database
is around 500 bytes.

Note that the zlib and delta compression is not an inherent part of the
Fossil file format; it is just an optimization.  
The enduring file format for Fossil is the unordered
set of artifacts. The compression techniques are just a detail of
how the current implementation of Fossil happens to store these artifacts
efficiently on disk.

All of the original uncompressed and undeltaed artifacts can be extracted
from a Fossil repository database using 
the [/help/deconstruct | fossil deconstruct]
command. Individual artifacts can be extracted using the
[/help/artifact | fossil artifact] command.
When accessing the repository database using raw SQL and the
[/help/sqlite3 | fossil sql] command, the extension function
"<tt>content()</tt>" with a single argument which is the SHA1 hash
of an artifact will return the complete undeleted and uncompressed
content of that artifact.  

Going the other way, the [/help/reconstruct | fossil reconstruct]
command will scan a directory hierarchy and add all files found to
a new repository database.  The [/help/import | fossil import] command
works by reading the input git-fast-export stream and using it to construct
corresponding artifacts which are then written into the repository database.

................................................................................
this information (and the user credentials and privileges too) is
local to each repository database; it is not shared between repositories
by [/help/sync | fossil sync].  That is because it is entirely reasonable
that two different websites for the same project might have completely
different display preferences and user communities.  One instance of the
project might be a fork of the other, for example, which pulls from the
other but never pushes and extends the project in ways that the keepers of
the other website disapprove of.  

Display and processing information includes the following:

  *  The name and description of the project
  *  The CSS file, header, and footer used by all web pages
  *  The project logo image
  *  Fields of tickets that are considered "significant" and which are
................................................................................
  *  Ticket report formats and display preferences
  *  Local values for [/help/setting | settings] that override the
     global values defined in the per-user configuration database.

Though the display and processing preferences do not move between
repository instances using [/help/sync | fossil sync], this information
can be shared between repositories using the
[/help/config | fossil config push] and 
[/help/config | fossil config pull] commands.
The display and processing information is also copied into new
repositories when they are created using
[/help/clone | fossil clone].

<h4>2.2.4 User Credentials And Privileges</h4>

Just because two development teams are collaborating on a project and allow
push and/or pull between their repositories does not mean that they 
trust each other enough to share passwords and access privileges.
Hence the names and emails and passwords and privileges of users are
considered private information that is kept locally in each repository.

Each repository database has a table holding the username, privileges,
and login credentials for users authorized to interact with that particular
database.  In addition, there is a table named "concealed" that maps the 
SHA1 hash of each users email address back into their true email address.
The concealed table allows just the SHA1 hash of email addresses to
be stored in tickets, and thus prevents actual email addresses from falling
into the hands of spammers who happen to clone the repository. 

The content of the user and concealed tables can be pushed and pulled using the
[/help/config | fossil config push] and
[/help/config | fossil config pull] commands with the "user" and
"email" as the AREA argument, but only if you have administrative
privileges on the remote repository.

................................................................................

The set of canonical artifacts for a project - the global state for the
project - is intended to be an append-only database.  In other words,
new artifacts can be added but artifacts can never be removed.  But
it sometimes happens that inappropriate content is mistakenly or
maliciously added to a repository.  The only way to get rid of
the undesired content is to [./shunning.wiki | "shun"] it.
The "shun" table in the repository database records the SHA1 hash of 
all shunned artifacts.

The shun table can be pushed or pulled using
the [/help/config | fossil config] command with the "shun" AREA argument.
The shun table is also copied during a [/help/clone | clone].

<h3>2.3 Checkout Databases</h3>







|
|










|





|



|







 







|







 







|







 







<







 







|










|













|






|







|







 







|







 







|








|






|



|







 







|







3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
..
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
..
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
...
101
102
103
104
105
106
107

108
109
110
111
112
113
114
...
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
...
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
...
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
...
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
A Technical Overview<br>Of The Design And Implementation<br>Of Fossil
</h2>

<h2>1.0 Introduction</h2>

At its lowest level, a Fossil repository consists of an unordered set
of immutable "artifacts".  You might think of these artifacts as "files",
since in many cases the artifacts are exactly that.
But other "control artifacts"
are also included in the mix.  These control artifacts define the relationships
between artifacts - which files go together to form a particular
version of the project, who checked in that version and when, what was
the check-in comment, what wiki pages are included with the project, what
are the edit histories of each wiki page, what bug reports or tickets are
included, who contributed to the evolution of each ticket, and so forth.
This low-level file format is called the "global state" of
the repository, since this is the information that is synced to peer
repositories using push and pull operations.   The low-level file format
is also called "enduring" since it is intended to last for many years.
The details of the low-level, enduring, global file format
are [./fileformat.wiki | described separately].

This article is about how Fossil is currently implemented.  Instead of
dealing with vague abstractions of "enduring file formats" as the
[./fileformat.wiki | other document] does, this article provides
some detail on how Fossil actually stores information on disk.

<h2>2.0 Three Databases</h2>

Fossil stores state information in
[http://www.sqlite.org/ | SQLite] database files.
SQLite keeps an entire relational database, including multiple tables and
indices, in a single disk file.  The SQLite library allows the database
files to be efficiently queried and updated using the industry-standard
SQL language.  SQLite updates are atomic, so even in the event of
a system crashes or power failure the repository content is protected.

................................................................................
<li>Repository databases
<li>Checkout databases
</ol>

The configuration database is a one-per-user database that holds
global configuration information used by Fossil.  There is one
repository database per project.  The repository database is the
file that people are normally referring to when they say
"a Fossil repository".  The checkout database is found in the working
checkout for a project and contains state information that is unique
to that working checkout.

Fossil does not always use all three database files.  The web interface,
for example, typically only uses the repository database.  And the
[/help/all | fossil setting] command only opens the configuration database
................................................................................
database.  Whenever multiple databases are used at the same time,
they are all opened on the same SQLite database connection using
SQLite's [http://www.sqlite.org/lang_attach.html | ATTACH] command.

The chart below provides a quick summary of how each of these
database files are used by Fossil, with detailed discussion following.

<table border="1" width="80%" cellpadding="0" align="center">
<tr>
<td width="33%" valign="top">
<h3 align="center">Configuration Database<br>"~/.fossil"</h3>
<ul>
<li>Global [/help/setting |settings]
<li>List of active repositories used by the [/help/all | all] command
</ul>
................................................................................
     local edits
<li>The "[/help/stash | stash]"
<li>Information needed to "[/help/undo|undo]" or "[/help/redo|redo]"
</ul>
</td>
</tr>
</table>


<h3>2.1 The Configuration Database</h3>

The configuration database holds cross-repository preferences and a list of all
repositories for a single user.

The [/help/setting | fossil setting] command can be used to specify various
................................................................................
LOCALAPPDATA, APPDATA, or HOMEPATH environment variables, in that order.

You can override this default location by defining the environment
variable FOSSIL_HOME pointing to an appropriate (writable) directory.

<h3>2.2 Repository Databases</h3>

The repository database is the file that is commonly referred to as
"the repository".  This is because the repository database contains,
among other things, the complete revision, ticket, and wiki history for
a project.  It is customary to name the repository database after then
name of the project, with a ".fossil" suffix.  For example, the repository
database for the self-hosting Fossil repository is called "fossil.fossil"
and the repository database for SQLite is called "sqlite.fossil".

<h4>2.2.1 Global Project State</h4>

The bulk of the repository database (typically 75 to 85%) consists
of the artifacts that comprise the
[./fileformat.wiki | enduring, global, shared state] of the project.
The artifacts are stored as BLOBs, compressed using
[http://www.zlib.net/ | zlib compression] and, where applicable,
using [./delta_encoder_algorithm.wiki | delta compression].
The combination of zlib and delta compression results in a considerable
space savings.  For the SQLite project, at the time of this writing,
the total size of all artifacts is over 2.0 GB but thanks to the
combined zlib and delta compression, that content only takes up
32 MB of space in the repository database, for a compression ratio
of about 64:1.  The average size of a content BLOB in the database
is around 500 bytes.

Note that the zlib and delta compression is not an inherent part of the
Fossil file format; it is just an optimization.
The enduring file format for Fossil is the unordered
set of artifacts. The compression techniques are just a detail of
how the current implementation of Fossil happens to store these artifacts
efficiently on disk.

All of the original uncompressed and undeltaed artifacts can be extracted
from a Fossil repository database using
the [/help/deconstruct | fossil deconstruct]
command. Individual artifacts can be extracted using the
[/help/artifact | fossil artifact] command.
When accessing the repository database using raw SQL and the
[/help/sqlite3 | fossil sql] command, the extension function
"<tt>content()</tt>" with a single argument which is the SHA1 hash
of an artifact will return the complete undeleted and uncompressed
content of that artifact.

Going the other way, the [/help/reconstruct | fossil reconstruct]
command will scan a directory hierarchy and add all files found to
a new repository database.  The [/help/import | fossil import] command
works by reading the input git-fast-export stream and using it to construct
corresponding artifacts which are then written into the repository database.

................................................................................
this information (and the user credentials and privileges too) is
local to each repository database; it is not shared between repositories
by [/help/sync | fossil sync].  That is because it is entirely reasonable
that two different websites for the same project might have completely
different display preferences and user communities.  One instance of the
project might be a fork of the other, for example, which pulls from the
other but never pushes and extends the project in ways that the keepers of
the other website disapprove of.

Display and processing information includes the following:

  *  The name and description of the project
  *  The CSS file, header, and footer used by all web pages
  *  The project logo image
  *  Fields of tickets that are considered "significant" and which are
................................................................................
  *  Ticket report formats and display preferences
  *  Local values for [/help/setting | settings] that override the
     global values defined in the per-user configuration database.

Though the display and processing preferences do not move between
repository instances using [/help/sync | fossil sync], this information
can be shared between repositories using the
[/help/config | fossil config push] and
[/help/config | fossil config pull] commands.
The display and processing information is also copied into new
repositories when they are created using
[/help/clone | fossil clone].

<h4>2.2.4 User Credentials And Privileges</h4>

Just because two development teams are collaborating on a project and allow
push and/or pull between their repositories does not mean that they
trust each other enough to share passwords and access privileges.
Hence the names and emails and passwords and privileges of users are
considered private information that is kept locally in each repository.

Each repository database has a table holding the username, privileges,
and login credentials for users authorized to interact with that particular
database.  In addition, there is a table named "concealed" that maps the
SHA1 hash of each users email address back into their true email address.
The concealed table allows just the SHA1 hash of email addresses to
be stored in tickets, and thus prevents actual email addresses from falling
into the hands of spammers who happen to clone the repository.

The content of the user and concealed tables can be pushed and pulled using the
[/help/config | fossil config push] and
[/help/config | fossil config pull] commands with the "user" and
"email" as the AREA argument, but only if you have administrative
privileges on the remote repository.

................................................................................

The set of canonical artifacts for a project - the global state for the
project - is intended to be an append-only database.  In other words,
new artifacts can be added but artifacts can never be removed.  But
it sometimes happens that inappropriate content is mistakenly or
maliciously added to a repository.  The only way to get rid of
the undesired content is to [./shunning.wiki | "shun"] it.
The "shun" table in the repository database records the SHA1 hash of
all shunned artifacts.

The shun table can be pushed or pulled using
the [/help/config | fossil config] command with the "shun" AREA argument.
The shun table is also copied during a [/help/clone | clone].

<h3>2.3 Checkout Databases</h3>

Changes to www/webpage-ex.md.

62
63
64
65
66
67
68





69
70
71
72
73
74
75
     href='$ROOT/timeline?namechng'>Example</a>
     Show check-ins that contain file name changes

  *  <a target='_blank' class='exbtn'
     href='$ROOT/timeline?u=drh&c=2014-01-08&y=ci'>Example</a>
     Show check-ins circa 2014-01-08 by user "drh".






     <big><b>&rarr;</b></big> (Hint:  In the pages above, click the graph nodes
     for any two check-ins or files to see a diff.)
     <big><b>&larr;</b></big>

  *  <a target='_blank' class='exbtn'
     href='$ROOT/search?s=interesting+pages'>Example</a>
     Full-text search for "interesting pages".







>
>
>
>
>







62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
     href='$ROOT/timeline?namechng'>Example</a>
     Show check-ins that contain file name changes

  *  <a target='_blank' class='exbtn'
     href='$ROOT/timeline?u=drh&c=2014-01-08&y=ci'>Example</a>
     Show check-ins circa 2014-01-08 by user "drh".

  *  <a target='_blank' class='exbtn'
     href='$ROOT/timeline?from=version-1.34&to=version-1.35&chng=src/timeline.c,src/doc.c'>Example</a>
     Show all check-ins between version-1.34 and version-1.35 that make
     changes to either of the files src/timeline.c or src/doc.c.

     <big><b>&rarr;</b></big> (Hint:  In the pages above, click the graph nodes
     for any two check-ins or files to see a diff.)
     <big><b>&larr;</b></big>

  *  <a target='_blank' class='exbtn'
     href='$ROOT/search?s=interesting+pages'>Example</a>
     Full-text search for "interesting pages".