Fossil

Check-in [023fddee]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Do a better job of detecting misformatted structural artifacts and reporting the error. Ticket [15d04de574383d61].
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 023fddeec402930662a022c1e4bd2ea63027ed0c8384d9e8bfd1bd0540f47de3
User & Date: drh 2021-02-10 23:53:08
Context
2021-02-11
00:35
Improved hover colors for tree-view and file-ages in the darkmode skin. ... (check-in: eb535618 user: drh tags: trunk)
2021-02-10
23:53
Do a better job of detecting misformatted structural artifacts and reporting the error. Ticket [15d04de574383d61]. ... (check-in: 023fddee user: drh tags: trunk)
23:49
Give a better error message if structural artifact cards are out-of-order. ... (Closed-Leaf check-in: 72f5101e user: drh tags: manifest-sort-check)
21:21
Corrected the order of the N- and P-cards on edits made to technotes. See /forumpost/74fd8dac3a for details. ... (check-in: 1cd6c545 user: stephan tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to src/manifest.c.

532
533
534
535
536
537
538
539












540
541
542

543
544
545
546
547
548
549
  pContent = &p->content;

  /* Begin parsing, card by card.
  */
  x.z = z;
  x.zEnd = &z[n];
  x.atEol = 1;
  while( (cType = next_card(&x))!=0 && cType>=cPrevType ){












    lineNo++;
    if( cType<'A' || cType>'Z' ) SYNTAX("bad card type");
    seenCard |= 1 << (cType-'A');

    switch( cType ){
      /*
      **     A <filename> <target> ?<source>?
      **
      ** Identifies an attachment to either a wiki page or a ticket.
      ** <source> is the artifact that is the attachment.  <source>
      ** is omitted to delete an attachment.  <target> is the name of







|
>
>
>
>
>
>
>
>
>
>
>
>



>







532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
  pContent = &p->content;

  /* Begin parsing, card by card.
  */
  x.z = z;
  x.zEnd = &z[n];
  x.atEol = 1;
  while( (cType = next_card(&x))!=0 ){
    if( cType<cPrevType ){
      /* Cards must be in increasing order.  However, out-of-order detection
      ** was broken prior to 2021-02-10 due to a bug.  Furthermore, there
      ** was a bug in technote generation (prior to 2021-02-10) that caused
      ** the P card to occur before the N card.  Hence, for historical
      ** compatibility, we do allow the N card of a technote to occur after
      ** the P card.  See tickets 15d04de574383d61 and 5e67a7f4041a36ad.
      */
      if( cType!='N' || cPrevType!='P' || p->zEventId==0 ){
        SYNTAX("cards not in lexicographical order");
      }
    }
    lineNo++;
    if( cType<'A' || cType>'Z' ) SYNTAX("bad card type");
    seenCard |= 1 << (cType-'A');
    cPrevType = cType;
    switch( cType ){
      /*
      **     A <filename> <target> ?<source>?
      **
      ** Identifies an attachment to either a wiki page or a ticket.
      ** <source> is the artifact that is the attachment.  <source>
      ** is omitted to delete an attachment.  <target> is the name of

Changes to www/fileformat.wiki.

84
85
86
87
88
89
90


91
92
93
94
95
96
97
98
...
871
872
873
874
875
876
877







































878
879
880
881
882
883
884
885
(ASCII: 0x0a) character. Each card begins with a single
character "card type".  Zero or more arguments may follow
the card type.  All arguments are separated from each other
and from the card-type character by a single space
character.  There is no surplus white space between arguments
and no leading or trailing whitespace except for the newline
character that acts as the card separator.  All cards must be in strict


lexicographical order.  There may not be any duplicate cards.

In the current implementation (as of 2017-02-27) the artifacts that
make up a fossil repository are stored as delta- and zlib-compressed
blobs in an <a href="http://www.sqlite.org/">SQLite</a> database.  This
is an implementation detail and might change in a future release.  For
the purpose of this article "file format" means the format of the artifacts,
not how the artifacts are stored on disk.  It is the artifact format that
................................................................................

<a name="addenda"></a>
<h2>4.0 Addenda</h2>

This section contains additional information which may be useful when
implementing algorithms described above.








































<h3>4.1 R-Card Hash Calculation</h3>

Given a manifest file named <tt>MF</tt>, the following Bash shell code
demonstrates how to compute the value of the <b>R</b> card in that manifest.
This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are
shell input and other lines are output. This demonstration assumes that the
file versions represented by the input manifest are checked out
under the current directory.







>
>
|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|







84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
...
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
(ASCII: 0x0a) character. Each card begins with a single
character "card type".  Zero or more arguments may follow
the card type.  All arguments are separated from each other
and from the card-type character by a single space
character.  There is no surplus white space between arguments
and no leading or trailing whitespace except for the newline
character that acts as the card separator.  All cards must be in strict
lexicographical order (except for an
[./fileformat.wiki#outofordercards|historical bug compatibility]).
There may not be any duplicate cards.

In the current implementation (as of 2017-02-27) the artifacts that
make up a fossil repository are stored as delta- and zlib-compressed
blobs in an <a href="http://www.sqlite.org/">SQLite</a> database.  This
is an implementation detail and might change in a future release.  For
the purpose of this article "file format" means the format of the artifacts,
not how the artifacts are stored on disk.  It is the artifact format that
................................................................................

<a name="addenda"></a>
<h2>4.0 Addenda</h2>

This section contains additional information which may be useful when
implementing algorithms described above.

<a name="outofordercards"></a>
<h3>4.1 Relaxed Card Ordering Due To An Historical Bug</h3>

All cards of a structural artifact should be in lexicographical order.
The Fossil implementation verifies this and rejects any structural
artifact which has out-of-order cards.  Futhermore, when Fossil is
generating new structural artifacts, it runs the generated artifact
through the parser to confirm that all cards really are in the correct
order before committing the transaction.  In this way, Fossil prevents
bugs in the code from accidentally inserting misformatted artifacts.
The test parse of newly created artifacts is part of the
[./selfcheck.wiki|self-check strategy] of Fossil.  It takes a
few more CPU cycles to double check each artifact before inserting it.
The developers consider those CPU cycles well-spent.

However, the card-order safety check was accidentally disabled due to
[15d04de574383d61|a bug].
And while that bug was lurking undetected in the code,
[5e67a7f4041a36ad|another bug] caused the N cards of Technical Notes
to occur after the P card rather than before.
Thus for a span of several years, Technical Note artifacts were being
inserted into Fossil repositories that had their N and P cards in the
wrong order.

Both bugs have now been fixed.  However, to prevent historical
Technical Note artifacts that were inserted by users in good faith
from being rejected by newer Fossil builds, the card ordering
requirement is relaxed slightly.  The actual implementation is this:

<blockquote>
"All cards must be in strict lexicographic order, except that the
N and P cards of a Technical Note artifact are allowed to be
interchanged."
</blockquote>

Future versions of Fossil might strengthen this slightly to only allow
the out of order N and P cards for Technical Notes entered before
a certain date.

<h3>4.2 R-Card Hash Calculation</h3>

Given a manifest file named <tt>MF</tt>, the following Bash shell code
demonstrates how to compute the value of the <b>R</b> card in that manifest.
This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are
shell input and other lines are output. This demonstration assumes that the
file versions represented by the input manifest are checked out
under the current directory.