Fossil

Changes On Branch manifest-sort-check
Login

Changes On Branch manifest-sort-check

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch manifest-sort-check Excluding Merge-Ins

This is equivalent to a diff from 1cd6c545 to 72f5101e

2021-02-10
23:53
Do a better job of detecting misformatted structural artifacts and reporting the error. Ticket [15d04de574383d61]. ... (check-in: 023fddee user: drh tags: trunk)
23:49
Give a better error message if structural artifact cards are out-of-order. ... (Closed-Leaf check-in: 72f5101e user: drh tags: manifest-sort-check)
23:34
Modify the out-of-order card detection in the structural artifact parser to allow N and P cards of a technote to be in the wrong order. [Ticket 15d04de574383d61]. ... (check-in: 36620b74 user: drh tags: manifest-sort-check)
22:13
This seems to fix the missing is-sorted check in manifest_parse(), but (A) it needs more testing and (B) could use a better error message than the one it ends up generating. Edit: WARNING: fixing this will cause control artifacts which previously passed (but should not have) to fail. ... (check-in: 2e7a0aac user: stephan tags: manifest-sort-check)
21:21
Corrected the order of the N- and P-cards on edits made to technotes. See /forumpost/74fd8dac3a for details. ... (check-in: 1cd6c545 user: stephan tags: trunk)
14:25
Added 'L' pseudo-capability character to indicate is-logged-in, for use with th1 capexpr. Removed hard-coded addition of login/logout menu entries in skin headers, delegating them to the mainmenu setting, and added Login/Logout entries to the default mainmenu value. ... (check-in: e5240c97 user: stephan tags: trunk)

Changes to src/manifest.c.

532
533
534
535
536
537
538
539












540
541
542

543
544
545
546
547
548
549
  pContent = &p->content;

  /* Begin parsing, card by card.
  */
  x.z = z;
  x.zEnd = &z[n];
  x.atEol = 1;
  while( (cType = next_card(&x))!=0 && cType>=cPrevType ){












    lineNo++;
    if( cType<'A' || cType>'Z' ) SYNTAX("bad card type");
    seenCard |= 1 << (cType-'A');

    switch( cType ){
      /*
      **     A <filename> <target> ?<source>?
      **
      ** Identifies an attachment to either a wiki page or a ticket.
      ** <source> is the artifact that is the attachment.  <source>
      ** is omitted to delete an attachment.  <target> is the name of







|
>
>
>
>
>
>
>
>
>
>
>
>



>







532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
  pContent = &p->content;

  /* Begin parsing, card by card.
  */
  x.z = z;
  x.zEnd = &z[n];
  x.atEol = 1;
  while( (cType = next_card(&x))!=0 ){
    if( cType<cPrevType ){
      /* Cards must be in increasing order.  However, out-of-order detection
      ** was broken prior to 2021-02-10 due to a bug.  Furthermore, there
      ** was a bug in technote generation (prior to 2021-02-10) that caused
      ** the P card to occur before the N card.  Hence, for historical
      ** compatibility, we do allow the N card of a technote to occur after
      ** the P card.  See tickets 15d04de574383d61 and 5e67a7f4041a36ad.
      */
      if( cType!='N' || cPrevType!='P' || p->zEventId==0 ){
        SYNTAX("cards not in lexicographical order");
      }
    }
    lineNo++;
    if( cType<'A' || cType>'Z' ) SYNTAX("bad card type");
    seenCard |= 1 << (cType-'A');
    cPrevType = cType;
    switch( cType ){
      /*
      **     A <filename> <target> ?<source>?
      **
      ** Identifies an attachment to either a wiki page or a ticket.
      ** <source> is the artifact that is the attachment.  <source>
      ** is omitted to delete an attachment.  <target> is the name of

Changes to www/fileformat.wiki.

84
85
86
87
88
89
90


91
92
93
94
95
96
97
98
(ASCII: 0x0a) character. Each card begins with a single
character "card type".  Zero or more arguments may follow
the card type.  All arguments are separated from each other
and from the card-type character by a single space
character.  There is no surplus white space between arguments
and no leading or trailing whitespace except for the newline
character that acts as the card separator.  All cards must be in strict


lexicographical order.  There may not be any duplicate cards.

In the current implementation (as of 2017-02-27) the artifacts that
make up a fossil repository are stored as delta- and zlib-compressed
blobs in an <a href="http://www.sqlite.org/">SQLite</a> database.  This
is an implementation detail and might change in a future release.  For
the purpose of this article "file format" means the format of the artifacts,
not how the artifacts are stored on disk.  It is the artifact format that







>
>
|







84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
(ASCII: 0x0a) character. Each card begins with a single
character "card type".  Zero or more arguments may follow
the card type.  All arguments are separated from each other
and from the card-type character by a single space
character.  There is no surplus white space between arguments
and no leading or trailing whitespace except for the newline
character that acts as the card separator.  All cards must be in strict
lexicographical order (except for an
[./fileformat.wiki#outofordercards|historical bug compatibility]).
There may not be any duplicate cards.

In the current implementation (as of 2017-02-27) the artifacts that
make up a fossil repository are stored as delta- and zlib-compressed
blobs in an <a href="http://www.sqlite.org/">SQLite</a> database.  This
is an implementation detail and might change in a future release.  For
the purpose of this article "file format" means the format of the artifacts,
not how the artifacts are stored on disk.  It is the artifact format that
871
872
873
874
875
876
877







































878
879
880
881
882
883
884
885

<a name="addenda"></a>
<h2>4.0 Addenda</h2>

This section contains additional information which may be useful when
implementing algorithms described above.








































<h3>4.1 R-Card Hash Calculation</h3>

Given a manifest file named <tt>MF</tt>, the following Bash shell code
demonstrates how to compute the value of the <b>R</b> card in that manifest.
This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are
shell input and other lines are output. This demonstration assumes that the
file versions represented by the input manifest are checked out
under the current directory.







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|







873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926

<a name="addenda"></a>
<h2>4.0 Addenda</h2>

This section contains additional information which may be useful when
implementing algorithms described above.

<a name="outofordercards"></a>
<h3>4.1 Relaxed Card Ordering Due To An Historical Bug</h3>

All cards of a structural artifact should be in lexicographical order.
The Fossil implementation verifies this and rejects any structural
artifact which has out-of-order cards.  Futhermore, when Fossil is
generating new structural artifacts, it runs the generated artifact
through the parser to confirm that all cards really are in the correct
order before committing the transaction.  In this way, Fossil prevents
bugs in the code from accidentally inserting misformatted artifacts.
The test parse of newly created artifacts is part of the
[./selfcheck.wiki|self-check strategy] of Fossil.  It takes a
few more CPU cycles to double check each artifact before inserting it.
The developers consider those CPU cycles well-spent.

However, the card-order safety check was accidentally disabled due to
[15d04de574383d61|a bug].
And while that bug was lurking undetected in the code,
[5e67a7f4041a36ad|another bug] caused the N cards of Technical Notes
to occur after the P card rather than before.
Thus for a span of several years, Technical Note artifacts were being
inserted into Fossil repositories that had their N and P cards in the
wrong order.

Both bugs have now been fixed.  However, to prevent historical
Technical Note artifacts that were inserted by users in good faith
from being rejected by newer Fossil builds, the card ordering
requirement is relaxed slightly.  The actual implementation is this:

<blockquote>
"All cards must be in strict lexicographic order, except that the
N and P cards of a Technical Note artifact are allowed to be
interchanged."
</blockquote>

Future versions of Fossil might strengthen this slightly to only allow
the out of order N and P cards for Technical Notes entered before
a certain date.

<h3>4.2 R-Card Hash Calculation</h3>

Given a manifest file named <tt>MF</tt>, the following Bash shell code
demonstrates how to compute the value of the <b>R</b> card in that manifest.
This example uses manifest [28987096ac]. Lines starting with <tt>#</tt> are
shell input and other lines are output. This demonstration assumes that the
file versions represented by the input manifest are checked out
under the current directory.