Fossil

Changes On Branch sec2020-deadend
Login

Changes On Branch sec2020-deadend

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch sec2020-deadend Excluding Merge-Ins

This is equivalent to a diff from 917917aa to 1e34705e

2020-08-18
19:49
Rework the "permissive-manifest-parser" idea to be simpler and to call it "strict-manifest-syntax". ... (check-in: 4df8c856 user: drh tags: sec2020)
17:25
permissive-manifest-parser setting is now marked as sensitive to keep an attacker from turning it on. ... (Closed-Leaf check-in: 1e34705e user: stephan tags: sec2020-deadend)
16:07
Added and applied permissive-manifest-parser setting to permit parsing of manifests which have F-cards containing now-illegal names. Required for rebuild of historical data and support of repositories we now know to contain such files. ... (check-in: 9e59cf18 user: stephan tags: sec2020-deadend)
14:02
Merge in the latest trunk changes. ... (check-in: 917917aa user: drh tags: sec2020)
14:00
Allow <del> and <ins> markup in wiki and in markdown. ... (check-in: ae9a9db5 user: drh tags: trunk)
13:17
More missing db_unprotect() calls. ... (check-in: 06d3789a user: drh tags: sec2020)

Changes to src/db.c.

3924
3925
3926
3927
3928
3929
3930











3931
3932
3933
3934
3935
3936
3937
** If enabled on a client, new delta manifests are prohibited on
** commits.  If enabled on a server, whenever a client attempts
** to obtain a check-in lock during auto-sync, the server will 
** send the "pragma avoid-delta-manifests" statement in its reply,
** which will cause the client to avoid generating a delta
** manifest.
*/











/*
** SETTING: proxy            width=32 default=off
** URL of the HTTP proxy.  If undefined or "off" then
** the "http_proxy" environment variable is consulted.
** If the http_proxy environment variable is undefined
** then a direct HTTP connection is used.
*/







>
>
>
>
>
>
>
>
>
>
>







3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
** If enabled on a client, new delta manifests are prohibited on
** commits.  If enabled on a server, whenever a client attempts
** to obtain a check-in lock during auto-sync, the server will 
** send the "pragma avoid-delta-manifests" statement in its reply,
** which will cause the client to avoid generating a delta
** manifest.
*/
/*
** SETTING: permissive-manifest-parser  boolean default=off sensitive
** By default, fossil fatally fails if any files are found in a
** manifest which have a name matching a checkout database name. In
** order to support repositories where such files were inadvertently
** checked in, this setting, when on, allows such files to be handled
** as if they were normal files. Only enable this if absolutely
** necessary to support older repositories which have such files
** checked in (anywhere in their history). It should never be enabled
** for new repositories or old ones which do not contain such files.
*/
/*
** SETTING: proxy            width=32 default=off
** URL of the HTTP proxy.  If undefined or "off" then
** the "http_proxy" environment variable is consulted.
** If the http_proxy environment variable is undefined
** then a direct HTTP connection is used.
*/

Changes to src/file.c.

2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
    }
  }
}

/*
** COMMAND: test-is-reserved-name
**
** Usage: %fossil test-is-ckout-db FILENAMES...
**
** Passes each given name to file_is_reserved_name() and outputs one
** line per file: the result value of that function followed by the
** name.
*/
void test_is_reserved_name_cmd(void){
  int i;







|







2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
    }
  }
}

/*
** COMMAND: test-is-reserved-name
**
** Usage: %fossil test-is-reserved-name FILENAMES...
**
** Passes each given name to file_is_reserved_name() and outputs one
** line per file: the result value of that function followed by the
** name.
*/
void test_is_reserved_name_cmd(void){
  int i;

Changes to src/main.c.

218
219
220
221
222
223
224




225
226
227
228
229
230
231
#endif
  int useLocalauth;       /* No login required if from 127.0.0.1 */
  int noPswd;             /* Logged in without password (on 127.0.0.1) */
  int userUid;            /* Integer user id */
  int isHuman;            /* True if access by a human, not a spider or bot */
  int comFmtFlags;        /* Zero or more "COMMENT_PRINT_*" bit flags, should be
                          ** accessed through get_comment_format(). */





  /* Information used to populate the RCVFROM table */
  int rcvid;              /* The rcvid.  0 if not yet defined. */
  char *zIpAddr;          /* The remote IP address */
  char *zNonce;           /* The nonce used for login */

  /* permissions available to current user */







>
>
>
>







218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
#endif
  int useLocalauth;       /* No login required if from 127.0.0.1 */
  int noPswd;             /* Logged in without password (on 127.0.0.1) */
  int userUid;            /* Integer user id */
  int isHuman;            /* True if access by a human, not a spider or bot */
  int comFmtFlags;        /* Zero or more "COMMENT_PRINT_*" bit flags, should be
                          ** accessed through get_comment_format(). */
  int permissiveManifest; /* Tells manifest_parser() whether it may run in
                          ** "permissive" (compatibilty) mode. <0=not yet determined,
                          ** 0=no, >0=yes. MUST be set to a negative value early on
                          ** in app-init (before CLI flags are processed). */

  /* Information used to populate the RCVFROM table */
  int rcvid;              /* The rcvid.  0 if not yet defined. */
  char *zIpAddr;          /* The remote IP address */
  char *zNonce;           /* The nonce used for login */

  /* permissions available to current user */
681
682
683
684
685
686
687

688
689
690
691
692
693
694
                 sqlite3_libversion());
  }
  sqlite3_config(SQLITE_CONFIG_MULTITHREAD);
  sqlite3_config(SQLITE_CONFIG_LOG, fossil_sqlite_log, 0);
  memset(&g, 0, sizeof(g));
  g.now = time(0);
  g.httpHeader = empty_blob;

#ifdef FOSSIL_ENABLE_JSON
#if defined(NDEBUG)
  g.json.errorDetailParanoia = 2 /* FIXME: make configurable
                                    One problem we have here is that this
                                    code is needed before the db is opened,
                                    so we can't sql for it.*/;
#else







>







685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
                 sqlite3_libversion());
  }
  sqlite3_config(SQLITE_CONFIG_MULTITHREAD);
  sqlite3_config(SQLITE_CONFIG_LOG, fossil_sqlite_log, 0);
  memset(&g, 0, sizeof(g));
  g.now = time(0);
  g.httpHeader = empty_blob;
  g.permissiveManifest = -1;
#ifdef FOSSIL_ENABLE_JSON
#if defined(NDEBUG)
  g.json.errorDetailParanoia = 2 /* FIXME: make configurable
                                    One problem we have here is that this
                                    code is needed before the db is opened,
                                    so we can't sql for it.*/;
#else

Changes to src/manifest.c.

452
453
454
455
456
457
458



459
460
461
462
463
464
465
466
    isRepeat = 1;
  }else if( bag_find(&seenManifests, rid) ){
    isRepeat = 1;
  }else{
    isRepeat = 0;
    bag_insert(&seenManifests, rid);
  }




  /* Every structural artifact ends with a '\n' character.  Exit early
  ** if that is not the case for this artifact.
  */
  if( !isRepeat ) g.parseCnt[0]++;
  z = blob_materialize(pContent);
  n = blob_size(pContent);
  if( n<=0 || z[n-1]!='\n' ){







>
>
>
|







452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
    isRepeat = 1;
  }else if( bag_find(&seenManifests, rid) ){
    isRepeat = 1;
  }else{
    isRepeat = 0;
    bag_insert(&seenManifests, rid);
  }
  if(g.permissiveManifest<0){
    g.permissiveManifest =
      db_get_boolean("permissive-manifest-parser", 0);
  }
  /* Every structural artifact ends with a '\n' character.  Exit early
  ** if that is not the case for this artifact.
  */
  if( !isRepeat ) g.parseCnt[0]++;
  z = blob_materialize(pContent);
  n = blob_size(pContent);
  if( n<=0 || z[n-1]!='\n' ){
628
629
630
631
632
633
634


635
636

637
638
639
640
641
642
643
644
      ** other control file.  The filename and old-name are fossil-encoded.
      */
      case 'F': {
        char *zName, *zPerm, *zPriorName;
        zName = next_token(&x,0);
        if( zName==0 ) SYNTAX("missing filename on F-card");
        defossilize(zName);


        if( !file_is_simple_pathname_nonstrict(zName) ){
          SYNTAX("F-card filename is not a simple path");

        }else if( file_is_reserved_name(zName,-1) ){
          SYNTAX("F-card contains a reserved name");
        }
        zUuid = next_token(&x, &sz);
        if( p->zBaseline==0 || zUuid!=0 ){
          if( zUuid==0 ) SYNTAX("missing hash on F-card");
          if( !hname_validate(zUuid,sz) ){
            SYNTAX("F-card hash invalid");







>
>


>
|







631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
      ** other control file.  The filename and old-name are fossil-encoded.
      */
      case 'F': {
        char *zName, *zPerm, *zPriorName;
        zName = next_token(&x,0);
        if( zName==0 ) SYNTAX("missing filename on F-card");
        defossilize(zName);
        assert(g.permissiveManifest>=0
               && "Must have been set at app init");
        if( !file_is_simple_pathname_nonstrict(zName) ){
          SYNTAX("F-card filename is not a simple path");
        }else if( g.permissiveManifest==0
                  && file_is_reserved_name(zName,-1) ){
          SYNTAX("F-card contains a reserved name");
        }
        zUuid = next_token(&x, &sz);
        if( p->zBaseline==0 || zUuid!=0 ){
          if( zUuid==0 ) SYNTAX("missing hash on F-card");
          if( !hname_validate(zUuid,sz) ){
            SYNTAX("F-card hash invalid");

Changes to src/rebuild.c.

626
627
628
629
630
631
632







633
634
635
636
637
638
639
  int showStats;
  int runReindex;
  int optNoIndex;
  int optIndex;
  int optIfNeeded;
  int compressOnlyFlag;








  omitVerify = find_option("noverify",0,0)!=0;
  forceFlag = find_option("force","f",0)!=0;
  randomizeFlag = find_option("randomize", 0, 0)!=0;
  doClustering = find_option("cluster", 0, 0)!=0;
  runVacuum = find_option("vacuum",0,0)!=0;
  runDeanalyze = find_option("deanalyze",0,0)!=0;
  runAnalyze = find_option("analyze",0,0)!=0;







>
>
>
>
>
>
>







626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
  int showStats;
  int runReindex;
  int optNoIndex;
  int optIndex;
  int optIfNeeded;
  int compressOnlyFlag;

  g.permissiveManifest = 1
    /* We always allow permissive manifest parsing when mass-dealing
       with batches which are likely to include historical, but no
       longer used/relevant, manifests. Though rebuild will not fail
       for bad manifests, it will consider them to be non-manifests,
       so would necessarily elide them from the timeline.
    */;
  omitVerify = find_option("noverify",0,0)!=0;
  forceFlag = find_option("force","f",0)!=0;
  randomizeFlag = find_option("randomize", 0, 0)!=0;
  doClustering = find_option("cluster", 0, 0)!=0;
  runVacuum = find_option("vacuum",0,0)!=0;
  runDeanalyze = find_option("deanalyze",0,0)!=0;
  runAnalyze = find_option("analyze",0,0)!=0;