Fossil Forum

fossil bisect option linear on
Login

fossil bisect option linear on

fossil bisect option linear on

(1) By MBL (RoboManni) on 2022-06-22 07:22:54 [source]

I just saw that with checking 42f61b67 from 2022-06-05 we will get with next release this beautiful option:

Add the "fossil bisect option linear on" command that allows the "fossil bisect run" command to invoke a test script on every check-in along a path between two boundary check-ins. The "linear" option resets automatically opon "fossil bisect reset"

Question:

As this new option got to trunk only after the last recent snapshot binary, can we get an update for testing already before release of version 2.19 ?

I would love to try with the fossil bisect run [OPTIONS] COMMAND to see if I can achieve what I was already looking for to do already now for long time.

 fossil bisect reset

      Reinitialize a bisect session.  This cancels prior bisect history
      and allows a bisect session to start over from the beginning.

 fossil bisect run [OPTIONS] COMMAND

      Invoke COMMAND repeatedly to run the bisect.  The exit code for
      COMMAND should be 0 for "good", 125 for "skip", and any other value
      for "bad".  Options:

         -i|--interactive    Prompt the user for the good/bad/skip decision
                             after each step, rather than using the exit
                             code from COMMAND

The help text of run for the new option may have to be updated as well by the addition that in linear mode the first bad response will stop the whole continuation - in opposite to the bisection method where the upper or lower section would be selected only where to continue next.

How to select the stepping direction from good to bad or from bad to good ? Is there an option for this to select?

(2) By Stephan Beal (stephan) on 2022-06-23 13:29:17 in reply to 1 [link] [source]

The help text of run for the new option may have to be updated as well by the addition that in linear mode the first bad response will stop the whole continuation ...

Is that not implicit in this snippet:

      Invoke COMMAND repeatedly to run the bisect.  The exit code for
      COMMAND should be 0 for "good", 125 for "skip", and any other value
      for "bad".  Options:

Presumably someone who uses bisect understands the semantics of "good" and "bad" in that context, or is that too native/optimistic of me? The latter is definitely a possibility, but when looking at the help text to implement your suggestion i'm not finding a formulation which is as succinct and accurate as what it already says.

(3) By MBL (RoboManni) on 2022-06-23 17:28:24 in reply to 2 [link] [source]

how about:

LINEAR automatic traversal stops with the first appearance of "bad".

Regarding the stepping direction I am thinking about the choice from old to new along a timeline of a branch or from new to old. In one case I am interested to find the first appearance of something and in other case the last appearance of something specific. Between the first and the last appearances there might be more than one check-in; I would like to be able to find that timeframe range of check-in's that show the appearance.

I hope to be understood - english is not my mother tongue.

(4) By Stephan Beal (stephan) on 2022-06-23 21:14:42 in reply to 3 [link] [source]

LINEAR

Rather than add that to the 'help' text, since there's not really a good place for it ("linear" is not mentioned anywhere in that text), it's been added to the "options" list:

$ f bisect options
  auto-next        on      Automatically run "bisect next" after each "bisect good", "bisect bad", or "bisect skip"
  direct-only      on      Follow only primary parent-child links, not merges 
  display          chart   Command to run after "next". "chart", "log", "status", or "none"
  linear           off     Do a linear scan rather than a true bisect, stopping at the first "bad" result

That will be checked in soon and more documentation of the linear feature is pending before the release.

Regarding the stepping direction ...

i'm not 100% certain, but it looks like it will step in the direction from the current version to "bad", whichever direction that is.

(5) By MBL (RoboManni) on 2022-06-24 09:07:13 in reply to 4 [link] [source]

linear off Do a linear scan rather than a true bisect, stopping at the first "bad" result

Yes, I agree, that this is enough hint at the right place.

(6) By MBL (RoboManni) on 2022-07-24 09:58:48 in reply to 1 [link] [source]

In initial post I asked

How to select the stepping direction from good to bad or from bad to good ? Is there an option for this to select?

And now, where the new fossil version 2.19 [1e131febd3] 2022-07-21 16:10:55 UTC is out as fossil.exe for windows, too, I tried the new feature to find out the answer to my question. I want to share by observations for the community now.

1st the direction from older to newer checkin should be linear processed:

D:\Sandbox>fossil.exe reset
D:\Sandbox>fossil.exe bisect option linear on
D:\Sandbox>fossil.exe bad <newerHash>
D:\Sandbox>fossil.exe good <olderHash>
D:\Sandbox>fossil.exe run sqlite3.exe
sqlite> .exit 0
sqlite> .exit 0
sqlite> .exit 0
.... and so on

2nd the direction from newer to older check should be linear processed:

D:\Sandbox>fossil.exe reset
D:\Sandbox>fossil.exe bisect option linear on
D:\Sandbox>fossil.exe bad <olderHash>
D:\Sandbox>fossil.exe good <newerHash>
D:\Sandbox>fossil.exe run sqlite3.exe
sqlite> .exit 0
sqlite> .exit 0
sqlite> .exit 0
... and so on
  1. The sequence of calls has always to be the order of bad hash first and then the good hash.
  2. But the order of the hashes determines the linear walking direction on the timeline from checkout to checkout.

I really like this new feature !

The prior use of fossil.exe bisect ui before bisect run seems to unexpectedly reset the option linear to off.

Is it possible somehow to provide parameters to the COMMAND? I did not yet succeed in adding them.

(7) By hanche on 2022-07-24 11:05:51 in reply to 6 [link] [source]

I think the usual procedure is to write a small ad hoc shell script for the COMMAND. It can, in turn, call other commands with whatever parameters you’d like.

(8) By Stephan Beal (stephan) on 2022-07-24 14:41:31 in reply to 6 [link] [source]

I tried the new feature to find out the answer to my question. I want to share by observations for the community now.

We admittedly overlooked that feature in the brief run-up to the 2.19 release. Richard implemented linear bisect to support an acute need he had in another project but the feature hasn't otherwise seen any use nor does it have full documentation. (It occurs to me now that it's also not in the 2.19 changelog, which is arguably a good thing as it gives us time to clean it up for 2.20.)

Is it possible somehow to provide parameters to the COMMAND? I did not yet succeed in adding them.

Bisect has never supported that, presumably because fossil's CLI flag handling is inherently incompatible with doing so. For example:

fossil bisect run my-command -i -x

That -i is a bisect flag and would be consumed by bisect (so not passed on to my-command). The -x flag is not a bisect flag and would cause bisect to fail with something along the lines of "unknown flag: -x". Command handlers in fossil don't really know where in the flag list each flag is, they just ask fossil "is flag -x set?", so they cannot, without going well out of their way to do so, handle special cases such as passing on flags to external commands.

If needed, one workaround is to use environment variables to pass options to your command:

MY_OPT="--foo" fossil bisect run my-command

(9) By MBL (RoboManni) on 2022-07-24 16:09:14 in reply to 8 [link] [source]

Bisect has never supported that, presumably because fossil's CLI flag handling is inherently incompatible with doing so. For example:

fossil bisect run my-command -i -x

but in fossil help bisect you can find the optional [OPTIONS] being placed before the COMMAND on command line ... there would be no ambiguity having optional arguments for the COMMAND call behind them.

Help text:

  fossil bisect run [OPTIONS] COMMAND

      Invoke COMMAND repeatedly to run the bisect.  The exit code for
      COMMAND should be 0 for "good", 125 for "skip", and any other value
      for "bad".  Options:

         -i|--interactive    Prompt the user for the good/bad/skip decision
                             after each step, rather than using the exit
                             code from COMMAND

What I was looking for was

  fossil bisect run [OPTIONS} COMMAND [ARGUMENTS...]

      ARGUMENTS can be forwarded to the COMMAND if they are supplied

(10) By Stephan Beal (stephan) on 2022-07-24 16:49:54 in reply to 9 [link] [source]

being placed before the COMMAND on command line ...

The OPTIONS part has to appear somewhere jn the description and flags conventionally come before non-glag arguments, but fossil's flags parser does not care where the flags go, so long as they're after the command name.

What I was looking for was...

Fossil's args handler can't differentiate between flags for itself and flags for the external command. Such support would require that bisect bypass the framework-level flag handling and do all of that work itself. Even then, command flags which just happened to be the same as a global fossil flag would be consumed (removed from the arguments) before bisect was ever called. Its design precludes uses like the one you're looking for.

(11) By hanche on 2022-07-24 17:01:28 in reply to 10 [link] [source]

Fossil's args handler can't differentiate between flags for itself and flags for the external command. Such support would require that bisect bypass the framework-level flag handling and do all of that work itself. Even then, command flags which just happened to be the same as a global fossil flag would be consumed (removed from the arguments) before bisect was ever called. Its design precludes uses like the one you're looking for.

A (very) common way to deal with this sort of problem is to have a special command flag -- (two dashes) that will stop the program's main command line parser from interpreting the remaining arguments, treating them all the same – whether they start with a hyphen or not. It is not hard to imagine fossil's flag handling to work the same way. The step from imagining this to actually implementing it, on the other hand, could be harder than I would expect, not having looked at the code.

(12) By Richard Hipp (drh) on 2022-07-24 17:10:01 in reply to 9 [link] [source]

I typically run the command (on Ubuntu) like this:

fossil bisect run 'make clean sqlite3 && ./sqlite3 <test-script.txt'

Except, my command-string is normally more complex - I simplified it somewhat here for readability. The key point: Put the entire command string inside a single argument using '...'.

(13) By MBL (RoboManni) on 2022-08-04 17:13:10 in reply to 12 [link] [source]

Thank you very much for all these explanations. However, I am not only using fossil on Linux but also on Windows (even more of the time).

I am now looking for required changes in source code to enable the bisect run command to do other than normal update but just to switch the version to the next checkout WITHOUT modifying the files in the checkout folder.

This is what one can achieve with the fossil checkout VERSION --keep --force --force-missing command; however, the fossil bisect next always uses the update with no options.

In the file src/bisect.c1 in lines 644 to 6562 I found the responsible call of function update_cmd() and the preparation of array g.argv[1] with the "update" string.

Question: would a change of string "update" into "checkout --keep --force --force-missing" cause the expected behavior?

If the answer is yes, then I will have to look next how to add a new bisect option to cause the change in behavior at that point.

If the answer is no, where would I have to look for the desired change in the source code?

    if( pMid==0 ){
      fossil_print("bisect complete\n");
      db_lset_int("bisect-complete",1);
    }else{
      int nSpan = path_length_not_hidden();
      int nStep = path_search_depth();
      g.argv[1] = "update";
      g.argv[2] = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", pMid->rid);
      g.argc = 3;
      g.fNoSync = 1;
      update_cmd();
      fossil_print("span: %d  steps-remaining: %d\n", nSpan, nStep);
    }

You now might want to know also WHY I am looking for such a functionality and the answers are the following:

1) I sometimes get a copy of files from a checkout folder which is months or even years old and may/will contain meanwhile some changes which are not yet reviewed or committed to the repository. Some of these changes are most likely copies from other (newer) checkouts (config file changes). I do not want to alter the local copy but just use fossil gdiff to run WinMerge.exe (Windows application) on the checkout folder with the version in fossil which is just current at that time. My main activity is to identify from where the changes were copied and, in case of missing manifest files, which version copy I got. Finally I will commit what I got into a new branch.

2) Another use-case is similar but I want to collect all version differences which I got to the checkin's in repository into a single log-file. The repeated fossil status respective fossil changes OPTIONS I want to collect into a single log file. This is good to identify a checkin on timeline with the minimum number of deviations to the unknown folder copy.

I know that there are work arounds to achieve the same (and I already use some of them) but with the new bisect option linear on it should be possible much easier to do that work. While the case 1) can be done also with the usage of a shadow copy where WinMerge is used to compare files in folders, the case 2) would mean a lot of error-prone copy actions. I hope to get some hints how to do better WITH fossil as a tool and with less disk space requirements than nowadays.


  1. ^ how to input a hyperlink here to highlight that section with one click?
  2. ^ version 2.19 [1e131febd3] 2022-07-21 16:10:55 UTC

(14) By MBL (RoboManni) on 2022-08-05 09:37:34 in reply to 13 [link] [source]

The Global structure instance g (defined in src/main.c) is used almost everywhere in fossil files. The command function calls are identifiable with _cmd(); suffix. Both, update_cmd() and also checkout_cmd() are existing.

From my source code study the "switch only" activity, which I am looking for, should be achievable simply with the adaptation in bisect.c as shown below.

But what will be the easiest to provide either an OPTION from bisect command line or a bisect option modify flag to the "if then else selection"? Suggestions are appreciated.

    }else{
      int nSpan = path_length_not_hidden();
      int nStep = path_search_depth();
      g.argv[1] = "checkout";
      g.argv[2] = db_text(0, "SELECT uuid FROM blob WHERE rid=%d", pMid->rid);
      g.argv[3] = "--keep";
      g.argv[4] = "--force";
      g.argv[5] = "--force-missing";
      g.argc = 6;
      g.fNoSync = 1;
      checkout_cmd();
      fossil_print("span: %d  steps-remaining: %d\n", nSpan, nStep);
    }