Is it too difficult for Fossil team to provide the [--] switch? How many lines of code would build this option? Two? Three? Plus docs, a real deal indeed.

It wouldn't be difficult nor would it (IMHO) be likely to break anything (nobody can have relied on the current behaviour so far because it doesn't do anything useful). A cursory examination suggests that at least (and possibly only) the following functions would need to be modified (probably 2-3 lines/each):

verify_all_options(), find_option(), has_option() (all in main.c).

All of which would need to recognize --.

As always: patches for changes which scratch a personal itch are welcomed (though they admittedly don't always get applied).

Without the [--], we would have fruitful disputes like that where Stephan defends heroicly the right for Fossil to remain unchanged.

Correction: where i "defend heroically" the stance that one specific feature be retained as-is and, in passing, mention that fossil has historically been averse to "change for change's sake" (as opposed to change for utility's sake).

-- is not change for change's sake IMHO, it's a well-known idiom to say options stop here, everything that follows is a filename or such things, whether it starts with a dash or not.

If Fossil accepts +foo options too, then the same issue would exists for + files too. -- solves this neatly, and there's ample precedent for it.

My $0.02...

If Fossil accepts +foo options too

(Just to ensure that we're not talking about two different things: i understand +foo to mean the opposite of -foo. The getopt man page suggests an alternative use for +foo, but i've never seen it used that way.)

So far fossil has never used + options (as in negated options), and i'm struggling to think a case where it would make sense to do so for fossil. (i'm admittedly uncreative, though - certainly some of you can come up with interesting uses for it.) Historically, relatively few apps have used + this way, probably because their semantics are seemingly reversed from what intuition suggests they should be: "+x removes/negates option x."

OTOH, here's what my local getopt man page says about +:

If the first character is '+', or if the environment variable POSIXLY_CORRECT is set, parsing stops as soon as the first non-option parameter (i.e. a parameter that does not start with a '-') is found that is not an option argument. The remaining parameters are all interpreted as non-option parameters.

but i've only ever personally seen + used as a negated form of a flag.

(To be clear: fossil doesn't use getopt - it's mentioned here only as a reference point, since it's widespread and therefore provides a usage precedent.)

Fossil's argument handling isn't done in a way which is compatible with the above man page snippet: at startup, fossil copies all arguments into a single list without applying any meaning to them, with one exception: --args FILE treats the contents of the given file as a list of arguments to inject into the argument list, replacing the --args FILE part with that list. All argument processing beyond that is left to the routines which consume those arguments (some flags are consumed by the top-level app before any given command is run, but those are consumed in the same way the higher-level routines do it). As a rule, once a command is done collecting its argument, it asks fossil "are there any more arguments left?" and, if so, it errors out with something along the lines of "unknown/unhandled argument." getopt requires knowing, when the options are parsed, exactly which options are available and whether they have optional or required values. Fossil, on the other hand, parses them "agnostically" and expects each command to consume/confirm the arguments it wants. (Fossil also, unlike getopt, treats -foo and --foo identically.)

Once a fossil routine fetches an option, fossil removes that option from its internal list, making it no longer possible to (A) check/consume that flag again and (B) apply any logic such as "all arguments after +foo must be treated as non-flags" (+foo's position in that list, and even its prior existence, has been eliminated by that point).

My point there is only that fossil cannot, without non-trivial reworking, treat +foo as the getopt man page describes. It "could" potentially treat +foo as the negated form of -foo, but i'm having trouble conceiving of a case where doing so would be useful.

I think the biggest difficulty here is that Fossil obeys both GNU-style and Tcl-style option parsing. fossil ci --branch and fossil ci -branch do exactly the same thing. To make a distinction between - and -- in this scheme would therefore make Fossil's argument parsing internally inconsistent. Arguably, -- should be treated the same as -.

It might be worth following the POSIX standard on this, but only after due consideration. There's a trade-off involved.

That -foo and --foo are equivalent is one thing (not a great one, but lets leave that aside). But it does not follow IMHO that - is the same as --, especially since - often is used for stdin in some contexts, while -- is exclusively used in the POSIX sense you mentioned (thanks for the reference BTW).

Arguably, -- should be treated the same as -.

Kind of. If a flag is defined as one or two dashes followed by a non-dash, having - and -- as special cases isn't inconsistent. Fossil already uses - as stdin in a few places (including --args -). Fossil honors -- in one specific place: when looking for --args FILE, -- means "stop looking for the --args flag"). Why it does so there, and only there, and with that very specific meaning, i'm not certain.

i personally see no harm in extending the above-listed functions to treat -- as "end of args list". There would seem to be no possibility of breaking existing scripts/usages via such a change.

i personally see no harm in extending the above-listed functions to treat -- as "end of args list". There would seem to be no possibility of breaking existing scripts/usages via such a change.

It just occurred to me that there are lots of places which will break if we do this:

fossil COMMAND -arg1 -arg2 file1 file2 file3

Commands which follow that pattern (which is more than a couple of them) operate by:

1. Consuming their expected arguments using find_option() and friends. Each call to find_option() removes the requested option from the arguments list.
2. Calling verify_all_options() to confirm that there are no unconsumed options (unconsumed flags are a fatal error).
3. Iterate over all remaining arguments, assuming they are filenames.

If we take the simple approach of having find_option() and friends simply stop scanning at -- then routines which iterate over all non-flag options will see -- as a filename and will misbehave.

This "might" (might) be easily solvable by changing verify_all_options() to consume the first instance of -- from the arguments list.

This "might" (might) be easily solvable by changing verify_all_options() to consume the first instance of -- from the arguments list.

That's what I was thinking, but we will need to audit all 130 uses of verify_all_options() to be sure. We might need to add a flags parameter to verify_all_options() to indicate whether or not the -- should be discarded if it exists, or if -- should be an error.

We might need to add a flags parameter to verify_all_options() to indicate whether or not the -- should be discarded if it exists, or if -- should be an error.

Indeed - commands which don't accept filenames, or only accept them as flag values, should probably error out if given --.

i'm interested in taking on this change, but just got a new puppy last week and keeping her from making a mess everywhere requires more or less full-time observation for at least another couple of weeks.

If nobody beats me to in by the time she's less of a handful, i'll give this a go.

Richard said:

That's what I was thinking, but we will need to audit all 130 uses of verify_all_options() to be sure. We might need to add a flags parameter to verify_all_options() to indicate whether or not the -- should be discarded if it exists, or if -- should be an error.

My current strategy is to disallow that flag by default (it's a fatal error) and to add support for it to commands which can potentially make use of it. Those commands simply replace verify_all_options() with verify_all_options2() (mnemonic: "2" as in "two dashes"). verify_all_options2() removes the first instance of --, stops checking for unused flags, and returns the g.argv index at which it found -- (that index is unfortunately important for cases like (uv add -), in which we need to know whether the -- came before or after the -).

Alternately, we can:

1. Make verify_all_options() simply ignore/silently consume the flag.
2. Make verify_all_options() accept the option as verify_all_options2() currently does, eliminating the need for this distinction.

Since the historical/current behaviour of -- isn't useful/reliable, any approach we take is "probably equally compatible" vis a vis peoples' current usage/scripts. If they're relying on the current behaviour, they're effectively relying on a mild case of undefined behaviour.

And now a call to arms for anyone interested in seeing this support move along rapidly...

There are a lot of commands/subcommands to comb through and determine whether they can/should support this. You assistance in finding those would be greatly appreciated. In essence, almost any commands which accept arbitrary symbolic names or file names are candidates, noting that not all symbolic names in fossil are arbitrary: e.g., wiki page names have syntactical rules which disallow a leading -, so not all wiki commands could make use of -- (those which import/export files can, though).

I've read it all (but, some time ago), and reasoned about it for my own programs (also, some time ago), so here's my input. Please skip ahead if you quickly find out it's flawed or not helpful!

• verify_all_options() checks for --, and if found drops it, and sets a global flag.
• Commands that need to change behavior whether or or not -- is present query the global flag.

Yet I don't see that -- should alter the association of - → stdin. I'd prefer a repeatable --file, --fname, --fsname, --filename, --dirname, --literal, or whatever option here (maybe even allow multiple forms?), to make this case explicit:

• verify_all_options() could replace all --filename=NAME arguments with NAME, and flag them as literal file/directory names, so commands to support stdin could query this flag.
• Or, verify_all_options() could already do the checks for -, i.e. replace --filename=- with ./-, so no need to keep per-argument flags, and no need for each command to query them individually.

The last two points might even be useful to pass literal (non-expanded) globs on the command-line on Windows.

fossil set ignore-glob --literal=*.exe

Yet I don't see that -- should alter the association of - → stdin.

It doesn't alter that association: it provides the calling command with enough information to be able to make that decision. The command still has to decide (if it wants to) whether "-" preceded by "--" should be treated as "./-" or stdin/stdout. i'm currently refactoring that decision into a utility function for commands which treat "-" as stdin/stdout.

There's no new magic going on here - no automatic substitutions except for the removal of the -- flag from the argument list (because we have to remove it in order to avoid Breaking Everything).

Ok, I see. I'm not familiar with - having the "nature" of an option. That's why I thought an explicit --name=- → ./- command-line argument conversion would result in a minimal change set, as everything (also the removal of --) could be done in verify_all_options(), and it might be handy beyond - (for non-expanded globs on Windows).

I'm not familiar with - having the "nature" of an option.

In some, but not all, contexts a filename of "-" is treated as as alias for either stdin or stdout (that's an old Unix convention). The goal of this feature is to give the user a way to bypass that and say, "treat '-' as a literal filename instead of an alias for stdin/stdout." The Unix/POSIX convention for doing this is to use the -- flag to mean "all arguments after this flag have no special meaning." Fossil commands which treat - as stdin/stdout now (with these changes) have enough information to be able to honor that request (and similar requests, such as using a filename of --foo). However... each command which does so has to be slightly modified, and that's where the majority of the work/testing is.

I just passed some time down the rabbit hole at https://rachelbythebay.com after prompting in another comment. Included in that blog is the article:

http://rachelbythebay.com/w/2013/05/02/buffalo/

titled "You don't have to use the whole buffalo".

In essence, because one (or a few) hardy souls want to use "-" as a valid filename do the rest of us have to accommodate that desire?

I think not.

A similar argument happened recently in the golang world about building exceptions into the language. Enough thoughtful people disagreed with doing so despite the pressure of users from the C++, C#, Java, Ruby worlds that the proposal was abandoned.

Add my vote against "-" as a valid filename here. If that is the full justification of this forum thread then I would abandon it all.

Chris

Though I find the discussion and the current approach interesting, I had similar feelings ...

Wouldn't a minimal change to verify_all_options() to translate --filename=- to ./- cover all of this rare corner case?

Then all of the remaining handling for -- could be done in verify_all_options(), as well.

Wouldn't a minimal change to verify_all_options() to translate --filename=- to ./- cover all of this rare corner case?

The real fix is even simpler and doesn't require any code changes: simply require users to type ./- instead of - (in the handful of contexts where - is an alias for stdin/stdout).

- is a perfectly valid option/string in some contexts, e.g. it's apparently valid as a branch name (which would break if we translated it to ./-). If we do any sort of magical conversion of one argument to another, we will break things which don't need to get broken. This change has to have "zero impact" on existing features or it's not worth the trouble of implementing.

This change has a wider impact than just stdin/stdout aliases, though: it enables filenames which look like flags:

fossil add -- --my-silly-file

(fossil add hasn't yet been updated with this feature. That command will be the real test of whether or not this whole thing is worth the effort.)

As Chris points out, supporting such against-any-and-all-conventions filenames is certainly not a sensible requirement, but until/unless this experiment causes any grief, i see no harm in trying it out. (It's certainly far more effort than the "2-3 lines" suggested in the top post, though.)

(fossil add hasn't yet been updated with this feature. That command will be the real test of whether or not this whole thing is worth the effort.)

It turns out that adding this support to add/rm/mv was trivial because none of them use the - alias. That change makes the following legal:

fossil add -- --force
fossil mv -- --force --dry-run  
fossil rm -- --dry-run

(But anyone actually using such filenames probably deserves a hefty dose of Peer Pressure from their fellow developers.)

titled "You don't have to use the whole buffalo".

i agree 100% and i'm not yet committed to the -- support: if it introduces more than the very slightest bit of Grief or starts to feel like bloat, it will be abandoned. So far it's harmless enough, though, its footprint is minimal, and it allows us to have branches named --foo, which is cool in a nerdy kind of way.

Only a handful of fossil routines support - as stdin. It doesn't make sense, e.g., for fossil add because add architecturally requires a reference to a local file. uv add, on the other hand, can make perfectly good use of stdin:

minify-my-js infile.js

fossil uv add --as myjs.min.js -

Some commands parse/process their arguments in a way in which this doesn't make a difference one way or the other whether a filename starts with a dash (tag comes to mind).

Why this example doesn't work as-is, i'm not actually certain off hand:

fossil branch new - trunk
Usage: f branch new BRANCH-NAME BASIS ?OPTIONS?

But, in any case, it does with the addition of -- (in the new branch: the current/historical behaviour of -- depends both on the order of the flags on the CLI and what order they're processed in C):

fossil branch new -- - trunk
New branch: 367938bab2caecbaf7f7a0442219bdfd74a0e62a7f61a6cc0444561671eae6c7

Fossil honors -- in one specific place: when looking for --args FILE, -- means "stop looking for the --args flag"). Why it does so there, and only there, and with that very specific meaning, i'm not certain.

A closer inspection reveals that find_option() does indeed honor -- to mean "stop looking for flags", but it also consumes the -- the first time it's encountered, which means that future calls to find_option() will not know to stop. That particular consumption seems like a bug to me.

If we fixed that, how much of the work that you've done on the double-dash-flag branch could be altered or obviated?

Fixing it just means removing that consumption of that flag from find_option(): if that function consumes --, further calls to find_option() cannot honor --.

Unless i'm sorely mistaken/misremembering, removing the consumption from find_option() implements the majority of this feature set. It's that last 10% which takes up more effort: interpreting, in some contexts, - as variably stdin or a literal file. (Even so, the effort is/was minimal.) Doing that correctly requires knowing whether -- appears before or after the - argument, which the current branch does by remembering where in g.argv -- was seen/consumed.

Why would the insertion of '--' in the arguments for this subcommand ever mean that Fossil should treat that '-' as a filename instead of stdout?

That derived from an earlier discussion with the poster who requested this feature. In hindsight, that part very probably doesn't belong here.

This patch isn't quite right:

fossil foo -bar -- -baz

would work, or not, depending on which order find_option() is called:

find_option("x",..._);
find_option("bar",..._);
find_option("baz",..._);

The first call would consume --, treating -baz as a flag. The bahaviour depends on the order in which args are provided on the CLI and the order they're processed in C.

find_option() (and friends) need to simply stop when they see --, and verify_all_options() needs to silently consume it (then stop looping) so that commands which loop over g.argv after calling that function do not see the -- flag.

In any case: i've abandoned the double-dash-flag branch and will soon create a new branch with a much simpler patch... as soon as my puppy allows me to :/. (Sometime this evening, in any case.)

In any case: i've abandoned the double-dash-flag branch and will soon create a new branch with a much simpler patch... as soon as my puppy allows me to :/. (Sometime this evening, in any case.)

https://fossil-scm.org/fossil/timeline?r=double-dash-flag2

Some commands operate by reconstructing a new argc/argv, based on but differing from the original argc/argv, then restarting a new command using the new argc/argv. In such cases, we want verify_all_options() to run multiple times and do its job each time, do we not?

I looked around for an answer to this and the first example I came across is "fossil stash save" which does some processing and then invokes "fossil revert".

https://fossil-scm.org/fossil/artifact/c541e3bb5?ln=620

I seem to recall there are others, but I don't have an list, nor do I know of a good technique to create such a list.

Thoughts?

FWIW, the dash branch actually had such an assertion, but i removed it before the first commit because i was concerned about breaking something via an inadvertent second call. It was also my understanding that it must only be called once - commands modifying argv and re-dispatching is new news to me.

...or simply change the comment?

Done!

https://fossil-scm.org/fossil/timeline?r=double-dash-flag2

Is there any objection to merging this in?

The "real" changes are here:

https://fossil-scm.org/fossil/vdiff?from=be7c1bde92c8caae&to=6edf8bcd9ab0784a

All other changes shown in the trunk-to-branch diff are from me having merged in trunk. Well over half of that diff is documentation additions, not code changes.

Nobody could anyone have relied on the -- handling before this change, so it seems highly unlikely that there could be any backward-compatibility problems.

I'm up to my eyeballs in other stuff, so please take care of this for me.

That only covers part of the story, though: in some cases we need to know whether -- appears before or after -, so that - can be (in those few contexts which care) interpreted as stdin or a filename:

fossil wiki export pagename - # stdout

fossil wiki export pagename - -- # stdout

fossil wiki export pagename -- - # filename

Shouldn't the decision to interpret '-' as an alias for stdout depend on the subcommand, not the position of '-' in the argument list?

It's both at the same time:

fossil wiki export pagename -

Historically, that's used - to mean stdout.

fossil wiki export pagename -- -

When verify_all_options2() is called, -- gets consumed, placing - in the exact same position as it is in the first example. However, its interpretation changes because the export command now has enough information to ask whether - was "escaped" via --:

https://fossil-scm.org/fossil/artifact?udc=1&ln=1473&name=402cef20e9c0e584

Where get_dash_filename() takes a g.argv index: if that index is at or after the position where -- was consumed (by verify_all_options2()) AND the argument is precisely - then ./- is returned, otherwise the argument is returned as-is.

Exactly :). However... given that grep and friends do not use -- to escape -, i am now 90-ish% convinced that combining these is probably not the right thing to do. If forcing a user to type ./- when they mean literal filename - is good enough for grep, it's good enough for me.

i'm currently in the process of stripping that part of the handling out, which will, in essence, boil the thole thing down to your patch.

There's still time to talk me out of doing so, though, in the case that you and i have effectively swapped positions in the mean time ;).

Also, whether or not '-' means stdout or a literal filename is a separate discussion from whether or not '--' means end of arguments in my opinion (as I tried to illustrate in previous posts).

(Sorry, forgot to answer that part.)

Yes and no. i'm convinced that handling it as part of the same change is easy enough, and semantically fits well enough, that they belong together (in this context).

That said: grep does not use -- to change the meaning of -, so this aspect of my change may well be considered ill-advised.

date > ./-
grep 16 - # reads stdin
^C
grep 16 -- - # reads stdin
^C

Given that, i would not be averse to removing the - handling, and simply tell users that ./- is the only way to support a filename of - for commands which normally treat it as stdin.

Hi there,

Quoting:

> $ echo hello | cat - --  
> hello
> cat: --: No such file or directory
> $ echo hello | cat -- -
> hello
> 
> So in this case, cat treats -  as stdout *always* but somtimes it treats
> -- as a literal filename. :-)

Sort of, but not really.

"--" as an option generally means "this is the last option; any argument that follows is a non-option argument, even if it might look like an option in other circumstances".

And: options come before non-options.

(If my use of the terms option, non-option, or argument, are unclear, I can try to define them separately.)

In the first case above: "cat" does not take "-" as an option, therefore it is a FILE argument, and therefore anything after it is another FILE argument. Separate from that, "cat" treats the string "-" as "stdin". So "cat" will concatenate the content of two files: its stdin, and the file "--".

In the second case about, "cat" does take "--" as an option, and therefore concatenates the content of one file: its stdin.

(Compare the (probable) error message difference between "cat --input" and "cat -- --input".)

> $ ls - --
> ls: -: No such file or directory
> ls: --: No such file or directory
> $ ls --
> --name

Similar thing here:

"ls" does not have an option "-", so "-" is the first FILE argument, and "--" is the second.

"ls" does have an option "--", so there are no FILE arguments, and "ls" lists the content of "." (its default).

> Should '-'  ever be considered a  literal filename? If so,  how does the
> user indicate that he wants to use  a filename of '-' instead of have it
> be interpreted as stdout?

The answer to the second is "./-".

The answer to the first is an opinion; but I'd say "not if the command can reasonably use stdin or stdout", and "maybe otherwise" -- and the "maybe" is because it is probably easier not to write code to specially avoid handling "-" as a filename.

Any command that does use "-" as stdin or stdout should document that.

Cheers

Arguably, -- should be treated the same as -.

Not really. Both -- and - have meanings in the POSIX Shell guidelines.

-- is to be treated as end of option arguments. This allows for filenames that begin with -

- is optionally treated as a pseudo option indicating standard in or standard out, depending on context.

While the treatment of - is optional, using it as an equivalent to -- would be confusing.

This might be a resolution to the - impasse in another thread. After the other options, have -- then the file list, including allowing - as a filename rather than as a pseudo option for standard in or out.

How many lines of code would build this option? Two? Three?

Work has started on this, but it will end up being a couple hundred lines of additions/changes, as every routine which uses the argument processing routines has to be modified for this so that they can specify whether -- is legal or not for that command.

In short, -- is generally only legal for commands which accept an arbitrary number of file/wiki/branch/whatever name arguments after their required arguments. Commands which expect names to be provided either in specific argument positions or as flag values will not be affected the same way by this change (i.e. they'll very probably treat -- like an unexpected/unhandled flag, whereas that flag is currently effectively ignored (kind of: the current handling of -- appears to have a fundamental flaw)).

This work is in the double-dash-flag branch.

That said: my new puppy :) is taking considerably longer to house-train than anticipated :(, which means she has to taken outside every couple of hours 24/7 and i don't get much sleep, so there is no ETA for when this will be complete. Of course, anyone with checkin access is free to pitch in! The infrastructure-level changes have been made in the above branch, and the real work now involves going through all uses of verify_all_options() and figuring out which ones can/should allow -- and which cannot/should not, and modifying the verify_all_options() calls to reflect that by replacing them with verify_all_options_porting_crutch(FLAG_VALUE) (that routine will be renamed (back) to verify_all_options(FLAG_VALUE) once porting is complete).

Allrighty...

The double-dash-flag branch seems to be more or less complete. It was wide-reaching but easy/painless to implement. The branch's wiki entry (that's such a great feature!) lists all of the commands which were updated and a handful of commands which were not because doing so would potentially break existing usage. Each modified command underwent manual testing. Many commands simply can't make any sensible use of --, so they weren't updated.

The complete diff is here.

This feature works by adding a second variant of verify_all_options() which consumes the -- flag and records its position in the argument list so that we can later ask (if needed) whether argument - came before or after that flag. A handful of functions which treat - as a stdin/stdout alias will now treat - as a literal filename if the -- flag precedes the -.

After having made this change, i consider it to be harmless/risk-free (and possibly even desirable) to consolidate this new -- handling into verify_all_options(), as opposed to having a 2nd variant of that function. @Richard: if you agree with that assessment i'll update the branch to do so. The historical behaviour of -- was such that users could not depend on it for anything useful, so backwards compatibility of that flag is not a concern (see the branch's wiki entry for a compatibility analysis).

@Anyone interested in this support: please check out and test this branch!

- is a perfectly valid option/string in some contexts, e.g. it's apparently valid as a branch name (which would break if we translated it to ./-).

Ah, yes. So a "this is a literal argument, not an option" option would require two forms, one for plain strings (branch names, etc.), and one for file names, and the latter should prefix ./ to anything starting with one or more dashes -.

The Obligatory Exceptions

• stash ...
• search ...
• tag does not use verify_all_options(), instead using strictly positional arguments, thus it supports tags with leading dashes as-is.

Then, shouldn't these commands be updated, in the context of the current change, for consistency with the other commands? This wouldn't break backwards compatibility, but add flexibility?

And, is it necessary to add an extra syntax summary/example line with ?--?, and an explanation of -- to each individual command supporting it? It's a global option, *nix people know how to use it, and commands not supporting it should simply remove and ignore it? If we have it, why not stick to existing standards as closely as possible?

Another topic: If Fossil is spawning itself (likely more for the server/http/backoffice-related commands), or external programs (gdiff), should it maintain/forward the -- option to the child process? It seems inconsistent if Fossil accepts file name arguments for gdiff that the external diff utility won't be able to handle, or will treat them as options. But, at least the commonly used Windows graphical ui diff utilities (the venerable WinDiff, and the enhanced WinMerge) don't know anything about --. Moreover, Fossil already applies harder restrictions to file name arguments passed to external utilities, and allowing -- would create a new field of inconsistencies.

A simplified approach to prefix ./ to anything declared being a file name and starting with a dash - could probably solve this?

As usual for "easy" changes, quite some brainwork, just to enable typing - instead of ./- ;-)

Ah, yes. So a "this is a literal argument, not an option" option would require two forms, one for plain strings (branch names, etc.), and one for file names,

It's one flag with, in some contexts, both meanings. There is no Unix-conventional 2nd form for this flag: the conventional workaround for using - as a literal filename is for the user, not the software, to use either -- (if the software supports it) or type ./-.

Then, shouldn't these commands be updated, in the context of the current change, for consistency with the other commands? This wouldn't break backwards compatibility, but add flexibility?

i'm not willing to potentially break compatibility with this change. If it gets moved to trunk, we can discuss breaking backwards compatibility for bringing those few commands up to date. i agree that they should be updated, but i'm not willing to "break" them in the context of this change (where "break" really means "change behaviour in a way which might break certain existing uses").

And, is it necessary to add an extra syntax summary/example line with ?--?, and an explanation of -- to each individual command supporting it? It's a global option,

1. It's not yet a global option. Not all commands can make any use of it, so not all commands activate the -- handling.

2. For some small number of commands it provides the additional behaviour of treating - as a literal filename, for others it does not (because those don't use - to mean stdin/stdout). Thus we unfortunately must document those separately.

Another topic: If Fossil is spawning itself (likely more for the server/http/backoffice-related commands), or external programs (gdiff), should it maintain/forward the -- option to the child process?

i intentionally did not modify any commands which respawn fossil or an external process. Those behave exactly as before.

A simplified approach to prefix ./ to anything declared being a file name and starting with a dash - could probably solve this?

It's not that straightforward:

fossil wiki export MyPage.md -

Do i, as a user, intend - to mean stdout (as is the Unix convention for that filename) or a literal filename? We cannot "declare" a parameter to mean one or the other - it can mean both. In the above case, the command continues, as it always has, to treat - as stdout unless the user specifies otherwise using either:

fossil wiki export MyPage.md ./-

or (with this change):

fossil wiki export MyPage.md -- -

For some commands (like tag) a leading dash is never treated specially (because the command does not call verify_all_options()), so they support symbols with leading dashes without any changes.

Option flags which accept a value also accept values with leading dashes, so:

fossil timeline -R --foo

Will treat --foo as a repo filename. That's not a new behaviour, just a side-effect of how the argument parsing has always worked.

As usual for "easy" changes, quite some brainwork, just to enable typing - instead of ./- ;-)

That's only one aspect of this change. The primary aspect is that we can now have filenames with leading dashes without requiring the user to prefix all of them with ./:

fossil add -- --foo --bar --baz

The - behaviour is a secondary feature provided in the same context.

... We cannot "declare" a parameter to mean one or the other - it can mean both. ...

Sorry for the confusion, I was still talking about the alternative solution mentioned earlier.

Meanwhile, a real-world example (typed even more frequently than fossil) has crept from my "muscle memory" (dixit stephan) to my mind: The Windows built-in grep utility. To search for STRING, one can type either of:

• findstr "STRING"
• findstr /c:"STRING"

And, hence, one can type:

• findstr /c:"/c:"

(All quotes are optional -- "muscle memory".)

The /c: option encapsulates a literal string that may look like an option. Something like this could be added to Fossil, with a 2nd variant to declare the argument as a file name.

... i'm not willing to potentially break compatibility with this change. ...

From time to time, Fossil has backwards-incompatible changes°. If the -- changes are accepted for the mainline, I think it might be a good opportunity to also do the (potentially, but not sure) incompatible changes and harmonizations, and announce them as one group in the change log.

° Recently, Fossil changed the SQLite compile-time options to disallow double-quoted string literals, so I had to fix my SQL repository initialization, blob diffing, and other utility scripts -- that is, after I finally found the reason ;-)

The /c: option encapsulates a literal string that may look like an option. Something like this could be added to Fossil, with a 2nd variant to declare the argument as a file name.

Fossil already supports that approach: any flag which takes a value will read the part which follows the flag as a value, even if it looks like a flag. However, if we added separate flags for every command which accepts - to be stdin/stdout, it would be a more code than the current change (a new flag for every such command) and it would be unconventional (-- has a history of being used for that purpose). The -- handling is in a central place and requires only tiny changes to the commands which can make use of it (more docs than code).

From time to time, Fossil has backwards-incompatible changes. If the -- changes are accepted for the mainline, I think it might be a good opportunity to also do the (potentially, but not sure) incompatible changes and harmonizations, and announce them as one group in the change log.

i agree completely, but this change was made to scratch someone else's itch, not my own, so i honestly lack the inspiration to defend that stance in a debate. The "real" chance of breaking anyone's current usage with those changes would be near zero - it's strictly a hypothetical risk.

Recently, Fossil changed the SQLite compile-time options to disallow double-quoted string literals...

Sidebar: my understanding is that double-quoted strings were only added for compatibility with MySQL (which was still extremely populate back before sqlite took over the world). My SQL work started with MySQL and i had to unlearn several of its bad/sloppy habits when i finally started working with stricter SQL servers (Oracle and Postgres).

Fossil already supports that approach: any flag which takes a value will read the part which follows the flag as a value, even if it looks like a flag.

Yes, but only for option arguments, not for primary arguments:

> fossil ci --branch --sillybranchname ...
> fossil co --sillybranchname
unrecognized command-line option, or missing argument: --sillybranchname

Note that the following works, however:

> fossil co tag:--sillybranchname

And this would also work:

> fossil co /c:--sillybranchname

However, if we added separate flags for every command which accepts - to be stdin/stdout, it would be a more code than the current change (a new flag for every such command) and it would be unconventional (-- has a history of being used for that purpose). The -- handling is in a central place and requires only tiny changes to the commands which can make use of it (more docs than code).

Not separate flags, but two global flags, all handled by verify_all_options(). The hypothetical /c and /f (for file names) switches would simply convert this:

...
argv[1] = "/c";
argv[2] = "--sillybranchname";
argv[3] = "/f";
argv[4] = "--sillyfilename";

To this:

...
argv[1] = "--sillybranchname";
argv[2] = "./--sillyfilename"; /* sanitized */

This could all be done in verify_all_options(), without any modifications and without any new options for any commands. The sanitized file names (with ./ prefixed to leading dashes -) could even be reused as arguments for child processes, and - would become ./- to explicitly declare it as a file name, so no need to care about it at the individual command level.

Back to the current branch discussed here: I think some things are flawed.

Shouldn't the following add the unversioned file named -, which is present in the current directory (see output of dir /b)?

dir /b
-
fossil uv add -- -
(prompt for stdin)

And, the following works in an unexpected way:

fossil uv add -- - -R sample.fossil

If -- means any remaining arguments are not options, then I would expect to see a command-line syntax error message, since -R and sample.fossil should both be literal arguments, now?

And this would also work:

fossil co /c:--sillybranchname

i would rather cut off my hair than add Windows-style "slash" flags to a piece of software :/.

This could all be done in verify_all_options(), without any modifications and without any new options for any commands. The sanitized file names (with ./ prefixed to leading dashes -) could even be reused as arguments for child processes, and - would become ./- to explicitly declare it as a file name, so no need to care about it at the individual command level.

There's no way that verify_all_options() can reliably know whether or not it's legal to add ./ to any given argument. "Magic" conversion always leads to Grief at some point - i'm not willing to do that.

Shouldn't the following add the unversioned file named -, which is present in the current directory (see output of dir /b)?

fossil uv add -- - (prompt for stdin)

If you're getting a prompt for stdin then you're not using the new branch. The new branch will import that as a file:

[stephan@lapdog:~/fossil/blah]$ l
...
-rw-rw-r--  1 stephan stephan    30 Sep 28 01:40 --2date
-rw-rw-r--  1 stephan stephan    97 Sep 28 01:41 --date
-rw-r--r--  1 stephan stephan 61440 Sep 28 01:43 .fslckout
[stephan@lapdog:~/fossil/blah]$ date > ./-
[stephan@lapdog:~/fossil/blah]$ f uv add -- -
[stephan@lapdog:~/fossil/blah]$ f uv cat -- -
Tue Oct  1 14:23:42 CEST 2019

And, the following works in an unexpected way:

fossil uv add -- - -R sample.fossil

If -- means any remaining arguments are not options, then I would expect to see a command-line syntax error message, since -R and sample.fossil should both be literal arguments, now?

The new branch will never see -R as a flag in that case. It will instead see 2 literal filenames: -R and sample.fossil:

[stephan@lapdog:~/fossil/blah]$ ../fossil/fossil uv ls -v
[stephan@lapdog:~/fossil/blah]$ ../fossil/fossil uv ls 
-
-R
sample.fossil

i would rather cut off my hair than add Windows-style "slash" flags to a piece of software :/.

So would I, I was sticking to /c for the sake of the discussion, to make it clear what I was referring to. I'm sorry, I didn't mean to be offending to the *nix people in any way.

There's no way that verify_all_options() can reliably know whether or not it's legal to add ./ to any given argument. "Magic" conversion always leads to Grief at some point - i'm not willing to do that.

That would require a 2nd form of the option, named /f in the example above (again, no offence).

If you're getting a prompt for stdin then you're not using the new branch.

Oh, no, there it is. Wrong branch. I'm really sorry! I will try the correct branch -- but probably only in the next days, since I get a lot of compiler errors with MSVC, and I haven't time to look into it, at the moment.

I will try the correct branch -- but probably only in the next days

There's no rush - i'm not looking to have this branch considered for inclusion until after 2.10 is released.

i would rather cut off my hair than ...

Sidebar: Today i actually got my first "real" haircut in about 20 years :/ (and it had nothing to do with protesting Windows-style flag conventions!). This is the first time my hair has been above my shoulders since 2000 :/.

i would rather cut off my hair than add Windows-style "slash" flags to a piece of software :/.

"slash" flags aren't Windows style - they are DOS style, so really old and decrepit. Many of the more modern Windows CLI commands use "dash" instead of "slash" - see netstat (or accept both, like ipconfig.)

But either way, yea, I hate "slash" flags as well (and I'm primarily a Windows and PowerShell user).

"slash" flags aren't Windows style - they are DOS style, so really old and decrepit. Many of the more modern Windows CLI commands use "dash" instead of "slash" - see netstat (or accept both, like ipconfig.)

While CP/M didn't apparently use / as a switch or option character, some CP/M programs did. From what I've read, it was actually IBM that dictated use of / as a switch character. When DOS added subdirectories, it wanted to use /, but IBM insisted it must be different. DOS had an API for setting what the switch character should be, but not all programs used it. COMMAND.COM did so that you could type something like DIR/W without a space, and it would know / was the switch. If one used the API to change the switch character, it would be possible to type DIR-W instead. Some Microsoft engineers were known to change the system switch character to dash so that they could use POSIX style paths instead.

Many of the more modern Windows CLI commands use "dash" instead of "slash"

Historical side note: Some of the test systems where I work still use PCs that run MSDOS. Somehow, they were configured to use - instead of / for options. And they accept both / and as path separators. This annoys most of the test engineers, but nobody knows how to change it. No idea what will happen when all the spares are used.

Converting /f--sillyfilename to ./--sillyfilename is redundant. We already have ./ which is the same number of characters as /f

Also, I agree with Stephan about / style options: No.