Fossil User Forum

Probably a bug in 'fossil uv'
Login

Probably a bug in 'fossil uv'

Probably a bug in 'fossil uv'

(1) By Alex P (aplsimple) on 2019-09-06 15:59:25 [link] [source]

This works:

cd BIN
mkdir 1
fossil uv add *
fossil uv ls

But this works not:

cd BIN
mkdir -
fossil uv add *
fossil uv ls

That directory named "-" hangs Fossil at adding uv.

Independently on is the directory empty or not.

(2.2) By Stephan Beal (stephan) on 2019-09-06 16:27:19 edited from 2.1 in reply to 1 [link] [source]

The filename "-" is conventionally (in Unix-based programs) used as an alias for stdin or (depending on the context) stdout. What is almost certainly happening here is that fossil sees a name of "-" and is awaiting input from stdin.

FWIW, this is not a fossil-specific problem, but is a collision between common conventions and end-user practice. In short: don't use files/directories named "-". (Edit: likewise, don't use names which start with a -, as most Unix-style apps will treat those as flags unless they're prefixed with a path component.)

Note that removing or renaming such a directory will (on a Unix-like system) most likely require adding a "./" prefix to it.

rmdir ./-

or:

mv ./- foo

Edit: or apparently not - on my system rmdir and mv can handle those without a ./ prefix. Some commands cannot, though. e.g. grep treats it as an alias for stdin unless it is prefixed with a path component and (touch -) does absolutely nothing, implying that it's uselessly "touching" either stdin or stdout.

(3) By Alex P (aplsimple) on 2019-09-06 16:30:47 in reply to 2.1 [link] [source]

that Unix, a source of adrenalin :)

Thank you a lot!

(4.1) By Alex P (aplsimple) on 2019-09-07 19:18:05 edited from 4.0 in reply to 2.2 [link] [source]

Nonetheless, I think it's a bug 100%.

"fossil uv add" waits for input after "-" in ANY position. None behaves such way.

ALL OF

fossil uv add -
fossil uv add -- -
fossil uv add -- - qqq

HANG UP. THOUGH

fossil add -

HANGS NOT. It barks at me, sort of "not found: -" as usual.

And * expanding with "-" among other files means the same as the examples above.

(5) By anonymous on 2019-09-07 19:27:07 in reply to 4.1 [link] [source]

What happens if you do:

echo "This is a test" | fossil uv add -

(6) By Stephan Beal (stephan) on 2019-09-07 19:48:20 in reply to 4.1 [link] [source]

Have you trued using ./- instead? Wildcard expansion of a file named - is functionally identical to passing - on the command line, so no difference is expected.

(fossil add -) does not support reading file contents from stdin because every SCM'd file requires a name. That is apparently not the case with (fossil uv add), which internally works much differently from (fossil add).

The uv command should probably not treat - as stdin, but if it does then i have to assume that Richard had a good reasoning for allowing it.

(tap... tap tap... tap...)

The answer can be found here:

https://fossil-scm.org/fossil/help?cmd=uv

Note the --as option. The command can read from stdin and give the file a name of the user's choice.

Not a bug. It's arguably a bug that --as is not required when reading from stdin, though.

(7) By Alex P (aplsimple) on 2019-09-08 05:11:04 in reply to 5 [link] [source]

Yes, thank you, the workaround for this issue:

echo "" | fossil uv add -

really adds "-" to uv ls.

and

echo "" | fossil uv add *

really adds all of the current directory to uv ls.

But it's only a workaround, not a solution of problem.

(8.1) By Stephan Beal (stephan) on 2019-09-08 11:41:17 edited from 8.0 in reply to 7 [link] [source]

But it's only a workaround, not a solution of problem.

My response from yesterday explains why/how this is not a bug and why/how it is a feature.

The "problem" (if it can be called that) is that you are violating Unix naming conventions which predate Ronald Regan's presidency, and then expecting tools which follow those conventions to be "fixed." As i demonstrated earlier, common tools like grep behave the same way when provided the filename -.

The "fix" here is to stop using the name - for a directory or file. Instead, use % or _, or any other character which has no de facto standard conventions commonly applied to it.

Try mkdir ~. The name ~ is magical only by long-standing convention, not by a Law of Computing or filesystem limitation. After trying that, try:

mkdir ./~
cd ~

Similarly, naming a file or directory with only whitespaces:

mkdir "    "

or, even better:

mkdir $(echo -e "\n\b")

and then try looking for that directory from the CLI, using a graphical file manager, and Midnight Commander (the results are different for each one).

Unconventional names will cause grief of some sort or other, and the fault is not with the tools - they're simply conforming to age-old conventions associated with these characters.

(9) By anonymous on 2019-09-08 14:05:02 in reply to 8.1 [link] [source]

Having used Linux (and other Unix/POSIX systems) since I was a teen, I am certainly very familiar with - being used as a proxy for standard in.

However, does

command | fossil uv add - --as filename

really make sense in today's culture? I rarely use - to mean standard in, and have never used it with Fossil.

It might be better to make fossil uv add consistent with fossil add

~ is handled by POSIX compliant command shells (and some non-compliant shells, too). In a POSIX environment, Fossil wouldn't see it without extra effort by the user. I've never put in the effort, so I don't know if Fossil attempts to expand it.

(10) By Stephan Beal (stephan) on 2019-09-08 14:20:53 in reply to 9 [link] [source]

The fact is that the current behavior allows uses which would otherwise not be possible (e.g. creating downloadable tarballs or minified JavaScript without intermediary temp files). Removing this feature would disallow such potentially useful cases while providing literally no benefit to counter that loss.

(11) By Alex P (aplsimple) on 2019-09-08 18:12:05 in reply to 8.1 [link] [source]

The "-" and "+" are both absolutely legal as file names. I like them. They are short and meaningful.

Again, I try these:

fossil uv add -- -

and this hangs, though I force it to look at "-" as a file name.

Then I try

fossil uv rm -

and this does not hang, though I don't force it anyhow. Why, it should behave as "uv add", not?

Then I try

fossil add -

and it works OK. The "uv add" only behaves differently, not like other fossil commands. I consider this inconsistency a bug. You may be of other opinion.

After all, there is a workaround pointed by anonymous. Thank you both for the dispute.

(12) By anonymous on 2019-09-08 18:13:05 in reply to 10 [link] [source]

would disallow such potentially useful cases while providing literally no benefit to counter that loss.

I used to use - a lot. Now, with the wisdom of experience, I don't see those use cases as being particularly useful.

When I create tar/zip/whatever -balls, I always test them before "publishing" them. For that, I need the intermediary files.

Except for simple cases, having the intermediary files makes trouble-shooting much easier. (The real world always has more corner cases.)

In today's GUI world, users are used to fewer restrictions on file names than the command line world that you and I grew up in.[1]

So, which is the larger user surprise: An otherwise valid file name being rejected, or having to use intermediary files?

I think that the greater benefit for Fossil comes from having fossil uv add accept - as a file name just as fossil add does.

(Unfortunately, there are more valid file names that are a challenge for command line usage [1], but, I think, those all have to be dealt with at the command shell level, so outside of Fossil's scope.)

[1] At work, other people often send me files with & and other problem characters. For example

    `DV' Test Plan & Report.docx

Then, if I rename them and later have to send back an edited version, I get complaints that the files were renamed.

(13) By Alex P (aplsimple) on 2019-09-08 18:15:14 in reply to 8.1 [link] [source]

The "-" and "+" are both absolutely legal as file names. I like them. They are short and meaningful.

Again, I try these:

fossil uv add -- -

and this hangs, though I force it to look at "-" as a file name.

Then I try

fossil uv rm -

and this does not hang, though I don't force it anyhow. Why, it should behave as "uv add", not?

Then I try

fossil add -

and it works OK. The "uv add" only behaves differently, not like other fossil commands. I consider this inconsistency a bug. You may be of other opinion.

After all, there is a workaround pointed by anonymous. Thank you both for the dispute.

(14) By anonymous on 2019-09-08 18:38:07 in reply to 11 [link] [source]

Actually, I meant

    echo "This is a test" | fossil uv add -

as a diagnostic, not a work around.

The work around would be to use ./- as the filename, as Stephan pointed out.

Despite using - as Stephan describes, I actually agree with you.

fossil uv add - is inconsistent with fossil add -

Not all Linux/Unix command line tools use - as a proxy for standard in.

Just as it is reasonable for fossil add - to treat - as a filename, it is equally reasonable for fossil uv add - to do so as well.

I think that not surprising new users is a greater benefit than not inconveniencing expert Linux/Unix shell programmers.

And I say that as a Linux/Unix shell programmer with many years of experience.

(15) By Stephan Beal (stephan) on 2019-09-08 20:00:22 in reply to 11 [source]

Again, I try these:

fossil uv add -- -

and this hangs, though I force it to look at "-" as a file name.

The -- flag does not have that meaning in fossil: it simply tells fossil to stop looking for the --args flag:

https://fossil-scm.org/fossil/artifact?ln=426&name=52d2860fa03ae636

The "-" and "+" are both absolutely legal as file names. I like them.

i don't dispute that they're legal, but you must also recognize that they're highly unconventional and that - has long-standing conventional uses as a pseudo-filename.

i don't dispute that there are inconsistencies in how - is handled in fossil, but those inconsistencies are such far-edge cases that i personally consider them not worth addressing with code changes. They may not even be inconsistencies. e.g. fossil add - can only refer to a real file because not doing so would play havoc with its internal ideals about how files are addressed.

In my nearly 12 years of following this project, you are (unless i'm sorely misremembering) the first person who's raised any concerns about handling of - as a genuine filename. That implies something about how seldom such a filename is really used in practice.

They are short and meaningful.

They're meaningful to you. To me, - is literally always an alias for stdin or stdout (even if that's not technically true, it's functionally true in practice). If i were to stumble across files named - or + (which i haven't, in 25-odd years of Unix usage/administration), i wouldn't know what they meant without their creator explaining them to me and i would assume that they were created by someone who was not aware that the name - can confuse some tools (grep immediately comes to mind).

In any case, i'm not the one who has to be convinced that it's worth changing - that decision is Richard's to make. Fossil has a long history of being averse to changes "just for the sake of change," and this is a case which, IMHO, falls into that category. That's just my personal opinion, though. As far as i'm concerned, the uv add - behaviour is both correct and desirable:

fossil tar trunk - | gzip -d - | xz -c - | fossil uv add - --as trunk.tar.xz

and the perceived uv rm - inconsistency is logical because reading from stdin makes no sense in that context (noting that a uv name of - is possible with both uv add ./- or uv add - --as -).

For trivia's sake, here are the relevant portions of uv add and uv rm which would need to be modified/expanded to handle - differently than they currently do (though i don't think that there's any argument being made that rm should be changed):

(Sidebar: note that uv add does not allow spaces in names (line 308), so there is already precedence for naming inconsistencies vis-à-vis fossil add.)