I'm replying to this post to avoid threadjacking: I don't see that any of these questions have a thing to do with conference talk proposals. But maybe I'm wrong, and some of them will become a talk, in which case feel free to take the issue back up on the other thread.

a command to set contact info

You correctly give a command like this:

  $ fossil sql "update user set info='Bugs Bunny <bugs@example.com>' where login='bugs'"

…so what is it you want, exactly? A new command? An extension to the fossil user command? With what syntax?

Users can never be deleted

I think you first have to get past the idea that the user table syncs at all, except in the limited case where you do the clone/sync as a Setup user on the other side of the connection. Otherwise, your local user table contains only a new user record for the user you cloned as, plus the stock user categories. Even your own user record isn't copied, AFAIK. And if you clone anonymously, you don't even get that.

The point being, deleting a user on your local clone does zilch to affect the users on the remote, even if you're a Setup user. If you want to delete a row in the user table, you really need to go around to all of the clones and delete it individually from all of them, so in that sense, yes, you cannot reasonably delete users in any situation where you aren't in control of all the clones.

Furthermore, dropping rows from the user table doesn't affect the user names baked into commit manifest artifacts. Rebuilding the repo from those artifacts doesn't populate the user table, but those users still exist in that form. Short of shunning those artifacts everywhere — which is a similar problem to the above — you can't remove all trace of those users everywhere.

...Which brings us back to the GDPR discussion, doesn't it? :)

Anyway, these statements contradict each other.

Where do the docs say you can remove all traces of a given user everywhere, on all clones?

What is a 1st and 2nd tier command?

That's documented here.

[blame and praise] appear to be functionally equivalent

Yes. They're pure aliases.

is there way for me to be kinder and offer praise by default?

When you're using the command, aren't you usually trying to answer the question, "Who changed this line?" and if so, then is the reason more often because you're looking to give them praise, or is it to figure out who you have to talk to in order to answer that "why" question?

In my projects, the praise goes in the ChangeLog and AUTHORS files. These files' contents are manually extracted from the timeline commit messages as part of the release process, not by parsing "fossil praise" output. That would amount to an O(n²) search for new material on each release, since there is no "--after VERSION" option on this command. (Though that's not a bad idea!)

Blame is something we handle among the developers, with the goal to fixing a problem and getting past it.

[command line version of] the web page /artifact_stats

You can get a lot of that from the dbstat command. That's closer to the output of /stat, but extending it to compute the other stats should be mostly a matter of grunt work rather than human brilliance. Go ahead. :)

There are several reasons why the web interface is not necessarily easy or even possible to access.

Sure about that? :)

  $ echo "GET /artifact_stats HTTP/1.0" | fossil test-http

Rendering to other forms is an exercise left to the reader.

But fossil init --template doesn't copy users+passwords and their permissions

  $ fossil conf export user users.cfg -R from-repo.fossil
  $ fossil conf import      users.cfg -R   to-repo.fossil

Where is [the undo buffer]... in fslckout? Same for the stash?

Yes, and yes.

Technically speaking, it's the localdb.undo table, which exists only if you have done an undo-able thing on the repo and haven't done something that clears the undo buffer. See src/db.c for more on localdb, which is the DB schema name for .fslckout/_FOSSIL_.

I'm not sure where you'd expect to learn this short of reading the code, since it's very much a low-level internal detail. It shows up only indirectly, as in the check defeated by the fossil clean -f flag.

Removing [brackets] from fossil forum link shorthand?

I suppose it's considered a feature. I don't much like it, either, but it's easy enough to defeat by giving it a URL rather than a hash:

  [has some strange problems](/info/1772cb4a37) 

user-level login groups, not just repo-level. This would be great for when multiple repos share admins

I think you can achieve that by pointing these other repos at an admin-only repo as the source of truth for the login group.

If not, then I think this is deserving of a whole different thread. My sense is that few use login groups, so I would not expect a lot of engagement on the topic.

fossil help -a --single-column

Are you asking for paragraphs without line wrapping? Short of modifying the "help" command to add a "no wrapping" flag, something like this might work:

  $ fossil help --html clean | lynx -stdin -dump -width 9999

no command line equivalent to https://fossil-scm.org/home/test-all-help

What's wrong with this:

  $ fossil test-all-help | grep clean

Fossil has no "reset password by mail" option.

Yup. Your contribution is eagerly awaited. :)

Warren Young (wyoung) on 2020-10-31 02:06:03:

I'm replying to this post to avoid threadjacking

oops, I missed this until just now, and have done some more threadjacking over there in the meantime. Some of your replies are documentation material, noted.

You correctly give a command like this:

$ fossil sql "update user set info='Bugs Bunny bugs@example.com' where login='bugs'"

…so what is it you want, exactly? A new command? An extension to the fossil user command? With what syntax?

Do not underestimate how impressively complete the fossil command set is to a new user like me, meaning it is a surprise when something is missing, and my first instinct is to look for my oversight. This was in the context of me doodling a script to provision a set of Fossil instances. In this case it seems to be a minor omission in the "fossil init --admin-user" command, where it should also have a parameter such as --contact to specify the email address. The general picture is that if I'm provisioning a collection of Fossil repos, I can expect to use a good few "fossil sql" commands. Stet.

Furthermore, dropping rows from the user table doesn't affect the user names baked into commit manifest artifacts. Rebuilding the repo from those artifacts doesn't populate the user table, but those users still exist in that form.

this is really helpful. I'll see if I can find a way to reflect that in the docs in a precise way. Starting from this point of view makes the other bit (local users are in a local bubble, with mappings to remote users on demand) all quite simple. I suspect the word "user" or "username" is being overloaded here this giving rise to a bit of confusion but I'll have to think about it.

What is a 1st and 2nd tier command?

That's documented here.

The word tier doesn't appear on this page, nor in the HTML source.

However I tried again and Google (but not the Fossil search :-( took me to the file mkindex.c where drh has made it very clear in the comments at the top what the different tiers are. I'll find a way to include that in the page that mentions tiers extensively with no clue as to what they are. This does illustrate my point made elsewhere about the search; I suspect from my own explorations that if FTS was enabled for fossil-scm.org that it would have given me the same result. And of course grep would have told me the answer in even less time - but I'm trying to use Fossil, not study it.

There are several reasons why the web interface is not necessarily easy or even possible to access.

Sure about that? :)

echo "GET /artifact_stats HTTP/1.0"

fossil test-http

I love how Fossil just gives you a reason to say "Of course it can do that" at regular intervals. Thanks. One for the FAQ or some other doc. I'll try to think of a way of putting that.

However by "several reasons" I was meaning use cases that are close to me. That of physical assistance needs in particular, and also the automation use case. Your answer addresses both. You wouldn't believe (actually you would) how much grief I have had with the official CLI for github over this sort of thing, some of it in public. And here it is, with very little fuss. /me needs to add some github cli stuff to the git pages.

Where is [the undo buffer]... in fslckout? Same for the stash?

it's the localdb.undo table, which exists only if you have done an undo-able thing on the repo and haven't done something that clears the undo buffer. See src/db.c for more on localdb, which is the DB schema name for .fslckout/FOSSIL.

I'm not sure where you'd expect to learn this short of reading the code, since it's very much a low-level internal detail. It shows up only indirectly, as in the check defeated by the fossil clean -f flag.

I disagree, it's highly visible to system administrators, because otherwise why would it tell us in "fossil help undo" that there is a file size limit of 10MiB... I immediately wondered what happens when lots of users have lots of undo on one home filesystem. And conversely, how/where to change that limit and accommodate it at the filesystem level for people using much bigger files and who keep making typing mistakes so they need big undo. But you answered part of the question. I don't have lots of users or giant files, but this sort of thing has bitten me in many other scenarios.

fossil help -a --single-column

Are you asking for paragraphs without line wrapping? Short of modifying the "help" command to add a "no wrapping" flag...

I looked at what it would take to add almost that (I am not sure that --single-column and --no-wrap is the same, it doesn't look like it in the code to me.) Thankyou for (a) the kinda workaround and (b) the evidence that in fact a code change is the only way. I may have a reason to do that one.

Dan

[...] --contact to specify the email address.

The latest check-in provides this functionality as a subcommand¹ to
user, which may be of use in your script.

¹ fossil user contact USERNAME ?CONTACT-INFO?

Furthermore, dropping rows from the user table doesn't affect the user names baked into commit manifest artifacts. Rebuilding the repo from those artifacts doesn't populate the user table, but those users still exist in that form. Short of shunning those artifacts everywhere — which is a similar problem to the above — you can't remove all trace of those users everywhere.

...Which brings us back to the GDPR discussion, doesn't it? :)

Ha :-) not really. Only to say again that the GDPR does not have any such thing as a "Right to be Forgotten" despite that being a common error (and the GDPR isn't the crazy "cookie law" either.) The erasure requirements in GDPR Article 17 would not require a Fossil instance to erase Fossil manifest artifacts.

But what you say really helps on the privacy front. "Dropping rows from the user table doesn't affect the user names baked into commit manifest artifacts." It's the user table that the GDPR is concerned about and quite rightly too. I kind-of mystically assumed that there would be a referential integrity issue if you deleted a user row. As I discovered, Fossil already has some facilities in place to address the user table, and more could easily be added.

Dan Shearer

Only to say again that the GDPR does not have any such thing as a "Right to be Forgotten" despite that being a common error

Maybe this is splitting hairs of terminology, but the net seems to disagree with you on that:

https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/individual-rights/right-to-erasure/

The GDPR introduces a right for individuals to have personal data erased. The right to erasure is also known as 'the right to be forgotten'.

https://gdpr-info.eu/art-17-gdpr/

Art. 17 GDPR Right to erasure ('right to be forgotten')

https://gdpr-info.eu/issues/right-to-be-forgotten/

etc. etc.

Whether or not there is a "right to be forgotten," Article 17, in multiple authoritative sources, is called/nicknamed the "right to be forgotten," noting that Wikipedia says, "A right to be forgotten was replaced by a more limited right of erasure in Article 17..."

(That sounds like a splitting of hairs in the terminology to me.)

It's the user table that the GDPR is concerned about and quite rightly too.

Noting that fossil user names may be email addresses. It's not commonly the case but it can/does happen. In such cases, shunning in the only way to eliminate them from the history.

That said: i'm no GDPR expert, nor do i have aspirations of becoming one. This topic is better left to those who care about it deeply, so i'll keep my nose out of it.

It's also more than just about user names and email addresses. As I understand the way the GDPR's been applied, I could take a photo of a person in public in Europe, check that into a Fossil repository, and they might then have standing to demand that I remove that photo from my repository, or at least blur their face out or similar.

If that photo is at the leaf of the DAG, fine, shunning will solve it. But what if that artifact has become the parent of another by the time our hypothetical European acts? How do I follow the GDPR in that case without damaging the repo's structural integrity?

You can make similar arguments about text artifacts. What if I have a Markdown file that mentions some court case in progress, which is later settled, and the winning party demands that I remove mention of them in all historical versions of the document?

Warren Young (wyoung) on 2020-11-04 11:55:38:

It's also more than just about user names and email addresses.

Yes, anything which tends to identify... a photograph, an email address and an IMSI all tend to identify. This is all personal data and it doesn't make much difference what kind of data it is.

As I understand the way the GDPR's been applied

The point of the GDPR is to change practice in protecting human rights of private individuals, without unduly impacting on society. Here is the GDPR enforcement tracker, just 435 entries across 400 million people. The GDPR is about following guidance, often the kind of guidance that could have been written by the EFF years ago.

I could take a photo of a person in public in Europe, check that into a Fossil repository, and they might then have standing to demand that I remove that photo from my repository, or at least blur their face out or similar.

Yes they might have standing, but it may well not be not on GDPR grounds, and there are many reasons why consent would not be needed required. Public photography is legal and protected, while harassment is not. In-between these extremes are the things that one human should reasonably do to another, and if you are being reasonable then you aren't going to trigger the GDPR even if someone attempts to say you have done do (frivolous GDPR lawsuits don't seem to happen as far as I know.)

I feel you may be conflating a few different things:

1. That the GDPR only applies to organisations. If "your repository" as you describe it is your personal repo that you maybe maintain with friends or software enthusiasts, then no GDPR.

2. That laws concerning what you may do with photos you take of people in public are way older than the GDPR. Mostly it's fine in the everyday sense, including if you are a professional photographer. There are plenty of standard lawyers who specialise in this sort of thing without them needing to have specialist computer knowledge. That's not a question relating to Fossil software design.

3. That the GDPR gives an organisation multiple good reasons to have a photo in their public Fossil repo forever without consent. It isn't necessarily a lawyer you want to speak to at least in the first or second instance, this is about following the guidance with computing-savvy privacy professionals who may sometimes consult a lawyer. This does relate to Fossil software design, but not affecting the DAG in any way I have yet found.

4. That there are laws that have been around for decades that could also affect the DAG, and the GDPR is considerably less risky than these as far as I can see. Laws relating to copyright, patents and trademarks for example, or publishing classified information in $country, and in some countries, libel laws.

Dan Shearer

GDPR only applies to organisations

Do we get to sweep this issue under the rug just because there are private individuals using Fossil? Surely there are enough organizations using Fossil that their needs are worth considering? Ones like…oh, HWACI, the US corporation behind SQLite?

That laws concerning what you may do with photos you take of people in public are way older than the GDPR.

Yes, I am indeed conflating GDPR with things like "Blurmany," but I don't see how that technical legal distinction has any bearing on the question of shooting holes in the repo structure. How does it matter precisely which law causes the hole?

libel laws

I'm not talking about cases like that. My examples were taking a photo of a person standing on a public sidewalk, and republishing public court records, both quite legal in the US, where I am. If I have a Fossil repo making use of those freedoms, my options currently suck:

1. Do not do business in Europe, since there is no such thing (so far) as a US-only Internet.

2. Put up a geofence blocking Europeans from my repo.

3. Take on the risk of having Europeans shoot holes in my repo on "privacy" grounds.

Hopefully we have talked around GDPR enough to agree that contrary to non-factual claims elsewhere, the GDPR does not represent noticeable additional risk to public Merkle trees. The GDPR is useful to Fossil in terms of improving the existing Fossil privacy facilities, but that is nothing to do with imagined risk to the DAG. Full stop new paragraph.

New topic: you now raise something different to which there are also fairly definitive answers.

Warren Young (wyoung) on 2020-11-04 19:39:14:

How does it matter precisely which law causes the hole?

I'll turn this into two questions and answer them separately:

1. How can Fossil's design assist users targetted by takedown requests?

Answers:

• Fossil already does via shunning, although currently it doesn't quite mention legal action as a reason. I have added one more good reason for deleting content, and distinguished between DAG content and the user table thanks to your helpful comments. And that is the first point almost entirely addressed.
• Fossil provides via the DAG a Chain of Custody far superior to that offered by most other software that might host problematic content (such as a blog or mediawiki.) Therefore a Fossil user can credibly define any alleged problem to be the minimum scope possible, and reject a blanket takedown or maybe even prove the takedown is wrong. This is an argument for the store-forever Merkle tree as a defence, not a vulnerability.

2. Does the Fossil Store-Forever Merkle Tree Increase Risk to Users?

Answer:

• Yes, if the Fossil users behave in such a way that they are going to frequently trigger what are normally very rare takedown events. Such people probably shouldn't use Fossil.
• Otherwise no.

There are legal experts in other forums who love discussing the sources of potential sources of takedowns and that is where specific legal discussion belongs. The US has the DMCA, a patent troll industry and mandatory government registration for open source encryption software and no doubt other horrors. When it comes to libel, the laws of the state of Victoria or the country of England are open to abuse for takedown purposes and are exploited by wealthy injured parties. China has just enacted a new law covering software export controls, and so I'm told there are dozens more. I can't see that any of these impact Fossil design one way or another. Do you?

Dan Shearer

Dan Shearer (danshearer) on 2020-11-04 11:57:43:

Summary: The GDPR definitely does apply to anyone hosting a Fossil repo, and Fossil today provides some tools to be GDPR compliant. Which is great news.

Oops. I meant any organisation hosting a Fossil repo. The GDPR only applies to organisations.

Dan Shearer