I don't mean to suggest that we should cut off all philosophical argument before it begins.

On balance, I think that other thread is one of the best things that's ever happened in the Fossil discussion fora, because of the enlightenment it has produced.

What I regret is that it became so hard-fought and bitter at the end, with "winning" apparently being the main goal of its participants, rather than improvement of Fossil and enlightenment of its community.

This, ultimately, is why I started to give up at the end. I don't want that to happen ever again. That form of argumentation turns a positive force into something toxic, so it constitutes a danger to the community. This is what I hope to stop with this Authors' Voice appendix.

Is this something we can all get behind?

It seems unnecessary to me. My interpretation - with an attempt at an objective reading - is that it leaves a telltale suggestion of some sort of schism, and almost (just almost) goes so far as to suggest to the reader that they might be in violation of assuming every philosophy has to be bent to match their own.

No, it says you're welcome to your own philosophy, but don't go on a crusade trying to change ours.

The thing that amazes me the most about that battle is that I thought was supporting the community's existing standards, as previously expressed in that document. I thought I was philosophically in the mainstream. Then I acted on that and got piled upon.

Either I'm way outside the mainstream, based on the number of my opponents in that thread, or I'm backed by a still-silent majority.

No, it says you're welcome to your own philosophy, but don't go on a crusade trying to change ours.

Which can (depending on the reader) imply that we expect them to crusade. It comes across as unnecessarily defensive and emotional to me, in an otherwise purely technical/factual document. That the document represents only Fossil's current solution/philosophy, out of an infinite number of possible solutions/philosophies, as opposed to proclaiming the contents as Universal Truths, is certainly(?) implicit to any reader.

Either I'm way outside the mainstream, based on the number of my opponents in that thread, or I'm backed by a still-silent majority.

FWIW, i can't say - i quit following the thread yesterday or the day before. i'm not attached enough any specific view/result to feel the need to wade knee-deep into it. i prefer that those who have stronger views on the topic work it out and announce to the rest of us how it's going to be.

It comes across as unnecessarily defensive and emotional to me, in an otherwise purely technical/factual document.

That's the thing: not everyone in that thread is treating it as purely technical. You've got people accusing me of attempting to change their workflows, hiding information, misleading people... The only reason I kept replying is that people kept coming up with bogus — to me! — arguments, so I kept having to re-defend my position. If they'd been willing to just let me have my own philosophy — regardless of how far in or out of the mainstream is was — I'd have continued on in my own bliss.

That's not a purely technical way of looking at things. It's a philosophical crusade, and I don't think the community benefits when it starts getting emotional like that.

If we don't stop it this way, then how? I think I've behaved according to SQLite's current code of conduct, which is as close to a CoC as Fossil currently has, but I think some of the comments in that thread were outside it. The CoC didn't stop that, so I thought I'd try the occasional inline reminder, like this appendix.

i quit following the thread yesterday or the day before.

That's the thing: I don't know if I'm enemy-full or friend-less in this debate. If it's either or both, I'm about at the point of pushing my latest contributions off onto a branch and letting someone else choose how much — if any! — to cherrypick onto trunk.

I'm about at the point of pushing my latest contributions off onto a branch and letting someone else choose how much — if any! — to cherrypick onto trunk.

Given that there has been no definitive consensus, nor a declaration from Richard, for any specific change, my instinct would be to do just that - leave things how they were a week ago (when, as the German phrase goes, "the world was still round").

$0.02

Do any of the promoters of --allow-fork have a technical or pragmatic reason to encourage the creation of ambiguous unnamed branches?

One pragmatic reason for keeping the --allow-fork suggestion/hint: it has been there since the beginning of time and hasn't caused any grief (that's been aired on the mailing list/forum) until the past week. Removing it feels like a knee-jerk reaction to me.

That said: i'm not emotionally invested in any sides of the debate, am not going to debate it, and won't lose any sleep over whether it's removed or retained.

The other quote that comes to mind is "You know you've achieved a good compromise when everyone's unhappy."

So, with [acb5324c7925fa68], everyone can be unhappy now.

I've also removed this new Author's Voice section from the branching doc.

Now can we let this debate die?

My experience is different. I have had many many cases of unintentional forks that have caused grief in the team I support. This has been an issue for years and I brought it up a long time ago. I believe the automatic merge feature was added based on discussion I initiated related to hassles from unnamed and unintentional branches.

As an aside, training has helped but forks are still a productivity hit and if our team revisits the fossil/git debate I know that forks will be a factor that push a lot of otherwise happy fossil users to consider git.

I've said it a dozen times in this discussion and I'll say it again. For a new user of fossil working on a busy repository forks are extremely disconcerting and disruptive. With training most users learn to deal with it but it still wastes time.

Then instead of the fork existing where the support technician can deal with it (i.e. in his own copy that he just pulled down to get the fork so he can run "fossil merge")

Maybe:

    fossil pull -R URLofUSER --allow-forks
    # resolve the fork
    fossil push -R URLofUSER

So it's not the use of --allow-fork that is causing you grief

I don't think it's clear if mattwell has one problem or two....

unintentional forks that happen as a result of synchronization architecture?

...but that isn't one of the two possibilities. We've ruled out 3+ level deep cloning. He's using a simple star architecture, a.k.a. master-and-clones.

Rather, one of the two possible problems is a race condition baked into the way Fossil currently handles checkins, which is guaranteed to happen at some low probability, the chance of which increases as the number of checkins increases. The only reason more people don't see it is that most Fossil users don't work on projects with sufficiently high commit rates from multiple parties.

The only question in my mind is whether mattwell also has a problem with users giving --allow-fork to stop Fossil from complaining. That is my "Just make the pain stop" hypothesis.

What about a client-side option that could be enabled would cause a so-configured client to refuse to push until the fork no longer exists?

The proper solution is to remove the race condition, which has been known about for ~11 years. It's coming up now because mattwell's installation has reached the usage level where it starts showing up frequently enough to be a serious annoyance.

it's kind of a known, and predictable, condition of any DVCS

Not for the case of star toplogy and autosync=on, which is the condition at mattwell's company. There, all repos able to create forks are in communication via the central repo, so the central can be configured to reject fork-creating checkins, requiring "update first" behavior.

For 3+ level cloning or autosync=off, then yes, forks are inevitable, which is why those were some of my first questions to mattwell in the other thread.

The problem discussed in that old email is that there's a conflict window between the check "Am I at the tip of this branch?" and "Let's sync!"

1. Andy: Am I at the tip? Fossil: Yes.
2. Betty: Am I at the tip? Fossil: Yes.
3. Andy: Let's sync! Fossil: Okay.
4. Betty: Let's sync! Fossil: Okay.

Now there's a fork because Betty and Andy both checked in against the same tip-of-branch, because they were told it's safe to sync.

Clearly this is because these are separate SQLite transactions, so that another one can come in between and mess things up, so that the answer changes between query and response. The solution — from my outsider's perspective, not knowing the inner details of /xfer and such — is to stretch the SQLite transaction to cover both the initial Q&A through the final sync, so that Betty is blocked on her question until Andy gets done. Then she gets told "would fork," and if she then plays ball and updates first rather than force it, no fork occurs.

I don't think it's clear if mattwell has one problem or two....

Actually three problems:

1. Occasional usage of --allow-fork by a naive user.
2. Occasional race with overlapping pull, commit, push
3. Occasional remote site disconnected causing what you all are calling the 3+ level deep problem.

I am at peace with the #3 problem. I can explain that to users and they get it.

The #1 problem can be mitigated by education but IMHO the suggestion to use --allow-fork should be replaced with a suggestion to use --branch.

The #2 problem is technically feasible to resolve and in my opinion would make fossil a better, more resilient tool.

The thing that makes #2 a bigger problem is that any smart engineer or tech type can see that it is unnecessary and that it is technically solvable. Technical folks are rather unforgiving of a misfeature that wastes their time when they know there is no good reason to not fix it. Call it the tyranny of the perfectionists.

BTW, it was asked, why not use branches? This particular repo is part code and part configuration database and requiring branches would be artificial as there is no need for genuine divergence. If I suggest using branches it is only to work around this limitation of fossil.

If I suggest using branches it is only to work around this limitation of fossil.

Where I work, we do development on branches. This has advantages for us:

• Work in progress can be versioned without affecting production code.

• Facilitates review of changes.

• Facilitates system level testing in a pre-deploy environment.

• Releasing to production is a "fast forward" merge to trunk, final test, then commit.

I think these are real benefits to any project.

("fast forward merge" is borrowed from git. It means the branch being merged to trunk is fully up to date with trunk so there is no actual merge necessary.)

When I add the locking I propose to my test script the forks 100% go away (see below). I have a handy mechanism that we can use to implement this in our wrapper script "fsl". I'll post that script once we get that done in case anyone else runs into this challenge. NOTE: In no way does this compromise the disconnected use! We have to deal with it by adding detection. We'll check the status of autosync and if it is on add the lock, else no lock.

We can also suppress the --allow-fork message in our wrapper although I had rather hoped we'd get that for free. --allow-fork is an advanced feature that a naive user will use because it sounds like it might be the right thing to do to let them get their critical commit in and surely the fossil folks wouldn't list an advanced only, problem causing option in the message?

#!/bin/bash

rm  -rf forkrepo.fossil workarea[0-9]

fossil init forkrepo.fossil
# fossil rebuild --wal -R forkrepo.fossil
fossil set gmerge 'cat "%merge" > "%output"' -R forkrepo.fossil

LOCKFILE=$PWD/forkrepo.fossil

for a in {1..5};do
    wa=workarea$a
    mkdir $wa
    (cd $wa;fossil open ../forkrepo.fossil;touch $wa.txt;fossil add $wa.txt;fossil commit -m added-$wa.txt)
done

function makeforks () {
    local area=$1
    local a
    cd $area
    echo "Emulation user in $area"
    for a in {1..9};do
	fossil update trunk
	echo Editing $area.txt in area workarea$area
	echo $a >> $area.txt
	fossil update trunk
	lockfile-create -v -r 10 $LOCKFILE
	fossil commit -m "commit from $area"
	lockfile-remove $LOCKFILE
    done
}

for a in {1..5};do
   makeforks workarea$a &
done

wait

What about a client-side option that could be enabled would cause a so-configured client to refuse to push until the fork no longer exists?

The proper solution is to remove the race condition

But that only covers the simple star configuration with autosync on.

To cover other situations, enabling clients and middle servers to refuse to push while a fork is present is a reasonable measure.

I now doubt that there is a race condition here inside Fossil, and even if it does exist, its effect would be far smaller than the one created by the necessarily-separate update and commit commands.

It would be bad for Fossil to have a command that does both in a single transaction, as I've explained in my previous reply in this thread.

its effect would be far smaller than the one created by the necessarily-separate update and commit commands.

I'm not suggesting combining update and commit.

What I am suggesting:

• A sync operation should work the same as an autosync during a commit. Except the message would be "Fork detected. Please resolve then sync again."

• A pull operation should warn the user of the risk of creating a fork, suggest using sync, instead, then prompt for confirmation.

Not perfect. But would reduce the risk of introducing forks into the main repo.

A sync operation should work the same as an autosync during a commit.

One of the objections to the current behavior is that the fork creator isn't required to fix the fork. A Fossil user is allowed to create a fork and drop the problem of fixing it on someone else. This sound like more of the same, if you mean it happens to those trying to pull a repo with a fork in the checkout's current branch.

That's why we're so focused on methods to prevent creating the fork in the first place, or isolating the fork to the creator's clone somehow. There are partial solutions, such as --allow-fork on personal working branches, which do get sync'd, but if no one else is on that branch, that fork doesn't bother anyone but its creator.

If this happens to the creator of the fork instead, then how? We've shown a bunch of ways to create a fork. It won't solve the autosync=off problem, the off-network problem, or the --allow-fork problem. As for the race condition in commit problem, which just got confirmed by a third party, this is just another race condition. Both commit and update are already preceded by autosyncs when that setting is enabled, so what new sync are you suggesting, and how does it close the conflict window?

A pull operation should warn the user of the risk of creating a fork

You'd be closing off distributed development. No one could go off-network, do some work, check it into their local clone, then come back and sync it back to the parent repo, as Fossil currently works.

To fix that, you'd have to allow Git-style rebase of an un-sync'd branch, which I think we're philosophically against here. That'd require rewriting history — the existing checkins would have to be recomputed to be rebased — not to mention requiring re-testing everything to ensure that rebasing the branch doesn't break it diffs.

That's an awfully high cost to pay to avoid forks. The cure is worse than the disease.

so what new sync are you suggesting, and how does it close the conflict window?

Not a new sync, only that fossil sync should behave like the automatic sync done when autosync is on.

The idea is, when a fork happens (for any reason), to limit its spread. Yes, ideally, try to prevent forks from occurring. Also, should have measures to contain them.

If a user is offline or has autosync off, then when the user gets back online and tries to sync, Fossil will check for possible forks before it does the push part of the sync. If a fork is detected, then warn and suggest the user resolve the fork then try again to sync.

Then the fork doesn't escape the repo of the user that caused it.

If user is sync'ing to a middle server and the middle server is unable to contact the central server, this will prevent the fork from getting past the middle server. Of course, the operator of the middle will either notify the originating user of the problem or deal with it. But still, the fork should not get into the central server.

So, no, it's not "more of the same". It's keeping the fork close to the user who caused it.

You'd be closing off distributed development.

You left off:

suggest using sync, instead, then prompt for confirmation.

Like the "would fork" warning on commit, this suggests that using sync would be a better option. (see above)

Maybe the prompt could be:

    May cause a fork, try using sync instead.
    Sync, Push anyway, Cancel ? (s)

This would encourage sync, but still allow the option to push.

so what new sync are you suggesting, and how does it close the conflict window?

Not a new sync, only that fossil sync should behave like the automatic sync done when autosync is on.

It does, sync is sync.

Sync is about artifacts (see Fossil Concepts), not their contents, and is a two-way exchange of artifacts, push and pull happen together. A fork is a relationship between 3 artifacts, not something contained in an artifact, so forks can not be detected until the exchange is complete.

Anyway, a fork that already exists on either side will be synced to the other side (both ways), and new forks may appear on either or both sides after the sync. There is no way to identify the last kind ahead of the sync. You might want to glance at The Fossil Sync Protocol - who is going to volunteer to design and write an equally efficient and practical replacement with the ability to predict new forks? I'm not saying that nobody ever will, but ...

To cover other situations, enabling clients and middle servers to> refuse to push while a fork is present is a reasonable measure.

I wonder what would happen to Fossil's own servers if some kind of refusal to accept a push were introduced. Someone would commit against node A using www.fossil-scm.org, and another might commit against node A using www3.fossil-scm.org, and another might commit against node A using www2.fossil-scm.org. Because all commits against the same parent represent a fork, would the automatic synchronization that runs every X minutes between all of these sites break because someone introduced a fork?

Yes it would, if a refusal was introduced. Those repos would remain unsynchronized until somebody did something about the fork in one of them.

Would that then mean that there is now a worse mechanism for schism than was ever possible with a simple fork in a branch? Those who commit to www.fossil-scm.org can never more see the content that is committed against www3.fossil-scm.org and vice versa?

Yes, that's how it would be!

However, looking at any one of the repositories, there is no way to tell, in the situation you describe above, that the (post-sync) fork exists. This also means that there is no way to decide that the refusal is required!!

Those who commit to www.fossil-scm.org can never more see the content that is committed against www3.fossil-scm.org and vice versa?

Having a cluster as the central repository would be a special case.

The operator of the cluster, DRH in this case, would have to decide how the cluster handles this.

If the member servers of the cluster are to be treated as co-equal, then the fork would have to be shared. Otherwise, one member becomes superior to the others.

In our setup, we have 1 central server and a back up server. The back up server only accepts pushes from the central server. Clients can pull from either, but only push to the central server. If we had to make the back up server the central server, we could.

Count me as part of the silent lurkers. This is small enough to not really bother me either way even enough to take a stand. I have come to understand that people have way stronger opinions than mine.

In any case, Warren, thanks for all your contributions (in the code and on the forum).

As a reader and user, this seems like a gratuitous addition to an already-long document and adds no useful information. I doubt it will actually do anything to mitigate future complaints or “philosophical arguments”.

I just want to second what other's have said before:

this "author's voice" text feels inappropriate in this technical document. the more so if the reader has not followed the threat and just stumbles over this document I'd say... so I definitely would remove it completely.

and there is definitely too much talk of crusades, battles and enemies in this thread as well. this seems misguided to me, really.

so to reiterate: I second the attitude of several others here that the documentation should describe the technicalities as objectively and neutral as possible and to stay clear of strong recommendations which are not representing a clear consensus among all knowledgable users who have voiced their opinion in the forum.

personally, I anyway find the strict distinction between fork and branch somewhat artificial. I always have viewed manifest forks as "unnamed branches" created by fossil if necessary. also, the `update or use --allow-fork' message never was a problem for so many years. no idea why this should suddenly be the case now.

accidental forks have caused some confusion with co-workers here, too, who ended up continuing to commit on their fork without being aware of this. so the only thing which might be really helpful is not so much fiddling with phrasing of error messages or documentation but to make it more obvious to the user if he currently has more than one leave open on any branch and, especially, if he resides on the fork with CURRENT. so how about integrating `fossil leaves -m' or similar into status/info and/or timeline reports together with a "usually you should merge the fork ASAP" recommendation?

make it more obvious to the user if he currently has more than one leave open on any branch

That's why Fossil currently — and in fact since 2011-05-20 — complains when you're working on a forked branch in fossil commit:

  **** warning: a fork has occurred *****

The logic behind that message is that Fossil assumes this is a problem you want to fix before continuing with the commit.

Which is yet more reason I thought I was in the mainstream on this issue, philosophically: Fossil behaved and was documented to treat forks as accidental and generally unwanted before this whole debate began.

I wonder if what's really going on here is that I've just managed to draw out a fringe minority who happen to hold an incompatible philosophy, and that most people either agree with me or don't care either way.

usually you should merge the fork ASAP" recommendation?

Being more strident about this would apparently anger some number of Fossil users.

I was talking about this situation where you just end up with the fork in place (e.g. due to offline work of two people). afterwards I believe fossil is silent about the occurred fork, no? if one get's the warning you state every time afterwards at every checkin, yes, that would be enough of a "heads up!".

regarding the proposed more persistent notifcation/recommendation: not sure whether anybody would be angered by that. but anyway.

As I've seen, if a fork has occurred, fossil will warn you on an update that there are multiple leafs, prompting you to choose (and merge if you feel like it).

I feel very well addressed by "toxic", "dangerous", and "violating the code of conduct". So please allow me a few short statements:

• I never meant to attack or hurt anybody. If I did, I sincerely apologize.

• I tried to show not everything was deadly serious by being "funny" and using smileys -- but I paid too little attention that this could be interpreted as ridiculing others. If I did, I sincerely apologize.

• Some reproach may go back to others, for heating the debate by reversing every word twice, and searching for tiny contradictions in every argument. Also, certain (unnecessary) statements had an undertone of mocking, as well (for example, the Windows bashing -- but I'm getting used to it).

• I never tried to "indoctrinate" others who prefer branching over forking to change their views or work-flows, and I see no problems with the co-existence of either approach. Does the same also apply vice versa? Does the "code of conduct" say the opinions of others should be tolerated?

• I didn't mean to criticize the excellent (!) Fossil documentation in any way, and it's okay to give "recommendations", in addition purely technical information. By "obfuscation", I was referring solely to the way from the (technical) error message to a (technical, brief and neutral) list of options.

Ok, so now while(1)→break.

Do any of the promoters of --allow-fork have a technical or pragmatic reason to encourage the creation of ambiguous unnamed branches?

An example to represent alternative work-flows (not even requested, nonetheless much appreciated):

• https://www.fossil-scm.org/index.html/doc/trunk/www/branching.wiki#forking

I'm going to answer most of your message via private email, since it's getting personal, but I can take one part of it up here:

Do any of the promoters of --allow-fork have a technical or pragmatic reason to encourage the creation of ambiguous unnamed branches?

An example to represent alternative work-flows... [in the Justifications for Forks section of the branching doc]

Since I don’t believe --allow-fork is a good idea in general, I’m not the best person to come up with such an example. The best one I can think of at the moment is:

$ fossil up my-working-branch
…hack, hack, hack…
$ fossil ci -m 'my broken partial feature'
$ fossil up prev
…hack on new idea that doesn’t depend on the broken partial feature…
$ fossil ci --allow-fork -m 'new idea'
$ fossil up my-working-branch
…continue hacking on the broken feature…
$ fossil ci -m 'fixed the broken feature'
$ fossil merge       # heal fork by merging the new-idea checkin

Is that the sort of thing you were hoping to see? If not, I'll need you to come up with the example, because "alternative work-flows" is too vague for me to go on.

If you do see that as a good use of --allow-fork, then I don't see why you wouldn't just use the stash instead. And on those grounds, I'd need justification of the workflow before I could in good conscience add it to the doc.

My use-case for forking is a small team of scientists tasked with implementing a piece of software. One of my colleagues provides invaluable insight into the physical problem, but has limited programming experience and has never used a VCS of any kind before.

In case fossil update is not able to merge upstream and local changes, I would rather prefer that person to create a fork but have it pushed to the repo and merge it myself, than require merging on the colleague's part.

I'll readily admit that we are doing a lot of things wrong, but the alternative is not doing it at all.