There is an annoying behavior when using fossil with unstable connections. The issue is obvious when cloning large repositories.

When the connection is lost, fossil detect the loss and delete all the downloaded data. It does not seems possible to RESUME the cloning. The full data has to be downloaded one more time.

It's a real problem in some countries with strong internet restrictions, the cloning can fail at any time.

Fair enough. That is something we can work on.

It'd be better if it just detected the condition and auto-resumed it. Ideally, a local partial clone should be treated no differently than one that's just been offline for a year or so and so needs a lot of updates in sync.

...regarding "auto-resume"...Are you suggesting that perhaps one simply need to call "fossil clone" a second time...

Not primarily, but that should be one possible use case.

The one that I'm hoping ends up far more common is that the clone aborts at a point where you can still "open" the repo, because you have enough pulled down that Fossil can still give you something useful. This is, implicitly, another aspect of the long wished-for shallow cloning.

Having achieved a useful checkout, on next [auto]sync, you get as many more artifacts as the sync protocol can retrieve before your terrible Internet connection shuts the sync down again. Eventually, you get everything, and syncs get small enough that the chance of interruption becomes rare enough that it isn't a serious problem any more.

If it turns out that the clone is interrupted before you get enough pulled down to open anything useful, then you should be able to get to that point one of two ways:

1. Your proposal: fossil clone with the same arguments. Fossil realizes it's a partial clone and auto-resumes.

2. A manual sync: fossil sync -R /path/to/repo.fossil. You should be able to run that with further partial syncs until fossil open is able to succeed, per above.

the long wished-for shallow cloning.

...which in turn means it works best if Fossil's cloning algorithm sends the newest artifacts first, so that the chances are highest that you can open any branch tip even on a partial clone.

Conversely, Fossil's initial clone algorithm (or resumed sync) should send the oldest artifacts last, since those are the least likely to be useful.

Another thing I long had on my wish list is Git-style cloning: clone and open the repo in a single step. (The Fossil repo DB would end up named something automatic, like .fslrepo at the top of the checkout directory.) This in turn would let the client communicate the branch it intends to open, so the remote repo can send the tip of that branch first, being the most important artifacts to the client.

Another thing I long had on my wish list is Git-style cloning

Maybe add an --open option to clone.

But also add a warning if the user already has a clone of the remote repo. Maybe like:

    You already have a clone of _____.
    Do you really want another clone or just another working copy?

(I know git now has an equivalent to open, but many/most git users will be in the habit of using clone to make more working copies.)

If it turns out that the clone is interrupted before you get enough pulled down to open anything useful, then you should be able to get to that point one of two ways:

Your proposal: fossil clone with the same arguments. Fossil realizes it's a partial clone and auto-resumes.

Hypothetically, a flag is not necessary: if clone sees that the given repo file already exists, it could assume this mode of operation, rather than complaining like it currently does. It would possibly have to start syncing and compare the remote project code and local file's project code, to ensure that it doesn't inject the sync content into an unrelated repo clone, and then error out with "local repo already exists and has a different project code."

i don't think your ideal of "having enough to open" is attainable - AFAIK the sync pulls artifacts in a random order, and i don't see how it could do otherwise without the server parsing manifests/control artifacts in timestamp order, which it only knows from the timeline, not the blob table. Making the sync depend on "supplementary" tables like the timeline sounds undesirable to me. (That said: maybe it already does use the timeline - i'm not the slightest bit acquainted with the sync code.)

This is still an issue. I had a "fossil clone URL" command fail after pulling a few gigabytes of data.

$ fossil clone https://src.fossil.netbsd.org/
Round-trips: 569   Artifacts sent: 0  received: 2473980
response truncated: got 1425046 bytes of 10010812

$ ls -la
total 14804552
drwxr-xr-x   4 user group         128 Apr  1 16:34 .
drwxr-xr-x  17 user group         544 Apr  1 16:34 ..
-rw-r--r--   1 user group  7572877312 Apr  1 18:56 src.fossil
-rw-r--r--   1 user group       58880 Apr  1 16:34 src.fossil-journal

So I tried to resume:

$ fossil clone https://src.fossil.netbsd.org/ ./src.fossil
file already exists: ./src.fossil
$ fossil clone https://src.fossil.netbsd.org/
file already exists: ./src.fossil
$ fossil sync
use --repository or -R to specify the repository database
$ fossil sync -R ./src.fossil
SQLITE_NOTICE(539): recovered 14 pages from 
/dirname/src.fossil-journal
incorrect repository schema version: current repository schema version is "" but need versions between "2011-04-25 19:50" and "2015-01-24".
run "fossil rebuild" to fix this problem

OK, I ran "fossil rebuild as suggested:

$ fossil rebuild -R src.fossil 
  100.0% complete...

Oops, it's thrown away most of the data:

$ ls -l src.fossil
-rw-r--r--   1 user group  229376 Apr  2 10:03 src.fossil

At this point I gave up trying to resume, and just deleted everything and re-started the sync.

I'd suggest, for the future, that the first thing "fossil clone" should do is record enough information for "fossil sync" to be able to resume. I don't know what that entails, but I imagine it would include at least the repository schema, the project-id and the remote URL. Also, if fossil clone fails, it could print a hint about "use fossil sync to resume".

It looks as though the entire clone operation is wrapped in a single transaction, which will probably lead to most of the transferred data being rolled back after an error. I think I would prefer a new transaction at least for every "Round-trip" reported by the status display, which corresponds to each cycle of the while(go) loop in client_sync() in xfer.c.

There is a db_{begin,end}_transaction for each cycle of the while(go) loop in client_sync(), which makes sense, but the call to client_sync() from clone_cmd() in clone.c wraps it in an outer transaction. With nested transactions, the inner transactions don't really get committed until the outer transaction is also committed.

I think I would prefer a new transaction at least for every "Round-trip" reported by the status display ... but the call to client_sync() from clone_cmd() in clone.c wraps it in an outer transaction

The problem with a new top-level transaction per round-trip is that it could leave the repository in a useless state because the artifacts are transferred in what is essentially a random order. When committing each round-trip, a single failure could easily leave the DAG in a useless mess.

Working around that would(?) require(?) extending the sync mechanism to place received objects in a holding area until the sync is complete (committing them there on each round-trip), and then moving them all out of the holding area under a top-level transaction once all round-trips are finished. Patches to that effect (or functionally equivalent) would be gleefully considered.