Fossil embeds SQLite, yet it does not appear that Fossil has any features that ease the use of multiple repos. I've never really liked the way Git does it, and I wonder if there is any effort/interest in improving Fossil in that regard. Does anybody do something more elegant than "copy the amalgamation"?

My development cycle is generally this: projects start as snippets of code that test/prove out my basic idea. When it's clear it's not just a throw-away chunk, I version it in a "catch all" repo that holds a lot of little growing projects (when free time allows :-). They live in there until they grow to become proper projects, worthy of being broken out into their own repo. There, they usually continue to grow until they become large enough that it makes sense to spin-off separate repos (e.g., an app gets refactored to be app + web api + core lib). In the process, new projects are created that are dependent on some existing repo (could be linked as a lib, or could be embedded, or whatever).

Abstractly, I suppose, it has to do with managing the history of individual files/artifacts across databases. I haven't dug into the guts of Fossil, so I'm not sure how heavy of a lift that would be. Am I barking up the wrong tree? Is my development process so unusual that no SCM has it as a use case? Thanks.

Snippets that are not yet proven to be better than throw-away junk might belong in a PIM/IDE/outliner that can be used to organize them and keep notes about the development work. Leo is such a tool. Version control is not built in to it, but you can maintain multiple versions of a given snippet, either by giving each version is unique file name, or by putting each one in a folder of its own. Proper use of the outline can keep things organized.

Leo's Home Page []: https://leo-editor.github.io/leo-editor/

When it's clear it's not just a throw-away chunk, I version it in a "catch all" repo that holds a lot of little growing projects (when free time allows :-). They live in there until they grow to become proper projects, worthy of being broken out into their own repo.

FWIW: fossil makes setting up new repos, including remote access, almost entirely painles (and 100% painless if no remote access is involved0. That is, i opine, less pain than trying to migrate a given bit of code through three separate lifecycles as its evolves. i.e. putting it in its own repo from the start will involve less long-term effort.

It's one lifecycle as I see it, where code grows and gets restructured and shared. It happens between Fossil and SQLite, as I noted. While managing many open repos certainly seems to be made easy by Fossil, that still doesn't mean I want to create a hundred different databases each with the starting one-liner shell script that may or may not eventually be expanded on.

My hope was/is that Fossil would support some clever use of the database that made it easy to ATTACH (or whatever) to other repos so that they could be merged/split with ease. It's not a deal breaker or anything, and I certainly haven't found it in other tools, either (at least not in anything that doesn't target massive enterprise-level "cradle to grave" auditing). The whole "Mono Repo vs. Multi Repo" debate could stop if there was just a tool that erased the distinction.

My hope was/is that Fossil would support some clever use of the database that made it easy to ATTACH (or whatever) to other repos so that they could be merged/split with ease.

Combining repos "can" be done with the deconstruct and reconstruct commands, combining the results of several deconstruct calls into one reconstruct call, but it requires some manual cleanup to avoid things like duplicate trunk branches or overwritten wiki pages when two repos share the same wiki page names (done that!).

Splitting repos is "not possible" (for a given value of "not possible") because Fossil's architecture is designed to provide a 100% auditable history which does not/"cannot" contain holes (barring the obligatory exception of "shunning"). Yes, it can be done, but it's a hassle and requires a fair amount of scripting and/or manual effort.

Conceptually, I don't see a split repo as having "holes" so long as there is some method to reference another repo where the artifact can be found (by whatever mechanism). A larger Fossil-verse, if you will, that maybe expands on the idea of shun lists to represent something intentionally externalized. Or maybe the idea would be that the trunk of any given repo could be mapped to a branch of some larger repo that subsumes it, Barnsley fern style. I don't know the guts of Fossil well enough to know the best mechanism to base it on, and I appreciate it may be much harder to do than I'm thinking it is, which is why I don't already see any SCM that makes it simple.

Thank you for the detailed how-to! Definitely far more daunting/clunky than anyone would want to do on a regular basis, and I have to assume that if there were any cleaner solutions at hand that DRH would have already hit upon them in the co-development of Fossil and SQLite. I guess my hunt for a fractal SCM continues. :-)

FWIW,

A script to create fossil repos is fairly easy though I must admit I've solved the problem a few times and each time its taken a day to do properly.

If anyone is interested drop me a line at philip.maker@gmail.com and we can discuss it. The main challenge is the diversity of fossil web engines and setups which is fossils strength.

The current solution is to create a repo you specify a new name and zero or more templates (e.g. manual, code...):

% a-make new a-green-tree-frog template-animal

Which is a TCL script that builds a repo a-green-tree-frog using the template-animal as a basis for contents. This uses either a ssh login or a fossil /ext extension to do the work. Either way the real problem comes when you need to populate a working directory with all the repos you need.

So at the risk of causing version hell each repo has a USES file containing repo tag/version pairs so that if you run the following in a repo:

% a-make all  

It clones/updates/... all the ../repos according to USES in each repo so you have all the repos you need with the correct settings, e.g. remote-url

Then it does a build/install cycle breadth first across the repos USES and do installs into ../bin, ../include etc.

% a-make clone ;# clones every repo on the remote server
% a-make update ;# same as fossil all update for ../* (in case you have other collections).

Am I barking up the wrong tree? Is my development process so unusual that no SCM has it as a use case?

Possibly. I don't precisely understand how your work, though.

Remember: I wrote Fossil for one reason → to support the development of SQLite. As it happens, Fossil has proven useful on countless other projects as well. But the key point here is that I never intended for Fossil to solve every SCM problem. Nor should it.

If you can get Fossil to work for your scenario, that is great. Perhaps write up a detailed report about how you got it working, in case that will help others. But please do not expect Fossil to evolve to meet your specific needs, unless you yourself do that evolution.

... They live in there until they grow to become proper projects, worthy of being broken out into their own repo.

I may be speaking from a different experience, but when a project transitions through phases from a sketch to some "worthy" state, a lot of past changes may be seen as purely tactical which, indeed, belong to their own historical context. Thus it's just reasonable to let the earlier stages live in whatever repos they have been spawned. This minimizes the noise when analyzing the changes for the "worthy" state.

To maintain some historical continuity, I would just include a reference to the repo (or a specific path within) of the "past" state. This could be done via commit message (for the initial commit in the "worthy" repo), or via Tech-note (it also appears on the Timeline).

By design and principle Fossil does not allow physical changes to the history, so there's little gain in splitting-off -- it's just as good as duplication. That's why just including some reference to the past repo/repos serves the purpose without much overhead.

When an idea has been first materialized on a napkin (or back of an envelope), it makes sense to keep it as-is (if for some sentimental reason). A published work usually has some reference, so others may refer to it.

Thus it's just reasonable to let the earlier stages live in whatever repos they have been spawned.

That is indeed my current git process (init a new repo from some "final" checkout branch of the old). But that creates a situation where the entire history is essentially "shunned", not to mention all the other features Fossil provides for a project also dropping into a black hole. It would be awfully handy to have that "include a reference to the repo" functionality managed by the SCM itself. If I ever dig far enough into the guts of Fossil I may implement it myself, unless I discover along the way that it is indeed a fool's errand. :-)

That being said, I'm not sure about the Tickets and Wiki but you could experiment and report back. Maybe this is the "other features" that you mention above.

Yes; being new to Fossil I'm still playing around with what features it has and how they can best support the ways I'm doing things (e.g., I started converting my blog into a Fossil site and I experimented with Wiki vs. Tech Note vs. Forum vs. whatever as the replacement mechanism for posts). I'm still not sure I'm picking the right things, so I'm trying to think ahead on any and all operations I may have to leverage to clean up any repos I convert from Git. I like Fossil's "what you actually did" approach, but at the same time I don't usually want to bother others with my early confusion while learning.

You specifically mention a branch, you could use the "fossil bundle" command to export just a single branch and make that the beginning of your new project.

Thanks for that pointer to bundle. When I finally get a project into a state where it makes sense to split or join repos, I'll dig into all the commands that might do what I want. I was just hoping that there was some feature that either existed or was planned that made it dead-simple, based on the connection between SQLite and Fossil.

Why do you use version controll for throw away code?