Hi everyone,

I'm giving fossil a spin to try something different. Compared to git, fossil feels straightforward and easy to use. Doesn't get in my way and does what I need without over-complicating things. Really refreshing.

My background is mainly in Golang, and platforms like github & bitbucket have made it fantastic to provide back to the Go community. Code reviews via Pull Requests, integrations with CI/CD tools for running tests & linters before code is merged in, plus the additional features beyond what git provides (issues, labels, wiki, releases, etc).

I understand git != github, and making comparisons to between GH & fossil is apples-to-oranges. At the same time, I cannot but help see the various similarities between the two, and wonder how I could migrate the workflows which surround my github projects to be based on fossil instead.

I've seen some forum threads here asking for pull requests & similar functionality. Stuff like creating bundles and providing a mechanism to accept and merge them. And of course one could always export a fossil repo to git, leveraging all the existing tooling and infrastructure; although at that point I'm not convinced just sticking with Git wouldn't be easier if incorporating such tooling was that critical for accepting contributions.

Maybe my ideal workflow is not very compatible with fossil; but if so, that'd be a shame as otherwise I've been enjoying my time with the tool and the features it does provide.

What kind of tooling currently exists that might help migrate off of Github to relying on fossil? I've seen chiselapp, but that seeems to be focused primarily on hosting, not on supplying additional features beyond what fossil already has. If I wanted to provide a platform for hosting code reviews and running tests for external contributions, is there a solution in the wild?

Hi!

How did you discover Fossil?

To be honest, I do not remember. I came across it years ago (maybe 2016?) but never came back to it till now.

I am somewhat flirting with the idea of trying to de-google a bit more of my life from a privacy stance; and as such I've been revisiting all the tools I use daily.

While doing that, I eventually reached Github on my list I started to ponder how I could move away from it if I wanted to, and that led me back to fossil.

https://blog.avalos.me/2020/07/06/fossil-scm-and-concourse/

https://blog.avalos.me/2020/07/09/fossil-scm-and-concourse-v2/

Thanks! I will take a look at both. I'm interested to see what everyone has done.

As far as hooks go, how does that work running server-side? I'd assume they'd run when syncing with an upstream/remote repo? Would fossil commit or fossil push fail locally if the server's hooks failed?

I guess I will try it out to finding myself, but asking might be faster lol.

As an aside, I'm looking at the documentation for hooks you linked to. There is no mention on what exactly --type TYPE is on the page, although I did find it elsewhere. Is this info somewhere in fossil help , or purely as a document?

making comparisons to between GH & fossil is apples-to-oranges.

While we don't do absolutely everything GitHub does, or even aspire to, it's less an apples-to-oranges comparison than Fossil to Git.

I think "GitHub in a box" is a fair description of Fossil, within the limitations of a tagline's brevity.

…provide back to the Go community…Pull Requests…

This is where it's helpful to understand cathedral vs bazaar development styles.

Large, highly-distributed projects with many disparate contributors and a viral license allowing easy acceptance of drive-by contributions may indeed benefit from pseudonymous PRs.

As soon as you start dropping qualifiers out from that statement, PRs become a problem:

1. Small: The project is probably one of the far more common sort where the primary developer — or maybe a single-digit cadre of developers — is responsible for well over 90% of the code. PRs in that sort of world are rare occurrences. Good PRs are even rarer.

   Example: I've got a Fossil-based project mirrored on GitHub that's quite popular — it's in basically every Linux distro you can name — yet it's only received a few low-quality PRs since I mirrored it to GitHub about 18 months ago, shortly after the fossil git export feature was created. That isn't GitHub's fault, though: the rate of patch contribution wasn't much different through the bulk of the project's ~22 year development history where you sent patch files to the mailing list.

   (Which, incidentally, shows the lie of the oft-heard claim, "If you only allowed GH PRs, I'd contribute more!" It took over a year before my project got its first PR, despite being ~20 years old and widely distributed at the time, so you can't claim it's because no one's using it.)

   If PRs are rare and often of low-quality, what are they really buying over patches or bundles?

2. Centralized: If most of your developers come from a single organization, why do you need PRs?

3. Cohesive: If your developers are from multiple organizations, but they have a set of shared goals, why not give them commit bits on the central repo?

4. Restricted: Your license may not reasonably allow pseudonymous PRs anyway. If you require a signed CLA before you can accept a PR, why not just give the signer a commit bit?

5. Controlled: If you're running PRs through a serious code review process, is there much point in worrying about how easy it is to get code into the front end of that process? That seems like optimizing the first 10% of the problem: perfect optimization can only result in 10% savings, whereas a mere 11% optimization on the remaining 90% gives equivalent results from a systems perspective.

one could always export a fossil repo to git, leveraging all the existing tooling and infrastructure

For CI, that's easy. Just create the necessary .github/workflows/ci.yml file, then mirror it to GitHub.

For issues, you lose the direct link between solved tickets and commits, but once the mirror updates, you can do a manual link on closing the GH issue.

For PRs, you turn them into patches, apply them locally, and then respond to the PR on GH as you work through it using whatever out-of-GH process you develop. (Or, push the provider into the Fossil contribution stream.)

sticking with Git would be easier

We're back to the first part of this post again: how central are these features to your process? If day-to-day development almost entirely doesn't involve them, then the answer is no: sticking to Git will not be easier.

Maybe my ideal workflow is not very compatible with fossil

Then maybe you need to take a hard look at your ideals. :)

You bring up Golang, a project that doesn't use GitHub's code review feature, which requires a signed CLA before they allow a PR to be pushed up through Gerritt to merge into the repo. So, even your own example doesn't use GH PRs and code review as primary tools.

What stops you from using Gerritt as well, making use of its integration features to strap it to Fossil?

is there a solution in the wild?

I propose that the lack of such tools for Fossil should tell you something: there's a lot of buzz about the sort of thing you speak of, but they're not universally used, because they are not considered essential everywhere.

FOSS didn't begin in 2008 when GitHub came out.

It didn't even begin in 2005 when Git came out.

A great many development organizations get by just fine without third-party code review tooling, without CI/CD, and without pseudonymous drive-by PRs.

Hi Warren, Thanks for taking the time to chat.

I hope you do not mind me saying, but this response was a bit unexpected and a tad overwhelming.

That said...

Having spent more time after seeing your reply, I came across Fossil v Git: Dev Org. Enlightening. I can now understand why my question would be a weird fit to ask on a forum for fossil. Thats fair.

And having read the article after seeing your post, your response makes more sense, and I can appreciate where you are coming from.

I must admit that I am not swayed on the topic of Bazaar v Cathedral style, but that's okay. I find the benefits fossil provides with built-in tickets, wiki, docs, and the autosyncing outweighs any of the immediate cons I'd believe in from not having a bazaar-style development flow.

With that in mind, I still wanted to mention a couple points you brought up:

For CI, that's easy. Just create the necessary .github/workflows/ci.yml file, then mirror it to GitHub.

Absolutely true, sure. But if my goal is to move away from github, regardless of the reason, then what tools do I have to automatically run tests whenever changes are committed to trunk?

Even outside the framework of using a pull request, running tests when changes are made to trunk is, imo, a reasonable ask. People make mistakes. Protecting against those mistakes, even for trusted individuals, seems like a sensible ask.

At least one answer from Sean may be hooks, or possibly some existing CI tooling. That's all I'm really looking for.

What stops you from using Gerritt as well, making use of its integration features to strap it to Fossil?

That is in line with my original question. Does Gerritt support fossil? (I'd assume not, from your post). Is there any out-of-the-box solution that does? This isn't a veiled judgement on fossil; I am genuinely curious as I haven't came across much. This is what I am trying to find out.

You bring up Golang, a project that doesn't use GitHub's code review feature, which requires a signed CLA before they allow a PR to be pushed up through Gerritt to merge into the repo. So, even your own example doesn't use GH PRs and code review as primary tools.

But, it does? You just highlighted it. It requires a code review tool, Gerrit, to make contributions. Even if you skip the PR, Gerrit is still required.

And you can utilize pull requests to engage in discussion, although that is mostly through mirroring comments between Gerrit and GH.

Git has Gerrit. And plenty of similar tools. I want to know if Fossil has [anything?]. Even ignoring the pull-request aspect of it.

If I want to share feedback on code between contributors, what tools are at my disposal with fossil? From the looks of it, would it be fair to say tickets, forum, or potentially a mailling list of contributors would be the only current way?

A great many development organizations get by just fine without third-party code review tooling, without CI/CD, and without pseudonymous drive-by PRs.

Development organizations? Possibly, I wouldn't know. But projects? I would find that unlikely. A quick glance at Github would seem to say otherwise. It works for hundreds of thousands of projects, big and small.

And to be frank, it works for me, which is the sole reason why I am asking.

I am not saying a project needs bazaar-style development to be successful. I'm only asking what tooling exists to replicate portions of that flow with Fossil.

If the answer is "this question does not make sense because fossil isn't trying to support that kind of flow", that's fine. Just say that. Easier to digest lol.

I am not swayed on the topic of Bazaar v Cathedral style

I don't refer you to ESR's book to try to sway you toward bazaar-style development, but because he's drawn a useful metaphor we can use to think about development organization choices. Fossil's current design is better suited to cathedral style than bazaar, but PRs and external code review are more needed by bazaar style projects.

what tools do I have to automatically run tests whenever changes are committed to trunk?

I didn't bother listing them because they've already been given: the RSS-driven thing-runner article, or after-receive hooks + Jenkins, or...

Protecting against those mistakes, even for trusted individuals, seems like a sensible ask.

So write a Fossil before-commit hook to call the static analysis tools you trust to catch those mistakes.

Does Gerritt support fossil?

Out of the box, with a single config GUI click or preference file setting? I highly doubt it. However, I expect it's not hard to integrate them. Not only is Fossil not the only VCS on the planet, Git isn't the second-and-sole-other VCS. I fully expect that there are Subversion users using Gerrit and Jenkins and Travis and... And then there are the Perforce users, and the SCCS users, and...

Thanks.

Fossil's current design is better suited to cathedral style than bazaar, but PRs and external code review are more needed by bazaar style projects.

Okay. That aligns with the stuff Im reading here on the site then. It was my mistake to assume fossil would be a perfect jigsaw piece for whatever kind of development flow I may have been used to.

That help keep things in perspective then, for me. I appreciate the clarity.

I didn't bother listing them because they've already been given

Gotcha. So between what Sean mentioned and what I sussed out are the main options available then. Makes sense.

before-commit hook

The docs seem to indicate that After-Receive may be the way to go if I wanted something server-side. The page doesn't really explain what happens should the script(s) fail, other than that it wont stop a bad commit from syncing nor will the exit code from the script do anything. I can certainly try it out myself later to see what happens, but if I may, what would happen?

I'd assume nothing, unless I setup the scripts in such a way to, say, send an email or slack notification on a test failure or being unable to fetch a dependency. Is that the idea?

An after-receive hook can't prevent a commit from happening, because it's already happened on the client repo. If the server could refuse a commit, that would break fossil sync.

If you want to stop the commit from happening, you have to do it client-side, which means using a before-commit hook.

Yes, that means you have to install all of these checking hooks on each developer machine. That's a necessary cost if you're going to have pre-commit checking while not giving up on proper syncing.

If you want to stop the commit from happening, you have to do it client-side, which means using a before-commit hook

I tried that but a return code from the hook script of non-zero did NOT stop the commit from happening. (Tested on Windows with 2.13 preview)

What could help is an additional new after-commit hook on client side, which stops from auto-syncing. Then any after commit corrective work could be done (like moving the recent commit into a hidden branch) or generate additional automated commitments before letting them go wild all together.

Also the commit would have happened - but it would not go wild without additional review and user confirmation. I suggest let such an after-commit hook generate a message, which is shown in the fossil ui view until either a (new) release button is clicked ("pending SYNC, do auto-sync NOW") or the next successful commit happened.

Hi Curtis,

Feel free to search the forum for some info on the hooks. The feature (AFAIK) is still "experimental" so that may mean you will need to have some trial and error to get it just right.

Here's a forum post on hooks that may help you.

FWIW, I think the forum could work just fine for code reviews. You're likely going to be committing code to a branch, where you're able to make a wiki page about the branch and link specifically to a forum post to solicit feedback about the changes.

Here's an example of a branch with a wiki page: auto-toc

Hello Curtis:

I come from a slightly different direction than you. First C, then Fossil, then Golang, and minimally Github. I do not use Github any more than necessary for downloading the few libraries that I use. Golang is able to collect external libraries from many third party hosters, despite the overwhelming presence of Github. I self-host multiple project repositories on a VPS to provide the parent repository and off-site backup but otherwise use my development computer as the focus of my repository activity.

I consider it unwise to structure a workflow around a public repository hoster unless your focus is exclusively open source.

As a web software developer, I find that Golang working on the server is just one part of a project and there are significant other assets working with the browser. All are contained in the same repository for any given project.

The current popularity of javascript based support tools has passed me by and I stick with tools that can be driven from make as the integration tool of choice. It works well. And I avoid the thousands of tiny third party modules implicit in using node. Who knows when the next one will be withdrawn from service?

The rest of this note describes how I use golang. It makes activity largely independent of repository hosting.

Coupling the above points with a strong preference for hosting local copies of all dependent modules means they can be included in the Fossil repository since there are so few that I use. Keeping local copies of libraries subverts the usual operation of the go.mod file unless you force it to look locally. It took me some time to sort that out but it also works well. A file named makegoenv placed in the golang sub-directory:

  echo "GO111MODULE=on" >go.env
  echo "GOBIN=${PWD}/bin" >>go.env
  echo "unset GOPATH" >>go.env
  echo "GOPRIVATE=project-name" >>go.env
  echo "CGO_ENABLED=1" >>go.env

generates the project specific environment file in the project's golang sub-directory and that is included as a prefix for each golang install statement:

  command-name:
    cd ..; ./makegoenv
    export GOENV=../go.env; go install --tags "see" ${PROJECT}/cmds/$@

This allows for diverse activities on multiple independent projects without concern that the golang environment is defined appropriately.

Project specific dependent modules have their import statements preceded by the project-name value mentioned in the GOPRIVATE statement, so keep the name short and without spaces. It is also the name of the parent directory.

  import (
    "database/sql"
    "encoding/json"
    "fmt"
    "project-name/shared/jsonReply"
    "net/http"
  )

There are tools built into go for the following:

• source code (re-)formatting in the canonical layout (go fmt),
• linting (go vet),
• testing (https://golang.org/cmd/go/#hdr-Test_packages)
• documentation (go doc),
• and others

In web development, the browser user interface dictates much of the testing activity so tooling is dependent on the chosen browser-based technologies, which have nothng to do with the repository if done at the local level.

Chris

Hi Chris!

I hope you do not mind a slight tangent; a technical question:

echo "GO111MODULE=on" >go.env

Have you used any fossil-based golang packages within the last couple months? I ask because I ran into an issue doing exactly that. Seems like a change in fossil has caused go get to stop working for newer fossil repos. Since we are chatting, wanted to see if you've experienced the same pain?

I find your approach very interesting. I know languages like Python have mostly adopted virtualenv (or similar tooling) to deal with the need of changes in environment-level choices.

Using a makegoenv seems like a smart way to deal with those kinds of issues. Thanks for sharing.

..

I think the heart of my question around tooling was not about Golang per se, but rather how to enable outside contributors. I think Warren may have highlighted above why such a question is a bit... misinformed for me to be asking, given the expectations set with fossil around Bazaar v Cathedral style development.

I bring up Go specifically because of its vast ecosystem of tooling available on github and the like, a la Awesome Go. I and many have benefited from the time and effort people have made in contributing to all these packages, and hence why I am interesting in seeing how I can recreate a similar environment, outside of Github.

Sure, there are tools to do so for git, but I am interested in learning if such a thing may be tangible with fossil.

Hi Curtis:

I read your issue report and must say that no, I've not attempted to use go get (some fossil repo latest) so have no experience with same to relay to you. I use three third party go libraries and I started by downloading the master.zip file for each. Two of them are stored under golang/vendor and one is stored under golang/forked because I needed to make non-portable changes.

I was disappointed when I first attempted to use go.mod as described in the golang documentation because it fundamentally assumes a single golang parent directory for all projects without concern for non-go assets. To me golang is just one of a number of tools and I don't want my project split among multiple tool specific directory structures. That makes single repository per project very difficult.

---

If you choose Fossil as your primary repository tool then Warren's suggestion to export to github coupled with selective manual handling of pull requests will likely work for you.

Gotcha; understood!

I will chug along then and see how things work for me. If nothing the issue give's me a chance to try to make a contribution to golang and maybe make lives easier in the future for some fossil+go enthusiasts. I appreciate the info.

Chris (crustyoz) on 2020-11-03 19:40:15:

If you choose Fossil as your primary repository tool then Warren's suggestion to export to github coupled with selective manual handling of pull requests will likely work for you.

... and if you have Github tooling and integrations you like and haven't yet replaced, you can still get benefit from them too. I noticed while setting up mirroring from Fossil to Github that the output of "fossil git export" is worth capturing:

remote: 
remote: GitHub found 6 vulnerabilities on MyProject/myrepos's default branch (2 critical, 2 high, 1 moderate, 3 low). To find out more, visit:
remote:      https://github.com/MyProject/myrepo/network/alerts
remote: 

So it's potentially worth paying attention to the output of your cron job.

Incidentally the alerts I got all related to something mentioned earlier: node artefacts I haven't yet removed from an earlier iteration of my project :)

Dan Shearer