Hi, we have a fossil repo on a shared drive, can several users access this at the same time? What is the worflow when collaborating without a server?

Thanks!

The quick answer is "never, ever do that because file locking is notoriously buggy/problematic/slow on network shares and you may corrupt the repository". The preferred approach is for one person to run "fossil server" and have the other users sync with them. If there's no network path between the workstations then another approach is needed, but nothing comes immediately to mind. i'm certain another forum member will soon provide suggestions.

If there's no network path between the workstations then another approach is needed, but nothing comes immediately to mind.

hmmm, that is a tough one. I'd image sharing a master repo on a thumb drive is about the only solution I can think of. That would pretty much defeat the purpose of a distributed version control system, though. Merges could be difficult and files are bound to be overwritten.

If you have a shared network path between all machines, I'm really hoping fossil server will do the trick for you.

There is definitely a network path. The original post said the scenario involves a shared drive!

If there's no network path between the workstations then another approach is needed

I actually considered that problem when designing Fossil oh-so-many years ago. The scenario I envisioned was two (or more) sub-teams working on a high-security project that did not allow an internet connection. So the two sub-teams have no direct network connection.

The solution there is for each subteam to run their own Fossil server hosting the repository. Call the teams X and Y. Periodically, someone makes a clone of the X repository onto a thumb drive. Then they hand-carry the thumb drive to Y's office. There they sync the thumb drive with Y's repository. Then they hand-carry the thumb drive back to X and sync again. Now everybody is in sync.

The above procedure can be generalized. There can be any number of repositories. And there can be multiple "thumb drives" in transit at the same time, which might be important if the facilities are far apart and it takes time to move from one to the next.

The Fossil sync mechanism works fine in the scenario above. There is some latency, in that people on the Y network do not see the latest changes at X until the next thumb-drive arrives. For convenience and for human sanity, perhaps X and Y should work on separate branches, though Fossil itself doesn't really care. Fossil works fine regardless.

Fossil works fine in a disconnected environment such as the above. Git - not so much....

FWIW, I’ve worked for many years with folks sharing one fossil db on a NFS drive. This was sync’ed across sites every 5 minutes, but the team at each of the three sites used just one clone. I’ve had a couple of permission problems (.fossil becomes read-only) and one case where I had to re-clone one site from another.

I do not recommend or suggest it. Just sharing my experience :).

There is definitely a network path. The original post said the scenario involves a shared drive!

Interestingly, a network drive doesn't necessarily require that all machines connecting to that drive also be able to connect to each other. A drive can live in multiple subnets. By "network path" i meant that the workstations could connect directly to each other.

Clarification from OP: Its a shared network drive. We access it via the network. but Because of IT-regulations we're not allowed to run servers.

we're not allowed to run servers.

Does that mean you can't run an application on your computer that opens an outbound tcp connection? That's what the 'fossil server' command is doing.

I read up again on fossil server now, but I'm still not sure how running fossil server on my own computer would help, since the repo is located on the network drive? Thanks

The problem is simply that sharing an sqlite3 db on a network drive is risky, in terms of potentially corrupting it, where "risky" means "a non-zero chance". How close to zero is really anyone's guess, but it's much further away from zero than sharing via (fossil server) is.

The "best practice" solution would be to do away with the copy on the network drive altogether, start (fossil server) on one of your workstations, and have all participants clone from there. Assuming you run the server, each of your colleagues would do:

fossil clone your-ip:8080 blah.fossil
mkdir foo
cd foo
fossil open ../blah.fossil

The caveat is that if you stop the (fossil server) process, your colleagues can't sync with you, but they won't lose anything - they'll just be working in isolation until syncing is again possible.

If you can't run a server anyplace another option might be to use the network drive as a virtual thumb drive.

I think this would mean you need some mechanism to enforce who "possesses" the cloned repository (on the network drive) at any given time, to ensure that no two syncs are run against it at the same time. How you would do that would depend on your circumstances. If your users are at a single location you could even pass around a "talking stick".

Put the original repo on a server. Then each developer makes a clone on his or her personal workstation. All these clones keep all the same information, so if one dies due to a hardware malfunction (even the master repository) it is easily replaced by one of the clones. This is part of the beauty of a distributed version-control system - every clone contains all of the information. So you have automatic backups.

You can run your main repository on a RaspberryPi or similar that you stick in a drawer someplace and forget about. Or on a $5/month hosting service. Fossil does not require a beefy server.

For maximum survivability, it works best to have one or more clones off-site. That way your entire building can be destroyed by a disaster of some kind and your software will survive. The Fossil self-hosting repositories has clones at three different datacenters, run by two different companies, each no closer than 1000 miles from the next. A cron job does "fossil sync" once an hour.

just to visualize the clonable shared repo approach DRH descibed:

1. shared drive: /shared (or \\server\shared )
2. "shared" Fossil repo: /shared/repos/project.fossil ( or X:\repos\project.fossil
3. each team-member mounts the shared drive: /mnt/shared (or X:)
4. each team-member clones the shared Fossil repo locally: 'fossil clone /mnt/shared/repos/project.fossil repos/project.fossil'
5. checkout and work locally
6. auto-sync or pull/push to the shared

Optionally, administer the shared repo (add users, roles, etc.): 'fossil ui /mnt/shared/repos.fossil'

This way, every team-member works off a local clone; while the "master" repo fossil file is hosted on the shared network drive, hopefully it's backed up properly by IT.

If anything to test is a scenario when two users push to the shared master repo at the same time -- whether both would be queued or one would get bounced; either way the master repo should not get corrupted.

"The problem is simply that sharing an sqlite3 db on a network drive is risky, in terms of potentially corrupting it ..."

TLDR; yes you can use fossil shared by many users on a decent network file system. I've used it on NFS and Moosefs with minimal problems.

At work we have several hundred fossils shared off of an NFS share and we have very few problems. We did have some problems in the past and they were traced to the file lock queues on the NFS filers. If the lock queue got overwhelmed by a storm of locks then locks would be dropped and you would have the dreaded "locked database" and occasionally db corruption. The solution was to identify the other applications that were heavy on the file locks and reduce the locking frequency. This turned out to be easy as the other application using a lot of locks was one I wrote (I really love sqlite3 and build it into everything :), the application is Megatest: http://www.kiatoa.com/fossils/megatest). To reduce the lock frequency either wrap sequential reads or writes with transactions or move the db to /tmp or other local drive. We brought the lock rate down from 1000's per second down to a few a second and the problems went away.

Because of IT-regulations we're not allowed to run servers.

Ok, so that is the constraint. This is a bit like being asked to produce a beautiful painting using only your left hand while wearing an oven mitt. Nevertheless, rules are rules. Let's see what we can do to route around the IT-imposed damage...

I recommend that you establish a central repository on your shared filesystem. But do not checkin or checkout directly to or from this central repository. Instead use the shared repository only to clone and sync. Each developer does:

 fossil clone file:/shared/drive/repos/project.fossil myproj.fossil

In other words, each developer clones the shared repository into a local repository on their own local disk. Then do all check-outs and check-ins to from this local repository. Auto-sync still works with the file: URL scheme.

Periodically run:

 fossil test-integrity file:/shared/drive/repos/project.fossil

If any problems are seen, replace the shared repository with a copy of any of the developer clones and have all developers run "fossil sync".

The shared filesystem will eventually malfunction and corrupt the central repository. But because each developer has a private clone, you are safe from data loss because there is at least one backup on each developer workstation. And if the shared filesystem and network is well-maintained by IT, corrupting malfunctions should be relatively infrequent.

You also lose the ability to have a central site where you can go to see a timeline and get a big picture of everything that is going on. But you can still run

 fossil sync; fossil ui

You won't be able to use email/chat to share URLs of interesting files, diffs, tickets, or bisects since you don't have a common server location that everyone can use. But if all of the developers are in the same office, maybe that is not as important. You also won't have automatic distributed backups, but perhaps IT will do periodic off-site backups of the shared folders for you. Even so, if it were me, I'd probably keep a separate back-up clone on a thumb drive, just in case...

The Megatest skin for its fossil repository looks pretty cool. There is any place where we can get such skin?

Cheers,

Hi all, OP here. First I would just like to say thanks to everyone for taking the time to answer this. I have learned a lot about different workflows, and what to do's/dont's. You really have a friendly and welcoming community going on here, thanks again.

Now for the problem at hand it seems the easiest and least problematic (also for IT) to run fossil server. If I understand right, this command only opens a tcp port?

this command only opens a tcp port?

Yes, 'fossil server' will default to port 8080 on the host where the command is issued, unless that port is in use.

You could try this yourself on your non-work machine and on your own network: download the Fossil binary, create an empty repo, run 'fossil server', then browse to http://<yourfossilip>:8080 from your machine and a smart phone and see the repo. You should see the repo from both devices.

If your team are on different LANs, i.e. different offices, it's possible adjustments will need to be made in your firewall - but best to let your IT dept make that determination.

It would make sense to see this in the project context. It seems you're in the initial stages of setting up your project/team.

I'm not sure if you already took care of this, but here're some popular development project setup items:

1. Shared storage, practically structured
2. Version Control
3. Wiki or some document system
4. Issue tracker
5. Test evironment/server
6. Project planing/tracking system

Since you have "IT" dept., your organization may already have some options defined, so your IT would know how to set it up for your team. It's common to have some sort of server for development/testing purposes.

2,3,4 can be handled with Fossil. Possibly, 6 too if using low-tech approach with some shared spreadsheet.

If you're trying Fossil as an alternative VCS option, it'd still make sense to run it with IT, just to make sure backup is properly handled. If there's a development server at all, then Fossil can be served as easily as fossil server.

On the other hand, if you're trying to start fast and "below the radar", the easiest option is the Fossil file sync, as outlined above. However, we all know how "temporary" setup shoots deep-deep roots such that no one would dare to migrate it to anywhere more robust.

6 Project planning/tracking system

Requirements and task tracking can be done with an issue tracker. That may seem strange but requirements are basically the same as feature request issues and task tracking isn't much different. Each requirement and project task becomes a ticket. (Requirements often get revised and using issue tickets also provides for attaching comments to requirements.)

The IDs of requirement tickets become requirement IDs. Not as nice as sequentially numbered requirements, but when you insert additional requirements, you end up with non-sequential IDs, anyway. (Requirement IDs make requirement trace-ability easier.)

(Where I work, we used to use the DOORS requirements management system. But when you look at what it does, it's really just a half-baked issue tracker with a fancy report generator.)

Design documents are just files. And issue tickets for design documents are just like issue tickets for code. And for software design, if you use Doxygen, your design docs are in the comment blocks of your source files.

You are welcome to pull it and use it. DRH contacted me and may make it available.

You can run Fossil as a server (as suggested by others) or alternatively run Fossil behind an existing, IT-approved web server instance (if one exists) by using the CGI/SCGI capability. That way, it isn't running all the time and thus fossil is not a server. Fossil becomes an "app" of sorts. In this case, Fossil doesn't open any listening ports.

Of course, run this solution past your IT folks for approval.