Fossil User Forum

disconnected local fossil status check (without having repository access)
Login

disconnected local fossil status check (without having repository access)

disconnected local fossil status check (without having repository access)

(1.1) Originally by MBL (RoboManni) with edits by Stephan Beal (stephan) on 2020-08-25 09:49:51 from 1.0 [link] [source]

(Posted by By MBL (RoboManni), edited slightly by Stephan for broken link and markdown formatting.)

Use case

I make a checkout on a machine (Windows XP,7,10) and then it remains on shop floor disconnected from the repository and unattended for long time. When I get remote access I do not have connectivity to the repository folder (nor the global config database - but network connection to a fossil server instance) but the FOSSIL checkout file from times of rollout is still existing and also a copy of fossil.exe. - I have to check for any changes.

Question

How can I use the local checkout file _FOSSIL_ (rsp .fslckout) with fossil.exe to just do a status check for changes? For this to do the connectivity or access to the repository is not needed - isn't it? All checkout belonging information is staged in that local file.

Subject for discussion

Can the fossil status command get an option to not require the repository nor global config database!? Not even a network connectivity? Should be okay to check for "still origin as rolled out" or "changes exist on files..." (which then become subject for being reviewed and checked in as new status or reverted to previous version).

And additional benefitting from compare results with OS file system

I refer to my older thread regarding comparisons at url forum link. Getting the status check results as a comparison between information available in _FOSSIL_ and local file system that would match my use case perfectly and would really support my tasks very much. I hope that other users see also a big benefit in such standalone mode status checks.

Idea for realization

Calling fossil lock (a new command) to freeze the rolled out status (with change into disconnected standalone checkout only mode) and fossil unlock to unfreeze (and reconnect to local repository folder). While frozen a double click (on windows) or call of fossil without parameters would be the same like a disconnected standalone call as substitution for the fossil status. (Only fossil version makes also sense as one of few still supported parameters in "standalone" alias "checkout only" mode.)

Further improvement

Unexpected changes to the checked out FOSSIL version would be impossible while frozen. The "fossil unlock" could also be secured with a hashed password inside the FOSSIL checkout file which comes from an additional parameter: fossil lock withPassword -m "and optional comment" command. The change back into the normal known mode then look like fossil unlock withPassword .

Best regards MBL

(2.1) By Stephan Beal (stephan) on 2020-08-25 10:07:50 edited from 2.0 in reply to 1.0 [link] [source]

How can I use the local checkout file FOSSIL (rsp .fslckout) with fossil.exe to just do a status check for changes? For this to do the connectivity or access to the repository is not needed - isn't it?

  • If you want to check for changes against the latest remote copy, yes, access to the upstream remote is necessary.
  • If you just want to check for changes to your local checkout against the last-synced version in the repository, no, you don't need a network - that all happens locally on your machine.

All checkout belonging information is staged in that local file.

Kind of. The checkout database file (_FOSSIL_ or .fslckout) contains state about your checkout, but it gets out of sync with the files in that checkout when they are modified, until status (or equivalent) is run, which will update its "staged" state vis a vis the files in that checkout.

Can the fossil status command get an option to not require the repository

No. Fossil cannot open a checkout database without also immediately opening its associated repository database, and it will fail if the repo cannot be opened. Without the repository, the checkout db has no mapping of hashes to files, so it would be literally impossible for fossil to determine whether any changes have been made to the checkout.

nor global config database!?

The global config database is not strictly needed in all contexts, but its use is deeply embedded in fossil.

i seem to recall one of the developers trying to work on a solution for systems in which the global config db is read-only, or maybe even non-existent, but do not recall that ever being implemented.

The global config is local to any given system, and there is no requirement that the global configs in two computers have any of the same settings, so it "shouldn't" matter to you which copy of a global configuration you are using.

Not even a network connectivity?

The status command does not need the network. It only compares a checkout against its associated repository.

I refer to my older thread... Getting the status check results as a comparison between information available in FOSSIL and local file system that would match my use case perfectly and would really support my tasks very much..

i am personally not convinced that noting the file sizes for changes is in any way relevant or provides any information which a user can act upon (and if they cannot act upon it, it's just noise). i strongly suspect that you are "over-thinking" how useful that information would be to you in practice. Knowing that their hashes do not match is literally all one needs to know for purposes of knowing whether a file has been modified.

I hope that other users see also a big benefit in such standalone mode status checks

To the best of my recollection those features have never been requested before, the implication being that it's not something which people have been missing.

Calling fossil lock (a new command) to freeze the rolled out status (with change into disconnected standalone checkout only mode) and fossil unlock to unfreeze (and reconnect to local repository folder).

Locking is not reliable in a distributed SCM. This topic has been beaten to death in other threads, notably /forumpost/2afc32b1ab.

(3) By MBL (RoboManni) on 2020-08-25 12:00:06 in reply to 2.1 [link] [source]

Locking is not reliable in a distributed SCM. This topic has been beaten to death in other threads, notably /forumpost/2afc32b1ab.

Thank you for the feedback ... then I would like to not call it lock and unlock but connected and disconnected mode (or freeze and unfreeze). The mentioned case of locking a file is not what I meant. My users are able to do local required changes (however they are supposed to not do them on their own or to inform me about reasons). Local changes are allowed and possible (therefore files are not "locked") ... but not undetected on my next remote visit of the machine. They are not SCM users and all work to get in sync would remain with me. I am the only user in that environment doing the versioned changes; and changes done on one machine may become important also for the others. Having to have the repository connection is a lot more of work and in most cases would not be necessary as there are "still no changes".

For the SCM the disconnect or freeze would be like fossil close --force and the connect or unfreeze like fossil open --keep --force --nested --setmtime . Dragging-and-dropping a frozen and renamed status file (see below) would be enough to come back to a specific version without having to remember that many parameters. Such a status file would be like a memory file for one version or a command file to open such version with the mentioned parameters; but easier to use - at least while having access to the repository.

I have first to check if there are any changes - and if they are to see if I have to sync them or if they can remain out-of-sync (that's what the compare function would help me with before spending a lot more of time).

The command fossil disconnect [-m "COMMENT"] [PASSWORD] could do all necessary preparation of the local checkout file FOSSIL to act as a memory for last checked out version of each belonging file. Even file sizes could be calculated and collected and stored for offline usage with the status command. And also some of the stash commands could remain ready for usage (for a quick rollback to one of few known predefined status'). In the moment of disconnection the local checked out files are in sync with the frozen file in the repository as well as their sizes would also be current and valid while stepping into the disconnected mode.

...file sizes for changes is in any way relevant or provides any information which a user can act upon...

I would use that comparison information like I am used to use the shell commands dir respective ls with date/time and file sizes. The expected enhancement is more or less just a filter for related files for the external commands to use. Instead of getting all files I would just see the information of the files which are member of the last done (and disconnected) checkout. Even without getting additional information from the FOSSIL file this would support my tasks already.

This enhancement of disconnecting (one of many distributed) checkouts from the repository would enable fossil to become a handy tool for restrictive environments to just check whether the checked out files are still unchanged and intact or not (and with prepared stash ids even more).

...until status (or equivalent) is run, which will update its "staged" state vis a vis the files in that checkout

In disconnected mode the status command would have to not do that update anymore but just to compare local files with that staged information.

Calling fossil with a filepath as argument

could tell which local frozen status to use for the comparison (would allow the easy use of any renamed file of FOSSIL or .fslckout . An additional argument during disconnection could automatically do this renaming, too. - The filename during re-connection will become _FOSSIL or .fslckout again as we are all used to.

Even more use cases come to my mind like probing for "which checked out version is this? Which version would have least deviations? Which installation has grown or has shrunk?"; without having to have connection to the not installed repository. This could cover some night-shift actions of rolling a productive version back and forward or going back to another harddisk backup for any reason.

To the best of my recollection those features have never been requested before, the implication being that it's not something which people have been missing.

Thank you for this compliment to be the first with some really new features to offer to you

Best regards MBL

(4) By Warren Young (wyoung) on 2020-08-25 12:21:06 in reply to 3 [link] [source]

Some of what you want already exists as of Fossil 2.12: fossil remote off/on.

(5) By MBL (RoboManni) on 2020-08-25 13:14:12 in reply to 4 [link] [source]

Thank you for the reply

But remote on/off just affects sync/pull/push/autosync.

What I am dreaming of is some kind of a manifest.sqlite file for easy and fast use with fossil.exe directly to verify (and get an impact of) what has changed - or if the checked out file structure is still intact and unmodified.

This manifest file (a frozen and renamed version of local checkout file FOSSIL rsp. .fslckout) can be inherited from a previously open checkout and can be used for easy reopening (unfreeze) the same checkout without local changes, ready for status and commit* or **revert.

While working with the frozen manifest.sqlite file -to detect any change- no need to have any link to the repository nor network. A "living manifest"; not just a copy of the manifest text files but much more; with direct use of fossil.exe (Windows) rsp. fossil (Linux).

Regards MBL

(6) By Richard Hipp (drh) on 2020-08-25 14:55:11 in reply to 1.1 [link] [source]

I make a checkout on a machine (Windows XP,7,10) and then it remains on shop floor disconnected from the repository and unattended for long time. When I get remote access I do not have connectivity to the repository folder (nor the global config database - but network connection to a fossil server instance) but the _FOSSIL_ checkout file from times of rollout is still existing and also a copy of fossil.exe. - I have to check for any changes.

What you need to do is make a clone of the repository on the machine that will go disconnected, then check-out from that clone. Then, you can go to the machine while it is off network and do "fossil status" or "fossil status --hash to find out exactly which files have been changes or deleted.

The repository must be accessible to the fossil status command because the fossil status command needs to check the values for various settings, and those settings are stored in the repository, not in the _FOSSIL_ file.

You will still be able to keep the checkout machine in sync with the remote repository when the machine is connected to the network. Simply do "fossil up" and the local clone will automatically pull from the network-connected main repository.

(7) By MBL (RoboManni) on 2020-08-25 15:19:54 in reply to 6 [link] [source]

thank you, I think I got your point and that I cannot convince you with my feature request.

It is not how I think it would be easiest but now I go with another idea how I can do it in a non-easy way ... doing some wrapping around with some extra tools.

(8) By Marcio Gabe (marciogabe) on 2020-08-25 15:53:24 in reply to 3 [link] [source]

In disconnected mode the status command would have to not do that update anymore but just to compare local files with that staged information.

It looks to me that your setup for some reason is not what usually can or should be done with Fossil.

For what you've said, it looks like you are keeping one central fossil repository on a network folder or location and just using 'fossil open' command on each user's machine, so locally on that machine you just have the FOSSIL or the .fslckout file. Did I get it right? (Please describe your scenario if different).

If that's the case, you are missing out the proper way of setting up Fossil, that is, you can and should still keep a central repository, and maybe better than using a shared folder of some kind, use the 'fossil server' command on the server machine to keep it running as a server. Now on each client machine, instead of 'fossil open' you'll use 'fossil clone' command first, and this will create a local copy of fossil repository (not just the checkout db file). Then you can do all operations locally... like checking status with the 'fossil status' command, our 'fossil changes' and 'fossil extra' to see the changes user made or files added to the local working folder. This will not require any online connection to a network. It will perform just as you are asking, meaning even in a disconnected machine, fossil will still allow you to see all changes made, etc, etc. The only difference is that it will be using not only the local FOSSIL file, but also the local respository.fossil file (local cloned repo) that will be available on the machine as well.

Settings can be configured to not allow users to push to the main repository, and even to the local cloned copy if I'm not mistaken, so you still will have all the control you want to be the only one performing the actual committing of the files...

Does this sound like the type of feature you are looking for? If not, please, if you can provide some more input as to how exactly is your current repository setup, maybe it will be easy to figure out a way to set things up in a better way.

Just my 2 cents....

MG

(9) By MBL (RoboManni) on 2020-08-25 17:33:52 in reply to 8 [source]

Yes it sounds similar and I know this way to work and some times also use it this way also; e.g. for initial rollout and during development phases. With 2.12.1 and the backoffice hooks I may do more and better in near future, too. (after-receive runs a hook with fossil update --setmtime automatically; this brings me closer to CI/CD. Looking forward for a 'multi-remote sync' feature ;-) With some additional hook types it could be done by backoffice also, I guess.)

On disconnected productive machines I do not want to leave a clone of the whole repository on each machine for various reasons but the installation as small as possible; hence only the checked out files and a minimum in addition. My remote access is with UltraVNC and connection is often at slow speed and with short maintenance window.

For the distribution I copy my local checkout inclusive the 3 manifest files (option 'rut') and FOSSIL file to remote machines. For version identification (what was left years or months ago) I only need to get FOSSIL (or manifest.uuid) uploaded and can check which checkout version it was. - Next step is to see what has changed.

By uploading the whole distributed folder with sub-folders at once I get all files to my local PC. With fossil status I can then check for all deviations. UltraVNC does a local zip for a folder-upload and unzip once that file has arrived and that's why it is faster to upload much more than I need. (My harddisk fills up too quickly)

The step to upload all just to find out that nothing was changed (which should be the normal case) I would like to skip by just running fossil.exe with the frozen version of the checkout FOSSIL: "check deviations from manifested list of files". The file manifest contains that information but without having the repository connected they cannot be verified; fossil does not run 'status' without repository. What's missing are the file artifact hashes in FOSSIL and the ability to use them without repository connection or a tool that can use manifest file and generate hash codes the same way as fossil does.

Thats why I wrote a Lua script with lsqlite3x (x stands for my dll version with many extensions built in over many years), which accesses the manifest file to obtain the versioned file names and their hashes. Then I can check them with regenerated hashes from the file system. For now I keep the originally provided information in a database (like a duplicate of FOSSIL but with file hash codes from the manifest file. I may have to extend FOSSIL rsp .fslckout to keep the information from the manifest in one file (what I expected from my feature request to be done by fossil.exe itself).

This process could be done better and easier, that was my dream with the feature request for a fossil freeze to be used as 'runnable manifest verification' tool - as an enhancement of a pure SCM. Enabling fossil to become usable also in CI/CD environments in future on small devices.

I hope all is understandable and now clear enough. If not then do not hesitate to continue asking.