User on the check-in lock seems to be different than the default user
(1) By ritz on 2019-09-04 17:26:51 [link] [source]
We are using a shared fossil repository by file:// and by ssh://. The repository frequently gets locked by the same user(say Tom) and remains locked for hours until someone overrides the lock. Tom is the first user with "suv" permissions in the user list and we are pretty sure he does not access the repo at all.
On further debug, we noticed the "config" table is populated with the following entry whereas clearly the default fossil user is myself.
while sleep 1;do echo "Name: $(sqlite3 repo.fossil "select * from config where name like 'ci%';")";done
Name: ci-lock-991a|{"login":"tom","clientid":"ca3194e"}|156
fossil user default
ritz
(2) By sean (jungleboogie) on 2019-09-04 18:07:37 in reply to 1 [link] [source]
Which version of Fossil?
(4) By ritz on 2019-09-04 20:39:01 in reply to 2 [link] [source]
Fossil version 2.9
(3) By Richard Hipp (drh) on 2019-09-04 19:27:00 in reply to 1 [link] [source]
This happens when "Tom" starts a check-in, but then presses Ctrl-C or otherwise aborts the process. Or, perhaps "Tom" is just leaving the check-in comment editor open and going to lunch.
Either way, you can override by adding the --override-lock option.
(5) By ritz on 2019-09-04 20:42:08 in reply to 3 [link] [source]
Tom does not use the fossil anymore. He is no longer a part of the team.
We only see the message that the lock was held by him because he is the first super-user in the list of users. I have tried removing his permissions, but then the lock seems to be held by the next super-user in the list. So basically fossil is just detecting the user wrong and reporting an admin to be having the lock as opposed to the actual user.
(6) By ckennedy on 2019-09-04 20:50:15 in reply to 5 [link] [source]
Are you all opening checkouts from the same repository file? Or are you cloning the repository first to have a local copy before opening a checkout?
(7) By ritz on 2019-09-04 21:18:42 in reply to 6 [link] [source]
We are cloning first to get a local copy and then open the checkout. When we do the commit to this checkout area and try to sync back the commit, we see the commit lock I have mentioned in the original message on the central repo.
(8.1) By Warren Young (wyoung) on 2019-09-05 00:09:32 edited from 8.0 in reply to 1 [link] [source]
Tom is the first user with "suv" permissions in the user list
You're operating on a dangerous set of assumptions there. Fossil user capabilities have no effect at all over non-web interfaces. You can prove this to yourself with a simple test:
$ fossil init a.fossil
$ fossil user new bob -R a.fossil
$ fossil user cap bob '' -R a.fossil # deny all caps!
$ fossil clone a.fossil b.fossil
$ mkdir b
$ cd b
$ fossil open ../b.fossil
$ touch x
$ fossil add x
$ fossil ci -m 'added x'
$ fossil ui ../a.fossil
Now go to the Files tab in repo "a", and you'll find the "x" file that no-caps Bob added to his clone of the repo and was able to successfully push to the parent, repo "a".
Furthermore, you will notice that it wasn't added as "bob" at all, unless that happens to be your OS user name, or you've overridden Fossil's default somehow. This tells you that the user table isn't even consulted, much less have the user's caps checked before allowing the push to the parent repo.
When operating over file:// or ssh:// URLs, only local file permissions on the SQLite repo DB file matter. Fossil user caps don't enter into it.
(11) By Andy Bradford (andybradford) on 2019-09-05 02:35:19 in reply to 8.1 [link] [source]
> When operating over file:// or ssh:// URLs, only local file > permissions on the SQLite repo DB file matter. Fossil user caps don't > enter into it. That depends on how SSH is used. Fossil user caps do enter into the mix if Fossil is forced to use them: # fossil user -R test.fossil cap tester u u $ fossil clone -A tester ssh://fossil@remote//test clone.fossil Round-trips: 2 Artifacts sent: 0 received: 6 Clone done, sent: 547 received: 1469 ip: localhost Rebuilding repository meta-data... 100.0% complete... Extra delta compression... Vacuuming the database... project-id: ac8dc2e43fa3ec9ba6cdb92c7c6c448c774fa956 server-id: 48e9f2b08488d79fff6cf9db0e70d62ec93c56ad admin-user: tester (password is "wqeuBfEDL3") $ fossil user default -R clone.fossil tester $ echo test | fossil wiki -R clone.fossil create home Created new wiki page home. $ fossil sync -R clone.fossil Sync with ssh://fossil@remote//test Round-trips: 1 Artifacts sent: 1 received: 0 Error: not authorized to write Round-trips: 1 Artifacts sent: 1 received: 0 Sync done, sent: 595 received: 363 ip: remote Notice that this user tester is not allowed to push to the repository via SSH because Fossil is enforcing the capabilities. Now let's give the user the right capabilities: # fossil user -R test.fossil cap tester v v And now he can sync: $ fossil sync -R clone.fossil Sync with ssh://fossil@remote//test Round-trips: 2 Artifacts sent: 2 received: 0 Sync done, sent: 1163 received: 812 ip: remote Thanks, Andy
(12) By Warren Young (wyoung) on 2019-09-05 02:41:34 in reply to 11 [link] [source]
Fossil user caps do enter into the mix [over SSH] if Fossil is forced to use them:
That's interesting to know, but it's trivial to get around. Change your user to any other name on the local clone, and you can now push wiki articles and anything else you like to the remote.
I'd recommend use of syncing Fossil over SSH only for solo projects and for projects where everyone's trusted to have Setup capabilities...because they do!
(15) By Andy Bradford (andybradford) on 2019-09-05 03:33:47 in reply to 12 [link] [source]
> That's interesting to know, but it's trivial to get around. Change > your user to any other name on the local clone, and you can now push > wiki articles and anything else you like to the remote. The same can be done with any of the transports, HTTP, HTTPS, and SSH, so that can hardly be attributed solely to SSH. I could just as easily set my local Fossil user to drh and start pushing content to www.fossil-scm.org via HTTPS and Fossil would not reject that content. The commits would show up as having been committed by drh, not by me. The only thing that would show the truth of the matter are the records in the rcvfrom table on the remote server. Andy
(9.1) By Andy Bradford (andybradford) on 2019-09-05 01:58:00 edited from 9.0 in reply to 1 [link] [source]
What you have discovered is the long-standing behavior that Fossil assumes that if you are cloning via file:// that you have complete access to the file. Consequently, you get all the settings of that file when you clone. Observe: $ fossil new test.fossil project-id: 8c499a55726fdf98b55a948ec1d2156b337c5040 server-id: bc464f40c1ea7a8465af7c67acd4babd6dd0bd11 admin-user: amb (initial password is "8EVQYfgJmL") $ fossil user -R test.fossil new tester contact-info: password: Retype new password: $ fossil user default -R test.fossil tester $ fossil user default -R test.fossil tester $ fossil clone test.fossil clone.fossil Repository cloned into clone.fossil Rebuilding repository meta-data... 100.0% complete... Extra delta compression... Vacuuming the database... project-id: 8c499a55726fdf98b55a948ec1d2156b337c5040 server-id: bc464f40c1ea7a8465af7c67acd4babd6dd0bd11 admin-user: amb (password is "8EVQYfgJmL") $ fossil user default -R clone.fossil tester As you can see, Fossil clones it even down to the passwords. This is not the case, however, with SSH. When you clone via SSH, while by default it assumes that you have complete access to the file, it still uses the sync protocol to clone---this behavior can further be locked down by using ForceCommand. Thanks, Andy
(10) By Warren Young (wyoung) on 2019-09-05 02:19:27 in reply to 9.1 [link] [source]
While Fossil does use the HTTP/CGI-based sync protocol over SSH, user caps are still effectively ignored over SSH. Or rather, you are the Setup user on both sides of the conn, so you can do pretty much everything to the remote via modifications to the local clone.
Consider this small modification to my example above:
$ fossil init --admin-user ghandi a.fossil
$ fossil clone ssh://localhost/path/to/a.fossil b.fossil
$ mkdir b
$ cd b
$ fossil open ../b.fossil
$ echo x > x
$ fossil add x
$ fossil ci -m 'added x'
$ fossil ui ../a.fossil
This time, the check-in went in under your OS user name, who has no login at all on either copy of the repo, so you have no capabilities at all, just like Bob in my example above. Yet, your checkin goes into the local clone and is sync'd over SSH anyway.
If you want user caps to have meaning to Fossil, you need to set up a server.
That will probably also solve this locking problem as a bonus.
(13) By Andy Bradford (andybradford) on 2019-09-05 02:45:51 in reply to 10 [link] [source]
> While Fossil does use the HTTP/CGI-based sync protocol over SSH, user > caps are still effectively ignored over SSH. Or rather, you are the > Setup user on both sides of the conn, so you can do pretty much > everything to the remote via modifications to the local clone. As I have shown in another post, capabilities can be enforced when using SSH and it is not possible for the user to elevate those capabilities beyond what the Fossil administrator chooses. If the Fossil administrator sets up a special user who owns all the Fossils on the SSH server, Unix permissions will automatically deny access to those files (provided they are not widely open) and then Fossil capabilities does the rest. Thanks, Andy
(14) By Warren Young (wyoung) on 2019-09-05 03:01:40 in reply to 13 [link] [source]
I can't replicate your result above, with the remote Fossil denying the push as a user with only u capability.
I initially thought it was because I was doing the clone over localhost using the same user who created the repo, so I put the new repo on a remote host where I have a different user name than on my local machine, then I forced Fossil to use that user name in the ssh:// URL. Because if your -A flags, neither my user name on the local machine nor that on the remote got into the Fossil user table on either side. Everything was done as "tester" as far as Fossil was concerned.
The first time I tried your wiki test, and it went through, I realized that on clone, "tester" got capability s due to the -A flag in your command. So, I dropped the local user down to cap u to match the user's definition on the remote, and was able to add a "home2" wiki page the same way, which did sync up to the remote.
Bottom line, I can't make Fossil enforce user caps over SSH, and I doubt it's very secure even if you manage to make it happen.
I remain steadfast: if you want Fossil to enforce user caps, clone and sync over HTTP only.
(16) By Andy Bradford (andybradford) on 2019-09-05 04:04:28 in reply to 14 [link] [source]
> Bottom line, I can't make Fossil enforce user caps over SSH, and I > doubt it's very secure even if you manage to make it happen. Why would it be any less secure than any of the other methods? Isn't a server only as secure as the administrator makes it? I've written about this various times in the past (though maybe not as much in Fossil Forum as in the old fossil-users mailing list [1]) so the archives will have exact details. All it takes is a simple use of ForceCommand that SSH provides to users to restrict commands that clients are allowed to run. By default when you use ssh:// the fossil client sends a remote SSH command of "fossil test-http" which is what grants the Setup access to the user pushing content. But if you use a ForceCommand that instead forces the Fossil client to run "fossil http" then the Fossil server will no longer grant Setup access. It is impossible for the Fossil client to override this. It would be impossible for the user to gain access to the files even using his own SSH account on the same server, provided that the user account who owns the files doesn't grant permission to the files. To make it work, you'll need an SSH key and a force command that can be used to restrict the access that a Fossil client is allowed to make (examples can be found in [1]). I think this is probably not any more complex than other methods for setting up fine grain controlled access to a set of Fossil repositories. The only setup that is less complicated to set up is running "fossil server", however, the rest of the methods for setting up Fossil with CGI or various other web server technologies are just as complicated in my opinion as running it via SSH. I will concede, however, that given the claim that Fossil is best used by a small group of trusted developers, the default SSH behavior in Fossil fits perfectly. But for the administrator who wants tighter control, it can be easily accomplished. > I remain steadfast: if you want Fossil to enforce user caps, clone and > sync over HTTP only. And that is precisely what is accomplished when using "fossil http" instead of "fossil test-http", it's merely done over a secure channel provided by SSH. :-) [1] https://marc.info/?l=fossil-users&w=2&r=1&s=ForceCommand&q=b Thanks, Andy
(17.1) By Andy Bradford (andybradford) on 2019-09-05 04:26:36 edited from 17.0 in reply to 14 [source]
> I can't replicate your result above, with the remote Fossil denying
> the push as a user with only u capability.
The following should help you replicate the results above; apologies for
not providing more details earlier as I thought that you were already
familiar with this particular setup:
I have a Unix user named "fossil" which requires SSH key access; here's
an example:
# grep amb$ /var/fossil/.ssh/authorized_keys
command="./fossil.sh tester ./fossils" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPR6UePzS5AdFP+ZJK2GmpsccKftFcfWhjWR25/056e2 amb
And the contents of fossil.sh:
#!/bin/sh
REMOTE_USER="$1"
REPO="${2:-}"
set -- $SSH_ORIGINAL_COMMAND
while [ $# -gt 1 ]; do shift; done
export REMOTE_USER
if [ x"$REPO" != "x" ]
then
exec ./fossil http "$REPO"
fi
exec ./fossil http "$1"
Each repository in ./fossils will need to have REMOTE_USER enabled; for
example:
echo "INSERT OR REPLACE INTO config VALUES ('remote_user_ok',1,strftime('%s','now')); " | fossil sql -R test.fossil
Without this enabled, none of the Fossil users will work and no access
will be granted (well, anonymous and nobody will work if they are
enabled).
Now the Fossil administrator can then easily manage access to the Fossil
repositories just using the fossil command line tool.
Thanks,
Andy
(18) By Warren Young (wyoung) on 2019-09-05 04:37:50 in reply to 17.1 [link] [source]
exec ./fossil http
There's the key difference. Normally when you clone or sync using an ssh:// URL, it tunnels HTTP through SSH via fossil test-http, which "[w]orks like the http command but gives setup permission to all users."
Thus the behavior I saw here when trying to replicate your results.
(19) By Andy Bradford (andybradford) on 2019-09-05 04:47:07 in reply to 18 [link] [source]
> Thus the behavior I saw here when trying to replicate your results. Yes, again, my apologies. Details matter, right? :-) Thanks, Andy
(20.1) By ritz on 2019-09-06 19:03:32 edited from 20.0 in reply to 1 [link] [source]
Thank you all for your response. It definitely helps and we will see what we can do to get the right user showing up for the locks.
The bigger issue for us is that these locks(which sometimes end up inadvertently) expire after 24 hours. Ideally, each commit finishes within seconds so there is no reason to keep the lock hanging more than a minute or so. We verified by changing the following line in xfer.c to 60s, it works perfectly fine with our model.
xfer.c : const sqlite3_int64 maxAge = 3600*24
Is this something that can be implemented in fossil please, or if this can be provided as a configurable item. Thank you!!