Fossil Forum

User on the check-in lock seems to be different than the default user
Login

User on the check-in lock seems to be different than the default user

User on the check-in lock seems to be different than the default user

(1) By ritz on 2019-09-04 17:26:51 [link] [source]

We are using a shared fossil repository by file:// and by ssh://. The repository frequently gets locked by the same user(say Tom) and remains locked for hours until someone overrides the lock. Tom is the first user with "suv" permissions in the user list and we are pretty sure he does not access the repo at all.

On further debug, we noticed the "config" table is populated with the following entry whereas clearly the default fossil user is myself.

while sleep 1;do echo "Name: $(sqlite3 repo.fossil "select * from config where name like 'ci%';")";done
Name: ci-lock-991a|{"login":"tom","clientid":"ca3194e"}|156

fossil user default
ritz

(2) By sean (jungleboogie) on 2019-09-04 18:07:37 in reply to 1 [link] [source]

Which version of Fossil?

(4) By ritz on 2019-09-04 20:39:01 in reply to 2 [link] [source]

Fossil version 2.9

(3) By Richard Hipp (drh) on 2019-09-04 19:27:00 in reply to 1 [link] [source]

This happens when "Tom" starts a check-in, but then presses Ctrl-C or otherwise aborts the process. Or, perhaps "Tom" is just leaving the check-in comment editor open and going to lunch.

Either way, you can override by adding the --override-lock option.

(5) By ritz on 2019-09-04 20:42:08 in reply to 3 [link] [source]

Tom does not use the fossil anymore. He is no longer a part of the team.

We only see the message that the lock was held by him because he is the first super-user in the list of users. I have tried removing his permissions, but then the lock seems to be held by the next super-user in the list. So basically fossil is just detecting the user wrong and reporting an admin to be having the lock as opposed to the actual user.

(6) By ckennedy on 2019-09-04 20:50:15 in reply to 5 [link] [source]

Are you all opening checkouts from the same repository file? Or are you cloning the repository first to have a local copy before opening a checkout?

(7) By ritz on 2019-09-04 21:18:42 in reply to 6 [link] [source]

We are cloning first to get a local copy and then open the checkout. When we do the commit to this checkout area and try to sync back the commit, we see the commit lock I have mentioned in the original message on the central repo.

(8.1) By Warren Young (wyoung) on 2019-09-05 00:09:32 edited from 8.0 in reply to 1 [link] [source]

Tom is the first user with "suv" permissions in the user list

You're operating on a dangerous set of assumptions there. Fossil user capabilities have no effect at all over non-web interfaces. You can prove this to yourself with a simple test:

$ fossil init a.fossil
$ fossil user new bob -R a.fossil
$ fossil user cap bob '' -R a.fossil      # deny all caps!
$ fossil clone a.fossil b.fossil
$ mkdir b
$ cd b
$ fossil open ../b.fossil 
$ touch x
$ fossil add x
$ fossil ci -m 'added x'
$ fossil ui ../a.fossil 

Now go to the Files tab in repo "a", and you'll find the "x" file that no-caps Bob added to his clone of the repo and was able to successfully push to the parent, repo "a".

Furthermore, you will notice that it wasn't added as "bob" at all, unless that happens to be your OS user name, or you've overridden Fossil's default somehow. This tells you that the user table isn't even consulted, much less have the user's caps checked before allowing the push to the parent repo.

When operating over file:// or ssh:// URLs, only local file permissions on the SQLite repo DB file matter. Fossil user caps don't enter into it.

(11) By Andy Bradford (andybradford) on 2019-09-05 02:35:19 in reply to 8.1 [link] [source]

> When  operating  over   file://  or  ssh://  URLs,   only  local  file
> permissions on the SQLite repo DB  file matter. Fossil user caps don't
> enter into it.

That depends on how SSH is used.  Fossil user caps do enter into the mix
if Fossil is forced to use them:

# fossil user -R test.fossil cap tester u
u

$ fossil clone -A tester ssh://fossil@remote//test clone.fossil
Round-trips: 2   Artifacts sent: 0  received: 6
Clone done, sent: 547  received: 1469  ip: localhost
Rebuilding repository meta-data...
  100.0% complete...
Extra delta compression... 
Vacuuming the database... 
project-id: ac8dc2e43fa3ec9ba6cdb92c7c6c448c774fa956
server-id:  48e9f2b08488d79fff6cf9db0e70d62ec93c56ad
admin-user: tester (password is "wqeuBfEDL3")
$ fossil user default -R clone.fossil tester
$ echo test | fossil wiki -R clone.fossil create home
Created new wiki page home.
$ fossil sync -R clone.fossil                        
Sync with ssh://fossil@remote//test
Round-trips: 1   Artifacts sent: 1  received: 0
Error: not authorized to write
Round-trips: 1   Artifacts sent: 1  received: 0
Sync done, sent: 595  received: 363  ip: remote

Notice that  this user tester is  not allowed to push  to the repository
via SSH because Fossil is enforcing the capabilities.

Now let's give the user the right capabilities:

# fossil user -R test.fossil cap tester v
v

And now he can sync:

$ fossil sync -R clone.fossil                        
Sync with ssh://fossil@remote//test
Round-trips: 2   Artifacts sent: 2  received: 0
Sync done, sent: 1163  received: 812  ip: remote

Thanks,

Andy

(12) By Warren Young (wyoung) on 2019-09-05 02:41:34 in reply to 11 [link] [source]

Fossil user caps do enter into the mix [over SSH] if Fossil is forced to use them:

That's interesting to know, but it's trivial to get around. Change your user to any other name on the local clone, and you can now push wiki articles and anything else you like to the remote.

I'd recommend use of syncing Fossil over SSH only for solo projects and for projects where everyone's trusted to have Setup capabilities...because they do!

(15) By Andy Bradford (andybradford) on 2019-09-05 03:33:47 in reply to 12 [link] [source]

> That's interesting  to know,  but it's trivial  to get  around. Change
> your user to any  other name on the local clone, and  you can now push
> wiki articles and anything else you like to the remote.

The same can be  done with any of the transports,  HTTP, HTTPS, and SSH,
so  that  can hardly  be  attributed  solely to  SSH.  I  could just  as
easily set  my local  Fossil user  to drh and  start pushing  content to
www.fossil-scm.org via HTTPS  and Fossil would not  reject that content.
The commits would  show up as having  been committed by drh,  not by me.
The only thing that  would show the truth of the  matter are the records
in the rcvfrom table on the remote server.

Andy

(9.1) By Andy Bradford (andybradford) on 2019-09-05 01:58:00 edited from 9.0 in reply to 1 [link] [source]

What  you have  discovered  is the  long-standing  behavior that  Fossil
assumes  that if  you are  cloning via  file:// that  you have  complete
access to the file. Consequently, you  get all the settings of that file
when you clone. Observe:

$ fossil new test.fossil
project-id: 8c499a55726fdf98b55a948ec1d2156b337c5040
server-id:  bc464f40c1ea7a8465af7c67acd4babd6dd0bd11
admin-user: amb (initial password is "8EVQYfgJmL")
$ fossil user -R test.fossil new tester
contact-info: 
password: 
Retype new password: 
$ fossil user default -R test.fossil tester
$ fossil user default -R test.fossil
tester
$ fossil clone test.fossil clone.fossil
Repository cloned into clone.fossil
Rebuilding repository meta-data...
  100.0% complete...
Extra delta compression... 
Vacuuming the database... 
project-id: 8c499a55726fdf98b55a948ec1d2156b337c5040
server-id:  bc464f40c1ea7a8465af7c67acd4babd6dd0bd11
admin-user: amb (password is "8EVQYfgJmL")
$ fossil user default -R clone.fossil 
tester

As you can see, Fossil clones it even down to the passwords.

This is not the  case, however, with SSH. When you  clone via SSH, while
by default  it assumes  that you  have complete access  to the  file, it
still uses  the sync  protocol to clone---this  behavior can  further be
locked down by using ForceCommand.

Thanks,

Andy

(10) By Warren Young (wyoung) on 2019-09-05 02:19:27 in reply to 9.1 [link] [source]

While Fossil does use the HTTP/CGI-based sync protocol over SSH, user caps are still effectively ignored over SSH. Or rather, you are the Setup user on both sides of the conn, so you can do pretty much everything to the remote via modifications to the local clone.

Consider this small modification to my example above:

$ fossil init --admin-user ghandi a.fossil
$ fossil clone ssh://localhost/path/to/a.fossil b.fossil
$ mkdir b
$ cd b
$ fossil open ../b.fossil 
$ echo x > x
$ fossil add x
$ fossil ci -m 'added x'
$ fossil ui ../a.fossil 

This time, the check-in went in under your OS user name, who has no login at all on either copy of the repo, so you have no capabilities at all, just like Bob in my example above. Yet, your checkin goes into the local clone and is sync'd over SSH anyway.

If you want user caps to have meaning to Fossil, you need to set up a server.

That will probably also solve this locking problem as a bonus.

(13) By Andy Bradford (andybradford) on 2019-09-05 02:45:51 in reply to 10 [link] [source]

> While Fossil does use the  HTTP/CGI-based sync protocol over SSH, user
> caps are  still effectively ignored over  SSH. Or rather, you  are the
> Setup  user on  both sides  of the  conn, so  you can  do pretty  much
> everything to the remote via modifications to the local clone.

As  I have  shown in  another post,  capabilities can  be enforced  when
using  SSH  and  it is  not  possible  for  the  user to  elevate  those
capabilities beyond what the Fossil administrator chooses. If the Fossil
administrator sets up a special user who owns all the Fossils on the SSH
server, Unix permissions  will automatically deny access  to those files
(provided they  are not widely  open) and then Fossil  capabilities does
the rest.

Thanks,

Andy

(14) By Warren Young (wyoung) on 2019-09-05 03:01:40 in reply to 13 [link] [source]

I can't replicate your result above, with the remote Fossil denying the push as a user with only u capability.

I initially thought it was because I was doing the clone over localhost using the same user who created the repo, so I put the new repo on a remote host where I have a different user name than on my local machine, then I forced Fossil to use that user name in the ssh:// URL. Because if your -A flags, neither my user name on the local machine nor that on the remote got into the Fossil user table on either side. Everything was done as "tester" as far as Fossil was concerned.

The first time I tried your wiki test, and it went through, I realized that on clone, "tester" got capability s due to the -A flag in your command. So, I dropped the local user down to cap u to match the user's definition on the remote, and was able to add a "home2" wiki page the same way, which did sync up to the remote.

Bottom line, I can't make Fossil enforce user caps over SSH, and I doubt it's very secure even if you manage to make it happen.

I remain steadfast: if you want Fossil to enforce user caps, clone and sync over HTTP only.

(16) By Andy Bradford (andybradford) on 2019-09-05 04:04:28 in reply to 14 [link] [source]

> Bottom line,  I can't make  Fossil enforce user  caps over SSH,  and I
> doubt it's very secure even if you manage to make it happen.

Why would it be  any less secure than any of the  other methods? Isn't a
server only as secure as the administrator makes it?

I've written about  this various times in the past  (though maybe not as
much in Fossil Forum as in the old fossil-users mailing list [1]) so the
archives  will have  exact details.  All  it takes  is a  simple use  of
ForceCommand  that  SSH provides  to  users  to restrict  commands  that
clients are allowed to run.

By default  when you  use ssh://  the fossil client  sends a  remote SSH
command of "fossil  test-http" which is what grants the  Setup access to
the user  pushing content. But  if you  use a ForceCommand  that instead
forces the  Fossil client to  run "fossil  http" then the  Fossil server
will  no longer  grant Setup  access. It  is impossible  for the  Fossil
client to  override this. It  would be impossible  for the user  to gain
access to the files  even using his own SSH account  on the same server,
provided  that  the  user  account  who owns  the  files  doesn't  grant
permission to the files.

To make it work, you'll need an SSH  key and a force command that can be
used to  restrict the  access that  a Fossil client  is allowed  to make
(examples can be found in [1]).

I think  this is probably  not any more  complex than other  methods for
setting up fine grain controlled access to a set of Fossil repositories.
The only  setup that is  less complicated to  set up is  running "fossil
server", however, the rest of the methods for setting up Fossil with CGI
or various other  web server technologies are just as  complicated in my
opinion as running it via SSH.

I will concede,  however, that given the claim that  Fossil is best used
by a  small group  of trusted  developers, the  default SSH  behavior in
Fossil  fits perfectly.  But  for the  administrator  who wants  tighter
control, it can be easily accomplished.

> I remain steadfast: if you want Fossil to enforce user caps, clone and
> sync over HTTP only.

And  that is  precisely what  is accomplished  when using  "fossil http"
instead of  "fossil test-http", it's  merely done over a  secure channel
provided by SSH. :-)

[1] https://marc.info/?l=fossil-users&w=2&r=1&s=ForceCommand&q=b

Thanks,

Andy

(17.1) By Andy Bradford (andybradford) on 2019-09-05 04:26:36 edited from 17.0 in reply to 14 [source]

> I can't  replicate your result  above, with the remote  Fossil denying
> the push as a user with only u capability.

The following should help you replicate the results above; apologies for
not providing  more details earlier as  I thought that you  were already
familiar with this particular setup:

I have a Unix user named  "fossil" which requires SSH key access; here's
an example:

# grep amb$ /var/fossil/.ssh/authorized_keys 
command="./fossil.sh tester ./fossils" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPR6UePzS5AdFP+ZJK2GmpsccKftFcfWhjWR25/056e2 amb

And the contents of fossil.sh:

#!/bin/sh
REMOTE_USER="$1"
REPO="${2:-}"
set -- $SSH_ORIGINAL_COMMAND
while [ $# -gt 1 ]; do shift; done
export REMOTE_USER
if [ x"$REPO" != "x" ]
then
  exec ./fossil http "$REPO"
fi
exec ./fossil http "$1"

Each repository in ./fossils will  need to have REMOTE_USER enabled; for
example:

echo "INSERT OR REPLACE INTO config VALUES ('remote_user_ok',1,strftime('%s','now')); " | fossil sql -R test.fossil

Without this enabled,  none of the Fossil users will  work and no access
will  be granted  (well,  anonymous and  nobody will  work  if they  are
enabled).

Now the Fossil administrator can then easily manage access to the Fossil
repositories just using the fossil command line tool.

Thanks,

Andy

(18) By Warren Young (wyoung) on 2019-09-05 04:37:50 in reply to 17.1 [link] [source]

exec ./fossil http

There's the key difference. Normally when you clone or sync using an ssh:// URL, it tunnels HTTP through SSH via fossil test-http, which "[w]orks like the http command but gives setup permission to all users."

Thus the behavior I saw here when trying to replicate your results.

(19) By Andy Bradford (andybradford) on 2019-09-05 04:47:07 in reply to 18 [link] [source]

> Thus the behavior I saw here when trying to replicate your results.

Yes, again, my apologies.  Details matter, right? :-)

Thanks,

Andy

(20.1) By ritz on 2019-09-06 19:03:32 edited from 20.0 in reply to 1 [link] [source]

Thank you all for your response. It definitely helps and we will see what we can do to get the right user showing up for the locks.

The bigger issue for us is that these locks(which sometimes end up inadvertently) expire after 24 hours. Ideally, each commit finishes within seconds so there is no reason to keep the lock hanging more than a minute or so. We verified by changing the following line in xfer.c to 60s, it works perfectly fine with our model.

xfer.c : const sqlite3_int64 maxAge = 3600*24

Is this something that can be implemented in fossil please, or if this can be provided as a configurable item. Thank you!!