Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!") with --repolist
(1) By Jakob Stoklund Olesen (stoklund) on 2021-01-01 18:29:52 [source]
Hi,
I'm developing a plugin for the buildbot CI framework, and I'd like to use Fossil's JSON API when polling for new commits to test.
The JSON API works great from the command line and when fossil is serving a single repo, but when serving with --repolist
, I'm tripping an assertion:
~/Fossils % fossil ui --repolist .
Listening for HTTP requests on TCP port 8080
From another terminal:
~ % curl -i http://localhost:8080/jasmine/json/timeline/checkin
curl: (52) Empty reply from server
The fossil ui
process prints:
Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.
I am seeing the same assertion when running Fossil from inetd.conf
like this:
http stream tcp nowait www /usr/local/bin/fossil fossil http --repolist /mnt/fossils
Accessing the JSON API on that server returns the same assertion:
~ % curl -i http://fossil.local/jasmine/json/timeline/checkin
Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.
I'm seeing this behavior in two fossil versions:
This is fossil version 2.13 [e7bba4ff36] 2020-11-01 00:13:51 UTC
This is fossil version 2.12.1 [b98ce23d4f] 2020-08-20 13:27:04 UTC
I'm currently working around this issue by using the /timeline.rss
entry point instead. This doesn't provide a list of changed files in each commit, though.
(2) By Stephan Beal (stephan) on 2021-01-02 00:52:18 in reply to 1 [link] [source]
Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.
That assertion message is very helpful - apparently a code path was neglected when we reworked how the JSON bits are bootstrapped sometime early/mid last year.
i'll take a look at this later on today.
(3) By Stephan Beal (stephan) on 2021-01-02 03:02:33 in reply to 1 [link] [source]
I'm currently working around this issue by using the /timeline.rss entry point instead
A better workaround for your case is to use fossil server --localauth
instead of fossil ui
because...
This problem was resolved, in an ever-so-slightly different form after a report in /forumpost/e4953666d6:
https://fossil-scm.org/fossil/info/dd490d17bec777c4
It was fixed for the server
and cgi
commands, but not the ui
command (which is just a special case of the server
command).
Your use case is now fixed in the trunk:
https://fossil-scm.org/fossil/info/e0b51eb2e76b31fe
So ui --repolist ...
will work with JSON mode starting with that version.
Please let us know if that does not resolve the problem for you. It "works for me":
# Before fix:
[stephan@nuc:~/fossil/fossil]$ curl -i http://localhost:8080/fossil-skins/json/timeline/checkin?n=2
curl: (52) Empty reply from server
# After:
[stephan@nuc:~/fossil/fossil]$ curl -i http://localhost:8080/fossil-skins/json/timeline/checkin?n=2
HTTP/1.0 200 OK
Date: Sat, 2 Jan 2021 02:52:59 +0000
Connection: close
X-UA-Compatible: IE=edge
Cache-control: no-cache
X-Frame-Options: SAMEORIGIN
Content-Type: application/json; charset=utf-8; charset=utf-8
Content-Length: 3722
{"fossil":"67d79d23e1929ee60c6b0668 ...
(Hmmmm. charset is getting duplicated on the Content-Type header.)
(4) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 05:20:54 in reply to 3 [link] [source]
Thanks for looking at this so quickly! I'll try and build from sources tomorrow.
BTW, did you also check fossil http --repolist
? That's my real use case as it happens.
(5) By Stephan Beal (stephan) on 2021-01-02 05:26:38 in reply to 4 [link] [source]
BTW, did you also check fossil http --repolist? That's my real use case as it happens.
i had not considered that one, but just checked that addition in. That command is not really intended for use by clients - it's an internal thing which is used by, e.g., ssh connections.
(6) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 05:39:27 in reply to 5 [link] [source]
Thanks! I must have gotten that command from the "running under inetd" documentation.
From looking at the function you fixed, I suspect that there may also be trouble with fossil all server
and fossil all ui
. I don't have a need for these commands, though, and certainly not their JSON API.
(7) By Stephan Beal (stephan) on 2021-01-02 05:48:34 in reply to 6 [link] [source]
I must have gotten that command from the "running under inetd" documentation.
It's not outright forbidden, it's just not "really" there for general client use.
From looking at the function you fixed, I suspect that there may also be trouble with fossil all server and fossil all ui.
fossil all
launches a new instance of fossil for each repository, stripping off the "all" part of the command, so those should be okay:
https://fossil-scm.org/fossil/file?ci=9d53f2f6a22037e4&name=src%2Fallrepo.c&ln=424-425
(9) By Andy Bradford (andybradford) on 2021-01-02 16:29:33 in reply to 6 [link] [source]
> I must have gotten that command from the "running under inetd" > documentation. And this is correct. "fossil http" is used by inetd configurations et al. I have mine setup with tcpserver and daemontools like this: #!/bin/sh exec 2>&1 exec softlimit -m20000000 tcpserver \ -vDRHl0 192.168.2.3 1951 chroot -u _fossil /var/fossil /usr/local/sbin/stunnel /etc/fossil/ssl.conf And in my ssl.conf: # cat ssl.conf #verify = 3 verify = 1 CApath = /etc/fossil/certs cert = /etc/fossil/ssl.pem exec = /usr/local/bin/fossil execargs = fossil http --https /fossil-ssl foreground = yes syslog = no debug = 4 Notice that execargs calls "fossil http" here which is precisely what is needed for this kind of configuration. Andy
(8.3) By Warren Young (wyoung) on 2021-01-02 15:51:11 edited from 8.2 in reply to 5 [link] [source]
it's an internal thing which is used by, e.g., ssh connections.
For ssh://
URLs at least, you're thinking of test-http
.
When you want Fossil's RBAC system to come into effect, as it does not with test-http
, then you want fossil http
, as in socket activation schemes like my macOS launchd and Linux systemd configs. I got that from drh's inetd
config which uses this method for the same reason I did in those articles. I expect the OP got it from the same place.
So yeah, the JSON API does legitimately need to work for fossil http
.
(10) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 16:57:42 in reply to 5 [link] [source]
Okay, now I've checked out the trunk sources which gave me Fossil 2.14 [49f68be83b] 2021-01-02 13:39:46
. I tested the JSON API with --repolist
, and I can confirm that it works with:
fossil server
fossil ui
fossil http
Thanks again for fixing this so quickly!
While I was at it, I also tested the JSON API with these commands:
fossil all server --localauth
fossil all ui
These are still tripping the same assertion, both with the released 2.13 and the newly built version with the fix. Again, I don't have a use for the JSON API with these commands, so this is not a problem for me.
(11) By Stephan Beal (stephan) on 2021-01-02 17:21:47 in reply to 10 [link] [source]
These are still tripping the same assertion, both with the released 2.13 and the newly built version with the fix. Again, I don't have a use for the JSON API with these commands, so this is not a problem for me.
That's unexpected. It would be impossible to make reliable use of the JSON API with those commands because of the unspecified execution order leading to unpredictable port numbers, so that's a low-priority bug but it's a bug nonetheless.
@Warren: indeed, i confused it with test-http. Thank you for the correction.
(12) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 18:09:12 in reply to 11 [link] [source]
I wonder if you are remembering a previous version of these commands?
When I run fossil all ui
, I get a single directory page at localhost:8080
containing links like these:
<a href='/Users/jolesen/Fossils/config/home' target='_blank'>/Users/jolesen/Fossils/config.fossil</a>
<a href='/Users/jolesen/Fossils/fossil/home' target='_blank'>/Users/jolesen/Fossils/fossil.fossil</a>
Each repo is served from port 8080, but with the full absolute path to the repo in the URL, for example http://localhost:8080/Users/jolesen/Fossils/config/timeline
. The regex in json_request_is_json_api()
only strips one leading directory from the URL path, which is why I suspected trouble. (I don't really understand how any of this code works, though).
(13) By Stephan Beal (stephan) on 2021-01-02 18:42:22 in reply to 12 [link] [source]
I wonder if you are remembering a previous version of these commands?
i've never once (in 13 years) had a user for/used "all" - i was working only from a cursory glance at the code. Indeed, ui/server are handled as special cases there. Sigh.
The regex in json_request_is_json_api() only strips one leading directory from the URL path, which is why I suspected trouble. (I don't really understand how any of this code works, though).
The resulting path for those can be arbitrarily long, meaning the easy fix for it is to have the regex assume that any number of directories before /json
are legal, but that's really only the case for "all server/ui". Hmmm. That level of laxness in the regex doesn't sit well with me, though, so i'll need to mull it over before fixing that.
Thank you for point that out.
(14) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 19:56:59 in reply to 13 [link] [source]
That level of laxness in the regex doesn't sit well with me, though, so i'll need to mull it over before fixing that.
I agree. Someone named "Jimmy Son" might keep all their repos in /home/json/fossils/*.fossil
. They would see HTML URLs like http://localhost:8080/home/json/config/timeline
when running fossil all ui
.
(15.1) By S. Ross Gohlke (rossgohlke) on 2023-02-17 16:30:39 edited from 15.0 in reply to 13 [link] [source]
Did anyone ever mull a solution?
I have a collection or repositories with a hierarchical depth of up to 6 directories and it would be nice to access /json
on all of them, not just those at the top level.
They are being served by a CGI script running under lighttpd, and everything besides /json
works fine.
I have also tested with "fossil server".
In all cases, I am getting the same error.
Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2311.
The funny thing is, I could swear this worked in an earlier incarnation.
Also, I never knew about "fossil all server", thanks for that tip.
(16) By Stephan Beal (stephan) on 2023-02-17 16:30:11 in reply to 15.0 [link] [source]
Did anyone ever mull a solution?
Nope, following the mantra of "it's not a problem until it's a problem." ;)
I have a collection or repositories with a hierarchical depth of up to 6 directories
My gut-instinct reactions to that are (A) "don't do that" and (B) use one CGI script per repository (where it's not a problem).
and it would be nice to access /json on all of them, not just those at the top level.
Unfortunately, the multi-repository approach to serving, in particular with directory depths greater than 1, is "simply problematic" with regards to the determination of whether or not the current path belongs to the JSON API. This is a problem because, for purposes of handling the GET/POST input, we have to know early on whether or not this is a JSON-centric request, and that handling has to be done before the dispatcher has determined, with certainty, that this is a /json path. Any attempts made up until that point are guesses based on the full path, and that breaks down when one fossil instance serves multiple repositories. It's possible for either end of a given path to contain the string "/json", either before or after a "real" "/json" path: /foo/json/myrepo/json/foo/json
. Until the dispatcher has done it's work, we don't reliably know for sure which of those 3 /json
parts (if any!) are intended to point to a /json
API endpoint.
i can't currently offer a reliable solution to this beyond using a separate CGI script per repository. When served that way, all path resolution is unambiguous.
(17.1) By S. Ross Gohlke (rossgohlke) on 2023-02-17 16:48:00 edited from 17.0 in reply to 16 [link] [source]
Thanks for the detailed response.
This is a problem because, for purposes of handling the GET/POST input, we have to know early on whether or not this is a JSON-centric request
I guess this explains why non-json requests might work where json requests do not, because of how early the json handling branches.
using a separate CGI script per repository. When served that way, all path resolution is unambiguous.
As I understand this thread, "fossil all server" was investigated and found lacking as a shortcut to the script-per-repository approach.
(18) By Stephan Beal (stephan) on 2023-02-17 16:55:50 in reply to 17.1 [link] [source]
I guess this explains why non-json requests might work where json requests do not, because of how early the json handling branches.
Exactly. If we knew for certain, at the time the GET/POST bits are handled, how the dispatcher would answer, it wouldn't be an issue. It's an order-of-operations problem with no straightforward fix.
As I understand this thread, "fossil all server" was investigated and found lacking as a shortcut to the script-per-repository approach.
AFAIK, "all server" is going to have a similar limitation, but i've never been a fan of the multiple-repos-per-instance serving modes and don't use them so cannot say for certain off hand.