Fossil Forum

Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!") with --repolist
Login

Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!") with --repolist

(1) By Jakob Stoklund Olesen (stoklund) on 2021-01-01 18:29:52 [source]

Hi,

I'm developing a plugin for the buildbot CI framework, and I'd like to use Fossil's JSON API when polling for new commits to test.

The JSON API works great from the command line and when fossil is serving a single repo, but when serving with --repolist, I'm tripping an assertion:

~/Fossils % fossil ui --repolist .
Listening for HTTP requests on TCP port 8080

From another terminal:

~ % curl -i http://localhost:8080/jasmine/json/timeline/checkin
curl: (52) Empty reply from server

The fossil ui process prints:

Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.

I am seeing the same assertion when running Fossil from inetd.conf like this:

http stream tcp nowait www  /usr/local/bin/fossil fossil http --repolist /mnt/fossils

Accessing the JSON API on that server returns the same assertion:

~ % curl -i http://fossil.local/jasmine/json/timeline/checkin
Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.

I'm seeing this behavior in two fossil versions:

This is fossil version 2.13 [e7bba4ff36] 2020-11-01 00:13:51 UTC
This is fossil version 2.12.1 [b98ce23d4f] 2020-08-20 13:27:04 UTC

I'm currently working around this issue by using the /timeline.rss entry point instead. This doesn't provide a list of changed files in each commit, though.

(2) By Stephan Beal (stephan) on 2021-01-02 00:52:18 in reply to 1 [link] [source]

Assertion failed: (g.json.gc.a && "json_bootstrap_early() was not called!"), function json_page_top, file ./src/json.c, line 2316.

That assertion message is very helpful - apparently a code path was neglected when we reworked how the JSON bits are bootstrapped sometime early/mid last year.

i'll take a look at this later on today.

(3) By Stephan Beal (stephan) on 2021-01-02 03:02:33 in reply to 1 [link] [source]

I'm currently working around this issue by using the /timeline.rss entry point instead

A better workaround for your case is to use fossil server --localauth instead of fossil ui because...

This problem was resolved, in an ever-so-slightly different form after a report in /forumpost/e4953666d6:

https://fossil-scm.org/fossil/info/dd490d17bec777c4

It was fixed for the server and cgi commands, but not the ui command (which is just a special case of the server command).

Your use case is now fixed in the trunk:

https://fossil-scm.org/fossil/info/e0b51eb2e76b31fe

So ui --repolist ... will work with JSON mode starting with that version.

Please let us know if that does not resolve the problem for you. It "works for me":

# Before fix:
[stephan@nuc:~/fossil/fossil]$ curl -i http://localhost:8080/fossil-skins/json/timeline/checkin?n=2
curl: (52) Empty reply from server


# After:
[stephan@nuc:~/fossil/fossil]$ curl -i http://localhost:8080/fossil-skins/json/timeline/checkin?n=2
HTTP/1.0 200 OK
Date: Sat, 2 Jan 2021 02:52:59 +0000
Connection: close
X-UA-Compatible: IE=edge
Cache-control: no-cache
X-Frame-Options: SAMEORIGIN
Content-Type: application/json; charset=utf-8; charset=utf-8
Content-Length: 3722

{"fossil":"67d79d23e1929ee60c6b0668 ...

(Hmmmm. charset is getting duplicated on the Content-Type header.)

(4) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 05:20:54 in reply to 3 [link] [source]

Thanks for looking at this so quickly! I'll try and build from sources tomorrow.

BTW, did you also check fossil http --repolist? That's my real use case as it happens.

(5) By Stephan Beal (stephan) on 2021-01-02 05:26:38 in reply to 4 [link] [source]

BTW, did you also check fossil http --repolist? That's my real use case as it happens.

i had not considered that one, but just checked that addition in. That command is not really intended for use by clients - it's an internal thing which is used by, e.g., ssh connections.

(6) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 05:39:27 in reply to 5 [link] [source]

Thanks! I must have gotten that command from the "running under inetd" documentation.

From looking at the function you fixed, I suspect that there may also be trouble with fossil all server and fossil all ui. I don't have a need for these commands, though, and certainly not their JSON API.

(7) By Stephan Beal (stephan) on 2021-01-02 05:48:34 in reply to 6 [link] [source]

I must have gotten that command from the "running under inetd" documentation.

It's not outright forbidden, it's just not "really" there for general client use.

From looking at the function you fixed, I suspect that there may also be trouble with fossil all server and fossil all ui.

fossil all launches a new instance of fossil for each repository, stripping off the "all" part of the command, so those should be okay:

https://fossil-scm.org/fossil/file?ci=9d53f2f6a22037e4&name=src%2Fallrepo.c&ln=424-425

(9) By Andy Bradford (andybradford) on 2021-01-02 16:29:33 in reply to 6 [link] [source]

> I  must  have gotten  that  command  from  the "running  under  inetd"
> documentation.

And this  is correct. "fossil http"  is used by inetd  configurations et
al. I have mine setup with tcpserver and daemontools like this:

#!/bin/sh
exec 2>&1
exec softlimit -m20000000 tcpserver \
  -vDRHl0 192.168.2.3 1951 chroot -u _fossil /var/fossil /usr/local/sbin/stunnel /etc/fossil/ssl.conf

And in my ssl.conf:

# cat ssl.conf 
#verify = 3
verify = 1
CApath = /etc/fossil/certs
cert = /etc/fossil/ssl.pem
exec = /usr/local/bin/fossil
execargs = fossil http --https /fossil-ssl
foreground = yes
syslog = no
debug = 4

Notice that execargs calls "fossil http" here which is precisely what is
needed for this kind of configuration.

Andy

(8.3) By Warren Young (wyoung) on 2021-01-02 15:51:11 edited from 8.2 in reply to 5 [link] [source]

it's an internal thing which is used by, e.g., ssh connections.

For ssh:// URLs at least, you're thinking of test-http.

When you want Fossil's RBAC system to come into effect, as it does not with test-http, then you want fossil http, as in socket activation schemes like my macOS launchd and Linux systemd configs. I got that from drh's inetd config which uses this method for the same reason I did in those articles. I expect the OP got it from the same place.

So yeah, the JSON API does legitimately need to work for fossil http.

(10) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 16:57:42 in reply to 5 [link] [source]

Okay, now I've checked out the trunk sources which gave me Fossil 2.14 [49f68be83b] 2021-01-02 13:39:46. I tested the JSON API with --repolist, and I can confirm that it works with:

fossil server
fossil ui
fossil http

Thanks again for fixing this so quickly!

While I was at it, I also tested the JSON API with these commands:

fossil all server --localauth
fossil all ui

These are still tripping the same assertion, both with the released 2.13 and the newly built version with the fix. Again, I don't have a use for the JSON API with these commands, so this is not a problem for me.

(11) By Stephan Beal (stephan) on 2021-01-02 17:21:47 in reply to 10 [link] [source]

These are still tripping the same assertion, both with the released 2.13 and the newly built version with the fix. Again, I don't have a use for the JSON API with these commands, so this is not a problem for me.

That's unexpected. It would be impossible to make reliable use of the JSON API with those commands because of the unspecified execution order leading to unpredictable port numbers, so that's a low-priority bug but it's a bug nonetheless.

@Warren: indeed, i confused it with test-http. Thank you for the correction.

(12) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 18:09:12 in reply to 11 [link] [source]

I wonder if you are remembering a previous version of these commands?

When I run fossil all ui, I get a single directory page at localhost:8080 containing links like these:

<a href='/Users/jolesen/Fossils/config/home' target='_blank'>/Users/jolesen/Fossils/config.fossil</a>
<a href='/Users/jolesen/Fossils/fossil/home' target='_blank'>/Users/jolesen/Fossils/fossil.fossil</a>

Each repo is served from port 8080, but with the full absolute path to the repo in the URL, for example http://localhost:8080/Users/jolesen/Fossils/config/timeline. The regex in json_request_is_json_api() only strips one leading directory from the URL path, which is why I suspected trouble. (I don't really understand how any of this code works, though).

(13) By Stephan Beal (stephan) on 2021-01-02 18:42:22 in reply to 12 [link] [source]

I wonder if you are remembering a previous version of these commands?

i've never once (in 13 years) had a user for/used "all" - i was working only from a cursory glance at the code. Indeed, ui/server are handled as special cases there. Sigh.

The regex in json_request_is_json_api() only strips one leading directory from the URL path, which is why I suspected trouble. (I don't really understand how any of this code works, though).

The resulting path for those can be arbitrarily long, meaning the easy fix for it is to have the regex assume that any number of directories before /json are legal, but that's really only the case for "all server/ui". Hmmm. That level of laxness in the regex doesn't sit well with me, though, so i'll need to mull it over before fixing that.

Thank you for point that out.

(14) By Jakob Stoklund Olesen (stoklund) on 2021-01-02 19:56:59 in reply to 13 [link] [source]

That level of laxness in the regex doesn't sit well with me, though, so i'll need to mull it over before fixing that.

I agree. Someone named "Jimmy Son" might keep all their repos in /home/json/fossils/*.fossil. They would see HTML URLs like http://localhost:8080/home/json/config/timeline when running fossil all ui.