Is there a way I can use the fossil json command to obtain the same information I'd get from fossil settings but in JSON format?

I need to automate a certain number of operations and I need to get the value of certain flags beforehand and restore them at the end; and I cannot do it reliably by parsing the output of fossil settings.

If it isn't possible, could it be? (*blush*)

Is there a way I can use the fossil json command to obtain the same information I'd get from fossil settings but in JSON format?

IIRC, this wil do it, but am currently on a tablet so can't readily verify that that outputs that same collection of settings.

IIRC, this wil do it, but am currently on a tablet so can't readily verify that that outputs that same collection of settings.

Nope, those are the "config" settings, which are distinct from the "settings" settings. The JSON API does not currently have any routines for the "settings". Yes, it's possible to add them, and it doesn't look like it would be much work, the caveat being that they would all be emitted as strings instead of their "native" types. That's now on my TODO list, but i cannot predict when you'll have it.

Sidebar: the json API was added before sqlite had built-in json support. "It would be cool" to see someone go reimplement it, or implement json API v2, using sqlite's built-in support. Patches to that effect would be gleefully received!

If it isn't possible, could it be?

Would something like this serve your needs?

        "payload":[
                {
                        "name":"access-log",
                        "versionable":false,
                        "sensitive":false,
                        "defaultValue":"off",
                        "valueSource":null,
                        "value":null
                },
                {
                        "name":"admin-log",
                        "versionable":false,
                        "sensitive":false,
                        "defaultValue":"off",
                        "valueSource":"repo",
                        "value":"1"
                },
   ...

The fields are:

• name, defaultValue, value: self-explanatory
• sensitive: a value which fossil has flagged as sensitive can only be fetched by a Setup user.
• valueSource: one of ("checkout", "repo", "versioned", null). If a setting is set in the checkout, its value is preferred. The global settings are not queried for this purpose because that seems likely to cause confusion and/or unwanted data exfiltration. A value of null means the property is not explicitly set in either the checkout or the repo.
• versionable: boolean true the setting is tagged as versionable, else false. If a checkout is open and .fossil-settings/SETTING-NAME exists, it will pull the value from there and valueSource will be "versioned". We do not (yet?) pull a versioned value from the repository because to do so we have to make an assumption about which version to pull it from or require the user to provide it.

This is still very much a prototype and open for discussion.

Edit: the code is currently in the json-settings-command branch

The global settings are not queried for this purpose because that seems likely to cause confusion and/or unwanted data exfiltration.

Hum. This could be a problem. Let's say I need to change the autosync setting. I may have set the value to 1 globally but I need to set it to 0 for an automated operation which implies several commits, and I don't want to sync each of them separately. Then I would need to restore it to its previous value. If it's global, I need to unset it. If it's local, it would need to be set back to the old value.

I think a global value should be returned and the valueSource key set to "global". In what situation(s) you envision this being a problem?

I may have set the value to 1 globally but I need to set it to 0 for an automated operation which implies several commits, and I don't want to sync each of them separately.

The JSON API will not support setting anything for the near-term future. That's a much larger operation than the hour or so of code it took to do fetching, with much more room for backfiring on users.

Consider this: user X is a setup user of repository Y. If the settings API enabled him to modify global-config settings, he could, via repository Y, change settings relevant to an arbitrary number of repositories to which he otherwise has no access.

If /json/settings/set support is added, there's no way it will include any option to set global settings because there's no way to keep such a scenario from happening. Only CLI-level access can change global settings.

I think a global value should be returned and the valueSource key set to "global". In what situation(s) you envision this being a problem?

The JSON API is only addressed through specific repository instances and is very much intended to only operate on the addressed instance. It would be not only counter-intuitive for it to return any information from another source, but it could hypothetically be used to exfiltrate (leak) data which users of the specific repository in question are simply not intended to be aware of.

For example, the web-browser setting can contain the path to the local preferred web browser, and that information alone might provide an attacker with a hint about which OS is in use, giving them insight about how to initiate an attack on the system. Does that sound far-fetched? Sure, it does, but clever attackers use all sorts of trivia like that to obtain access to systems.

Global settings access is not going to happen until/unless someone can convincingly demonstrate that such data leakage is harmless. Even for admin users, global-config access gives them info which might not be relevant to their one repo while being relevant for 100 other repos on the same machine.

The JSON API will not support setting anything for the near-term future.

I never requested that -- my intent is to use fossil setting to set/unset the values.

The scenarios you described are enough justification to avoid exposing global settings. And if I get a null value I would know that the value is global or non-existent locally and I have to unset it anyway.

Yeah, the current functionality in the branch should suffice.

The JSON API will not support setting anything for the near-term future.

I never requested that -- my intent is to use fossil setting to set/unset the values.

The scenarios you described are enough justification to avoid exposing global settings. And if I get a null value I would know that the value is global or non-existent locally and I have to unset it anyway.

Yeah, the current functionality in the branch should suffice.

I never requested that -- my intent is to use fossil setting to set/unset the values.

Ah, then i misunderstood. Even so, that misunderstanding gave me food for thought about why it Would Be Bad to permit writing to the global config db, so it wasn't a total loss.

Yeah, the current functionality in the branch should suffice.

There's one more feature i'd like to add before merging it, and that's specifying a checkin version/branch to pull versioned properties from, without requiring a checkout. Right now the versioned values are only pulled if a checkout is open. How best to do that without having to open a whole manifest (an expensive operation) currently eludes me.

There's one more feature i'd like to add before merging it, and that's specifying a checkin version/branch to pull versioned properties from, without requiring a checkout.

That's implemented now: if --version X is supplied, it will first check that check-in version for the any versioned settings, falling back to (1) local checkout files, (2) checkout vvar table, and (3) repository.config table, in that order.

There was a slight change in "sensitive" settings: those are now listed but their value and valueSource are always null for non-setup users (or if they aren't set). Previously they were not even listed for non-setup users.

The output now looks like:

[stephan@nuc:~/f/fossil/src]$ f json settings get -version prev | head -20
{
	"fossil":"632dfd26dda2fa1cf8ac90415c559174df81fef81ba29f3ba723551f0e9af659",
	"timestamp":1674096339,
	"command":"settings/get",
	"procTimeUs":13285,
	"procTimeMs":13,
	"payload":[
		{
			"name":"access-log",
			"versionable":false,
			"sensitive":false,
			"defaultValue":"off",
			"valueSource":null,
			"value":null
		},
		{
			"name":"admin-log",
			"versionable":false,
			"sensitive":false,
			"defaultValue":"off",
...

Noting that the web-access URL is /json/settings/get with an optional ?version=... part. If the given version is not found, an error is triggered:

[stephan@nuc:~/f/fossil/src]$ f json settings get -version nope 
{
	"fossil":"632dfd26dda2fa1cf8ac90415c559174df81fef81ba29f3ba723551f0e9af659",
	"timestamp":1674096398,
	"resultCode":"FOSSIL-3006",
	"resultText":"Cannot find the given version.",
	"command":"settings/get",
	"procTimeUs":1836,
	"procTimeMs":1
}

If the API in this shape would suit your needs, let me know and i'll get it added to the docs and merged.

If you (or anyone reading along) would like to see it reshaped, now is the time to speak up.

There's still no estimate on when/whether /json/settings/set will be implemented. There's no(???) really good argument against adding it, it's just a matter of the development time and energy.

Sidebar...

Note that i opted against the arguably more intuitive output structure of:

payload:{
  "access-log": { ... },
   "admin-log": { ... }
   ...
}

Because the order of properties in JSON objects are not guaranteed to be anything in particular and they may be reordered when passing through different JSON-parsing APIs and programming environments, depending on how JSON objects are actually modeled (e.g. hashtable-based impls have essentially random ordering). The array form is used to keep their order strictly alphabetic and is trivial to transform to the other structure client-side if desired.

I can see the merit in both formats (array and object), to say the truth

In any case, for the currently implemented version, simply using

fossil json settings get | jq -r '.payload[] | select(.name == "autosync") | .value'

is enough to get the value I need for my scripting purposes, so I consider the current version a net positive. Waiting for the final merge :-)

I can see the merit in both formats (array and object), to say the truth

Yeah, i hate to have to decide for one or the other. If you'd prefer the object structure, we can do that instead - it's not a big change, either way.

i'll give it a day or two to let you think it over, then document and merge it (and modify it if we decide to use the object structure instead - my current instinct is to do so, but i've not yet finished my coffee).

If you'd prefer the object structure, we can do that instead - it's not a big change, either way.

After thinking about it more, object structure makes more sense conceptually for me. Thanks!

After thinking about it more, object structure makes more sense conceptually for me.

How does this appeal to you?

  "payload":{
    "access-log":{
      "versionable":false,
      "sensitive":false,
      "defaultValue":"off",
      "valueSource":null,
      "value":null
    },
...
    "binary-glob":{
      "versionable":true,
      "sensitive":false,
      "defaultValue":null,
      "valueSource":"versioned",
      "value":"*.gif\n*.ico\n*.jpg\n*.odp\n*.dia\n*.pdf\n*.png\n*.wav\ncompat/zlib/contrib/blast/test.pk\ncompat/zlib/contrib/dotzlib/DotZLib.chm\ncompat/zlib/contrib/puff/zeros.raw\ncompat/zlib/zlib.3.pdf\nextsrc/pikchr.wasm\n"
    },
...
    "web-browser":{
      "versionable":false,
      "sensitive":true,
      "defaultValue":null,
      "valueSource":"repo",
      "value":"firefox"
    }
}

The JSON API will not support setting anything for the near-term future.

Okay, that was inadvertent misinformation. /json/settings/set accepts a map of settings and values and updates the repository's config with those values (it never updates the global or checkout databases):

$ cat in1.json
{
  "command": "settings/set",
  "payload": {
    "admin-log": false,
    "auto-captcha": null,
    "editor": "emacs"
  }
}
$ ./fossil json --json-input in1.json 

It has no output payload.

A value of null unsets (deletes) the entry from the config (noting that that may cause the value to be inherited from the global db in certain uses). Booleans are stored as 0 or 1. Doubles and strings are stored as-is, and any other type triggers an error (rolling back all changes).

Edit: it requires Setup permissions, of course.

If there are no objections to that interface i'll merge these changes sometime in the next couple of days.

If any changes are desired, chances of having them implemented are much higher if they're requested before it's merged.

Looking good, you can go ahead :)

I just tried compiling the latest version in Windows including these changes, and it fails with this warning which is treated as an error:

src\json.c(2246): warning C4113: 'cson_value *(__cdecl *)()' differs in parameter lists from 'fossil_json_f'

The same warning-turned-error also appears in the same file in lines 2251, 2252, 2256, 2258, 2260, 2261, 2262 and 2266.

In MacOS it compiled without a hitch.

The same warning-turned-error also appears in the same file in lines 2251, 2252, 2256, 2258, 2260, 2261, 2262 and 2266.

That's what i get for doing the right thing and correcting function signatures from:

int foo();

to

int foo(void);

The ones you've reported are now updated. It's very possible that more will follow from other files. Back when most of the json code was written, the distinction between those two declarations was unknown to me, and there may well still be some written in the first form.

I tried with 5dd632 but it still fails with the same error message including all the same line numbers (?)

Now it should work - you had forgotten some of them. Which makes me wonder why your compilers didn't complain... 😉

Probably for want of -Wstrict-prototypes (and -Werror) in the Makefile. In MSVC C4113 is a level 1 warning and we do compile there with /W2 and /WX (the pendant to -Werror).

This final change did it. Thank you Stephan for the implementation and Daniel for the last update to the function declarations!

Fossil now compiles correctly under Windows in my configuration.