Official binaries with support for TH1 docs and JSON API?
(1) By george on 2022-05-10 17:39:20 [link] [source]
Fossil supports embedded TH1 docs only
if it is compiled with the corresponding options
(see check-in 516cc0c91f7f82d1 from 2014).
JSON API is another
optional feature.
For the time being official Fossil binaries
are compiled
without support for these two features.
A more feature-full official binaries might be needed if
- a project requires that all participants use their local Fossil executable, but
- some participants lack expertise to compile Fossil for themselves
and also are unwilling to run non-official binaries.
For TH1 scripting I know
a workaround.
And I don't use JSON API myself yet
(but I have an idea that would require ability
to manipulate tags from within JavaScript).
I would appreciate feedback from the community on whether the issue seems real or imaginary.
Richard, could you please comment on whether the reasons for NOT including support for TH1 docs and JSON API in the official builds still hold? And if they don't, then what is the prospect for providing official binaries that support these features?
(2) By Richard Hipp (drh) on 2022-05-10 18:04:52 in reply to 1 [link] [source]
Yes - the reasons for not including TH1 in docs still hold. Allowing TH1 in documentation allows anybody who is able to create or edit a document (such as posting to this forum) to run arbitrary code on the server. It is a built-in RCE vulnerability.
I have disapproved of TH1 in docs from its very beginning. I allow it as a compile-time option only to appease those who want it, and understand the risks.
I will not encourage the use of TH1 in documentation by making precompiled binaries for it available. If someone does not have the skills needed to compile Fossil for themselves, then they probably don't have the skills needed to use TH1 in documents safely either.
(3) By Stephan Beal (stephan) on 2022-05-10 18:20:25 in reply to 1 [source]
For the time being official Fossil binaries are compiled without support for these two features.
th1-docs: it seems unlikely that we'll ever make official releases which enable "potentially dangerous" features. TH1 is exceptionally well sandboxed, and is not dangerous in and of itself, but it enables features which other users have to enable if they want to clone and make use of a repository which uses those features. Though i won't claim to have a controlling stake in any related decision, that "viral" aspect of the th1-docs feature does not sit well with me and i'd certainly moan a bit (but not overly) about enabling it by default. (For completeness's sake: i was a proponent of the feature early on, but was younger and naiver about the wider implications of adding general-purpose server-side scripting to fossil. Yes, it's useful, but it's also a two-edged sword with a "very non-zero chance" of security implications.)
JSON: (sidebar: for those who don't know, i'm the primary architect of those bits) Richard told me long ago (late 2011, maybe 2012) that JSON would remain defaulted to off at least until it was well fuzz-tested to ensure that it poses no threat when fed malformed or malicious inputs. That it has never been fuzz-tested is entirely my fault. Only in the past month did i learn that it's actually pretty easy to do and hooking up the JSON API to fossil's fuzz test infrastructure is most definitely on my TODO list, but i cannot commit (as it were) to any specific timeline for it. Just because it's fuzz-tested is not a guaranty that it will be enabled by default, but that is most definitely a blocking criteria for reaching that point.
At the end of the day, though, i always build my own fossil binary so don't concern myself too much with what's enabled or disabled by default.
I would appreciate feedback from the community on whether the issue seems real or imaginary.
IMO, entirely imaginary. We always, without exception, recommend that users build fossil from the trunk version and that they configure it to suit their needs. "Official" binaries are provided primarily because, for whatever reason, a large portion of people want an "official" release of all software they use. Shrug. It seems perfectly sensible to me that any and all such prebuilt binaries use conservative settings, e.g. no th1-docs and no JSON until we have a high level of confidence that the JSON interface cannot become an attack vector. Users who want corner-case or "potentially unsafe" features should (IMO) be required to take the steps themselves to create such a binary.
Fossil is, after all, a software development tool and its primary target users are software developers. The features mentioned above are not ones non-developers can sensibly make use of, so there's no compelling reason (IMO) to enable them in prebuilt binaries.
$0.02 and all that.
(4) By Stephan Beal (stephan) on 2022-05-10 18:39:39 in reply to 3 [link] [source]
Richard told me long ago (late 2011, maybe 2012) that JSON would remain defaulted to off at least until it was well fuzz-tested to ensure that it poses no threat when fed malformed or malicious inputs.
Sidebar: fossil's JSON bits were added long before sqlite had native support for JSON. Now that sqlite can process JSON, it would be technically feasible (and perhaps even advantageous) to reimplement fossil's JSON API on top of that, as opposed to using the 3rd-party JSON library it uses now (noting that that "3rd party" is me ;)). Such a reimplementation would, in and of itself, go a long way to ensuring that the software is well-behaved in the face of malicious JSON inputs by virtue of sqlite itself being continually fuzz-tested. "The problem," though, is that it would require a great deal of rewriting. If someone listening in is interested in taking that on, i'm certainly willing to help, but doing the rewrite myself is not currently on my radar. It would certainly require more than a weekend or three of hacking and it probably cannot reasonably be done incrementally, one JSON interface at a time, because the two approaches to handling the inbound and outbound JSON are mutually exclusive.
Volunteers are, however, encouraged to get in touch.