Currently, TH1 can be used for ticket customisation, but only for the creation, viewing and editing of tickets and for the listing of ticket reports. What I am missing is using TH1 for the ticket report itself. Currently, the report is defined as a single SQL query resulting in a table. For my use-case, I need something more sophisticated, essentially allowing to me to run more than a single SQL query and inserting custom HTML around those two calls.

In essence, I need a report that can show some introductory text explaining the report content, then showing a set of tickets including the comments (from table ticketchng), then some other set of tickets.

I took a look at the existing ticket C code and it doesn't seem to be too complicated but I do not have the time to implement that myself right now. So, I looked for other ways to achieve my goal and stumbled over --with-th1-docs. Would I be able to produce such pages using the embedded docs feature and using the corresponding TH1 commands there?

Failing that, I wonder whether I can cheat pikchr to execute the needed TH1 code for me. The documentation says I can embed TH1 code into a pikchr script and these pikchr scripts can be used in many places already (including the wiki). However, from this post, it seems that injecting TH1 into pikchr is only possible from the command line interface (CLI), not from within a wiki page. Is that correct?

Any hints as to achieve my goal is very welcome.

Regards, Torsten

So, I looked for other ways to achieve my goal and stumbled over --with-th1-docs. Would I be able to produce such pages using the embedded docs feature and using the corresponding TH1 commands there?

Very possibly. th1 docs are full-fledged th1 scripts and they can perform the types of queries you need. However, my experience with th1-docs is zero, so i can't offer any concrete advice on that topic.

Failing that, I wonder whether I can cheat pikchr ... However, from this post, it seems that injecting TH1 into pikchr is only possible from the command line interface (CLI), not from within a wiki page. Is that correct?

Correct. pikchrs themselves can't embed th1. The pikchr CLI command has an option to run its input through th1 before passing it off to pikchr, which isn't quite the same thing and it's not enabled in any web pages, only the CLI.

Any hints as to achieve my goal is very welcome.

Other than using th1-docs or adding new C code, nothing comes to mind.

OK, I see. Now for the pratical bit. I did compile Fossil with --with-th1-docs and enabled this feature in the settings. Then I commited a file, let's call it protocol.md to the top-level directory of the source tree.

As you can guess, the file is written in markdown. This is how it looks like:

# Aktuelles Protokoll

Dies ist der aktuelle Stand:

<th1>query "select * from ticket" {puts $comment}</th1>

> Allgemeines: bla bla ...

und dann noch:

<th1>foreach i {1 2 3 4} {puts "*$i* $i "}</th1>

So, we have some text and two places with TH1 code in the <th1> tag.

I then call that page with http:/localhost:8080/doc/trunk/protocol.md but this does not seem to work. I can see the TH1 code instead of the result of the query (from the first script) and some numbers (from the second script). Reading the Fossil documentation on embedded docs (section 2.3) I see that TH1 scripts should be enclosed in curly braces. But replacing <th1> with curly braces doesn'T change the output much, the scripts are still not evaluated.

I then tried the same with a protocol.html file and HTML tags (and also with curly braces). Same result.

So, what am I missing? What does it take to actually execute TH1 scripts in (preferably) markdown /doc pages.

Did you turn on the th1-docs setting?

Yes

It seems that indeed there are problems with rendering of both *.th1 files and the rendering of *.md documents that include <th1>...</th1> blocks.
This repository should demonstrate the problem (see "Pending issues" section).

My previous post is wrong.

1. Fossil's documentation does not claim that Markdown processor should interpret the content of <th1>...</th1> blocks via TH1.

2. Rendering of *.th1 files via /doc pages works fine.
   It turns out that only the content inside <th1>...</th1> blocks is subject for interpretation via TH1.

Hm,

interesting. I wii then give *.th1files a try.

Rereading closely the documentation on /docfiles, you may be right on markdown not supporting TH1. Nonetheless, the documentation on this is quite blurry and my interpretation was a different one. Section 1 of thedocumentation says that these embedded docs can be HTML, wiki and markdown. Section 2 says that Fossil can do some substitution of server-side information into the embedded document. It doesn't say explicitly that it is limited to HTML. Section 2.3 goes on to tell how TH1 expressions can be used in HTML, again without explicitly saying that this is not possible in wiki/md. – So, yes, it does not say that TH1 is interpreted in md documents but it doesn't say the opposite either.

That having said, I would find it very useful to have TH1 in markdown also, as suggested here.

There is no convincing reason why TH1 should be allowed in HTML only (when Fossil is compiled with --with-th1-docs) but not in markdown or wiki format when used as /doc files. Or is there?

Silly thought, but I wonder if wiki in this context means fossil wiki format and not markdown wiki format. The fossil wiki format predates markdown by quite a while. Maybe the docs haven't been updated since then?

There is no convincing reason why TH1 should be allowed in HTML only (when Fossil is compiled with --with-th1-docs) but not in markdown or wiki format when used as /doc files. Or is there?

The ability to run arbitrary th1 in *.th1 is a separate feature from embedding th1 in arbitrary wiki/embedded docs. Being able to run arbitrary server-side script code in any document is a security issue just waiting to happen. Remember that, with the proper configuration options, th1 can run full-fledged TCL code, giving the docs access to any features TCL supports. If any page can run any TCL, a malicious user who has checkin access to your repository could check in server-side script code to any embedded docs files, likely without it being scrutinized closely because it's "just a document change." Such changes could be "buried" within many other plain-text changes to help hide them in the "noise" of the checkin.

If your documentation requires th1 in order to display properly, you are effectively requiring that anyone who clones your repository enable those inherently unsafe features. (Noting that th1 itself is believed to be safe for this purpose, but its ability to run arbitrary TCL code is inherently risky.) Since such documents would be able to run arbitrary TCL code (via th1), you are effectively asking anyone who clones your repository to open up their system to TCL code your repository may provide and execute, and to trust that you will not do anything malicious with that. That's a huge security hole.

Yes, many very interesting things could be done by permitting server-side script code in arbitrary documents, but doing so is dangerous for anyone who clones that repository. There are lots of languages/environments far better-suited to that and many of them can be embedded behind fossil via the /ext external CGI support.

Thanks for the additional warning. My case is very special. I am trying to use the ticket system as a means to record discussions and decisions made in company meetings. Each topic is a ticket that can have metadata, an initial (editable) description and a row of comments (via ticketchng). I need to make the thing look like the current Word file used for these meetings, in order to convince using this new way of recording meeting discussions.

The Fossil ticket system is 99% useable for this ... the only exception being that I cannot produce custom ticket reports as the ticket report in Fossil is just one single SQL query (see my original post). Now, as I can't readily change that, I want to produce this report using TH1 in embedded /doc files.

You see, while I can see the possible security risk, in practice there is none. The repository will never be cloned, there will never be commits (unless the reports are done via committed /doc files), the only is myself, the other people are only watching watching what I type and can have a copy of the report later (as HTML or converted to PDF).

Let's see whether I can do it with *.th1 files, otherwise I use the patch in this post (which I still find very useful to have in Fossil by default for non-wiki pages) or external CGI support.

ticket system is 99% useable ... I cannot produce custom ticket reports
...
Let's see whether I can do it with *.th1 files...

Could you please share how did it go: was th1-docs helpful for your case?

Hi,

yes, I ended up being able to do what I intended with th1-docs. I use a custom fossil compiled with th1-docs enabled and have written an custom *.th1 page consisting of some html combined with some sql querying and displaying information from the ticket and ticketchng tables.

I then installed this page as fossil's home page so people would see this as the first thing when going on the site.

There are edit links on each topic so you could go and then edit the ticket directly from there (much like it is done on the original ticket page.

What about tclMakeSafe?

# Add this to th-setup
proc webpage_hook {} {
  # Use full TCL in TH1 before serving content.
  tclMakeSafe
  # Contributed TH1 gets a stripped-down interpreter.
  continue
}

You could even apply it conditionally based on the top level page.

Would I be able to produce such pages using the embedded docs feature and using the corresponding TH1 commands there?

Likely so. But take into account that TH1 query command is subject to the same data access restrictions as the report queries.

If you don't need instant portability of your repository then CGI Server Extensions may also be a solution. That is a notable if though...

Well, of course, if certain queries don't work then I need to figure out a different way. Following your link I can see that both "ticket" and "ticketchng" are among the allowed tables. Also, I would expect an error message when trying to access a table that is restricted. I also do not see that.

Further, shouldn't the second TH1 script work anyway, then? As neither of them are executed, I think the problem is not with data access restructions.

Thanks for the hint with CGI extensions. That could be a last resort!