Yes, the Schimpf book is still useful, even if outdated.

Although there are docs in the Fossil project repo, and more would be helpful and doubtless welcome, I think there's value in a single linear, coherent presentation, and I'm not seeing an especially good reason to rewrite this book from scratch.

The Schimpf book was last updated in December 2016. Although Fossil has gotten quite a few new features since then, none immediately come to mind that make large parts of this book useless. Even the SHA-3 feature changes only cosmetic elements of the book, mainly in screenshots and Fossil CLI output examples.

Cosmetics are a serious practical problem for the book: pretty much every screen shot needs to be re-taken, and many of the CLI outputs need to be re-captured as well. The default Fossil UI skin has changed, and many of its pages now look substantially different. (e.g. the capability list on Admin → Users.)

I was surprised to see typos and grammar errors still in the text. So, it needs an editing pass as well.

I've got the book repo checked out and the document opened in LyX, so in principle at least, I can fix the "outdated" problem. Can you give me a repo login, drh, or do I need to ask Jim?

If I take this project up, I'd be tempted to re-do it in reStructuredText. That gives us a mostly human-readable plain text file with semantic markup. Markdown is simpler, but it's got problems when used for long documents.

I've got a pretty clean looking conversion to reST, via pandoc, so it's not a major amount of work to reformat it. It does need a cleanup pass, but the screen shots are a far bigger problem.

Maybe we want to move the user manual project into the main Fossil repo?

The only reason I mentioned reStructuredText is because I went looking for an alternative somewhere between Markdown — easy to write, but weak semantic markup support — and DocBook — exactly the inverse — and stumbled upon reST. DocBook is very powerful, and I've successfully used it for production tasks several times, but it's a pain to work with. Inspection of the reStructuredText docs convinced me that it has sufficient semantic markup support for the purposes of this user manual project, so I recommended it instead of DocBook, which I actually had done in early drafts of the above email.

Now that you mention Asciidoc, though, I see that it's not only meant to be semantically equivalent to DocBook, it's actually endorsed by O'Reilly, one of the major driving forces behind DocBook. That's worth something.

I then went searching for comparisons and found the Hyperpolyglot comparison table for lightweight markup languages. Based solely on that, I think I like reST a smidge better, but it's a close thing. Since I found numerous errors in another comparison table on that web site, and my suggested edits were all ignored, I'm not putting too much weight on the opinion I've developed based on it.

To this point, the difference has been nearly equivocal, so I then decided that what would really sway me in my choice was availability of a liberally-licensed C or C++ implementation that we could drop into Fossil, allowing us to support one of these two document formats everywhere Fossil currently supports Markdown and/or Wiki format. That would not only be useful for other end users of Fossil, it would let me check my new manual edition into Fossil and render it directly to HTML on demand, which means I wouldn't have to check in a locally-rendered version as unversioned content or similar.

I went searching and found only one plausible candidate for Asciidoc. It claims to be a "portable version of AsciiDoc" in C++, but this GitHub project is just a Qt GUI which calls out to a bundled version of the Python implementation of Asciidoc. By "portable", they're using the "all files and dependencies install into a single directory you can copy around" definition, not the "builds on multiple platforms without fuss" definition. We could do much the same thing in Fossil, calling out to an external Python helper toolchain to render Asciidoc to HTML, but that would be completely un-Fossil, especially for rendering important checked-in content to HTML.

Then I went looking for reST implementations in C or C++, found one in each language [1] [2], both with BSDish licenses.

I can still be swayed, but that seems like a pretty good argument in favor of reST to me.

it would let me check my new manual edition into Fossil and render it directly to HTML on demand

I've just learned that reST has an include directive, which means we could potentially extract many pieces of this user manual to single-topic documents, then automatically re-insert them into the full user manual. So, rather than link to the full user manual just to refer to a single sub-section, someone could choose to link to just that one section. The resulting HTTP GET would return faster and render faster in the browser.

In effect, it's automatic chunking/de-chunking from a single set of input files.

Coooool!

I then checked whether AsciiDoc can do this, and was surprised to see that it can, since DocBook delegates that to XIncludes, which is technically outside of DocBook.

Hi,

Offray here, but I can not login into the web forum, and there is not any way to reset/remember the password I can find.

I would think that a good format to update the Fossil Book would be Pandoc. It is lightweight and goes beyond the minimal Markdown, to include several customizations (including LaTeX commands) to produce a good looking PDF. We have used such combination (Fossil + Pandoc) to create good looking books, like the Open Sourced Handbook for Data Journalism.

I think that AsciiDoc is a good candidate, but it as showed in the CommonMark Spec it can be difficult to read (despite of being a little bit easier to write). Particularly I would suggest to use a system like CodiMD for real time collaboration on the documentation. It has an impressive user experience, with real time preview and several Markdown extensions (Yaml metadata blocks, admonitions, code snippets. We have an instance of CodiMD hosted in the community infrastructure and during the last quarter of 2018 we will try some integration with Grafoscopio, Fossil and CodiMD. We could try this combinations also with Fossil Book, if there is enough community interest.

Cheers,

Offray

I really would recommend/support to have a serious look at AsciiDoc

I already gave my criteria above and showed how AsciiDoc comes up short. Until you deal with those objections, I'm unlikely to change my stance on this.

AsciiDoctor (it's very good current implemenation)

I fully expect that it's wonderful, but it's written in Ruby, which means we can't build it into Fossil in any practical sense, which means we can't treat AsciiDoc the same as Markdown and Wiki are currently treated within Fossil, which means we can only use it as an external processor. I don't want to have to "compile" the user manual to HTML, PDF, etc. and then upload them somewhere as static documents. I want to check the manual's "source code" into a public Fossil repo and have it render that to HTML on demand. I also want it to do this in both chunked and single-document forms, the latter being the chunks strung together with include directives. And above all, it needs to be in C or C++ so we can statically link it to Fossil, and it must be non-virally licensed.

Any markup language that gives me a way to achieve all of that will win over languages that do not, no matter how nice their syntax or preexisting tooling. So far, reST is the only language I've come across that's achieved all of that.

One option I've considered is using this lack of a liberally-licensed C implementation of an AsciiDoc to HTML translator as an excuse to learn Lemon, which should make it merely tedious to build my own rendering engine, rather than difficult. But, I suspect that would end up not matching your preconception of what it means to "use AsciiDoc." I suspect you're more jazzed by what comes out of the most popular rendering engines and what plugins are available than any special virtues of the markup language itself. That is, a plain-Jane AsciiDoc to HTML translator probably doesn't excite you.

Am I right?

If so, that argues in favor of continuing to pursue this current reST path until and unless I find that it won't work for some reason.

not as beautiful as a book typeset in LaTeX

I don't think this "user manual 2.0" project should prioritize print/PDF output. Although it would be nice to be able to get PDF output from whatever we end up using, there must be at least 20 features I want more than that, and probably a lot more.

This project also doesn't require TeX's famous mathematical typesetting abilities.

It's 2018, and this is Fossil: lightweight-formatted web output that integrates with Fossil UI is more important than niggly typesetting details. (And that's coming from a typography geek!)

We also have no good reason to prioritize tools that generate standalone static web sites, as with Sphinx: I'll happily give up things like an internal JavaScript search engine if it means I get Fossil integration. Fossil's built-in FTS engine should fill that role well enough anyway, especially once I've got the user manual sharded, so that search results are more likely to pull up user manual sections rather than the assembled user manual.

I would think that a good format to update the Fossil Book would be Pandoc

I think you may be confusing Pandoc with its Markdown implementation. Pandoc isn't a document format; Pandoc-flavored-Markdown is.

If you tell me that Pandoc's Markdown implementation is uncommonly powerful, I'm prepared to believe you without proof, since it must deal with all of the major flavors of Markdown, so to the extent that it is not a superset of all of them, that's going to create feature requests that are hard to brush aside.

However, Pandoc has many other problems here:

1. It's far more than just a Markdown-to-whatever translator. Do we want to drag all of that into Fossil?

2. It's written in Haskell. There's a C binding to the Pandoc library, but you must still have a Haskell toolchain or a precompiled Pandoc development package to make use of it if we're going to embed it in Fossil. I suspect that'll make the Windows users rebel right there.

3. It's GPL'd, which pretty much makes it incompatible with Fossil's license, insofar as we don't want Fossil to become GPL-licensed just because we have one feature off on the side here that is GPL.

4. We don't want to just call out to a third party rendering toolchain for the reasons I've given above.

That only leaves us with the option of hand-rendering our documents and shipping HTML and PDF outputs as static content. That goes entirely against the Fossil way of doing things, where marked up documents are rendered on demand to HTML by Fossil itself.

the CommonMark Spec

That settles it: even the most inclusive flavor of Markdown still doesn't have an "include" directive, so you must write the whole document in a single huge file.

Markdown's out.

Particularly I would suggest to use a system like CodiMD for real time collaboration on the documentation.

Fossil already is a collaboration system. And I've already got a Markdown/reST/etc. editor: it's called Vim.

I don't think the lack of a web 3.0 word processor is what lead to the Fossil user manual becoming outdated. I think the primary thing we've lacked is a person with the time, will, and skill to keep it updated. Since I'm volunteering to fill that role, we've got the main problem covered.

Arguments of the form "We can't do X because our tools are too primitive" are almost always bogus, evidenced by the fact that good content was created before all of these tools existed. I've got excellent technical manuals here produced before computer typesetting even existed! Whizzy web apps are nice, but the lack of whizzy tools is an excuse, not a serious blocker.

Who's gonna put pen to paper and make something?

I already volunteered. So far, my spare time's been taken up researching all of the "but what about" objections. :)

For those still not sure what I'm talking about when I refer to include directives, take a look at the top-level DocBook source file for a large user manual I wrote for a different FOSS project. Notice that all it does is set up whole-document metadata and then include the chapter documents. Those documents in turn pull in other sub-documents.

This is the sort of thing we expect from a document preparation system designed to produce book-length outputs.

This is another reason mandoc is unsuitable: man pages were originally meant to be just that: a single manual page. In the early days of Unix, they would aggressively prune a page's content to keep it on a single page, spilling over to multiple pages only when absolutely necessary. That inspired brevity of description and focused designs, which is probably a large part of why Unix succeeded. I once read an opinion piece from one of the creators of Unix (Thompson or Pike, probably) who said that book-length man pages are a sign you've failed somewhere, either by being too verbose or by creating a tool so complex that it can't be described properly in fewer words. I don't entirely believe that, but there's a kernel of truth in there.

Fossil already has the equivalent of man pages: fossil help foo. If you want to argue that you should be able to say man fossil-foo, as you can with Git, that's a separate discussion and a separate project.

I think you may be confusing Pandoc with its Markdown implementation. Pandoc isn't a document format; Pandoc-flavored-Markdown is.

Pandoc is a conversion program that includes a Markdown superset, so yes, when I'm suggesting usin Pandoc, I'm referring to such superset in the previous mail or, more specifically here I am referring to Pandoc, as a core part of a toolset for writing and publishing documents (including the extended markup and conversion tools).

It's far more than just a Markdown-to-whatever translator. Do we want to drag all of that into Fossil?

Do we need to drag X to Fossil because we are using X for documentation? Previous Fossil book was written in LyX , without dragging LyX into Fossil. Anyone can write into Emacs, Vim, Sublime, Atom or whatever without dragging anything of the tool used for writting to the tool that is being writting about.

It's written in Haskell. There's a C binding to the Pandoc library, but you must still have a Haskell toolchain or a precompiled Pandoc development package to make use of it if we're going to embed it in Fossil. I suspect that'll make the Windows users rebel right there.

It is extensible in Lua, which is already included with Pandoc (and pretty easy to learn), but I don't see how the language of the conversion tool, or writing format is affecting this. Again, the previous book was written in LyX, which is a front end for LaTeX, which is a set of macros for TeX. Nothing of this made any platform users rebel against anything in the documentation process.

It's GPL'd, which pretty much makes it incompatible with Fossil's license, insofar as we don't want Fossil to become GPL-licensed just because we have one feature off on the side here that is GPL.

Which is completely unrelated with the tools used for writing documentation. No part of your choice of Vim is argued because of the license behind it.

That only leaves us with the option of hand-rendering our documents and shipping HTML and PDF outputs as static content. That goes entirely against the Fossil way of doing things, where marked up documents are rendered on demand to HTML by Fossil itself

Which clarifies why you are considering and mixing such overcomplicated rationale (including licenses, programming languages, etc) over updating Fossil Book, which,BTW, went entirely against "the Fossil way", being a static PDF with a linear and accesible narrative about learning Fossil directed to non-tekies and non-devs. And hooks can be added so particular changes in the source document will produce new PDF versions automatically.

That settles it: even the most inclusive flavor of Markdown still doesn't have an "include" directive, so you must write the whole document in a single huge file.

That's not true for Pandoc, which understands CommomMark and can produce complete beautiful documents (like books) from small individual files with the desired granularity (sections, subsections, chapters, parts). The open sourced Data Journalism Handbook, linked before, was produced without starting from a huge monolitic file.

Fossil already is a collaboration system. And I've already got a Markdown/reST/etc. editor: it's called Vim

I don't think the lack of a web 3.0 word processor is what lead to the Fossil user manual becoming outdated. I think the primary thing we've lacked is a person with the time, will, and skill to keep it updated. Since I'm volunteering to fill that role, we've got the main problem covered.

Please, don't follow anybody's advice if you are the only one behind something. Use whatever fits the most for your solo endeavors. I'm talking about years experience writing and updating documentation together with communities, in synchronous (real time) and asynchronous ways. We are at opposite sides of the spectrum here and maybe we are going to keep creating and updating the documentation we need anyway, in our local community (including tutorials in Spanish about Fossil for non tekkies), using the workflow and toolset I have suggested for communities, not for individuals. And, with a proper license for the book, we will be able to adapt, remix and translate the parts we may use without major concerns about the formats and toolset of any solo approach, as we already did with the hundreds page long Data Journalism Handbook, wich used a pretty privative toolset for writing, "collaboration" and publishing (but had a pretty open license).

Arguments of the form "We can't do X because our tools are too primitive" are almost always bogus, evidenced by the fact that good content was created before all of these tools existed. I've got excellent technical manuals here produced before computer typesetting even existed! Whizzy web apps are nice, but the lack of whizzy tools is an excuse, not a serious blocker

Fortunately, nobody is advocating Whizzy web apps as a magical non primitive solution for anything here. What we are talking about is the experience (or lack of it) employing light documentation formats in individual and community endeavors and the (dis)advantages of some approaches and toolsets over others.

Cheers,

Offray

overcomplicated rationale

Okay, let me simplify it, then:

I want to add a third more powerful markup language to Fossil, because I believe I need some features for this user manual project that the current Fossil Markdown and Wiki formatters do not have.

I will then convert the existing manual to that markup language, then break it apart into useful pieces, then update the pieces. This will allow Fossil users to refer someone to a URL like https://fossil-scm.org/userman/doc/trunk/merge.rst so they can read just the chapter on merging in Fossil. It will also make searching the user manual more useful.

I want to do all of this inside Fossil so that we can use Fossil as our CMS for delivering this manual, with the "source code" to the manual being generated on the fly to HTML, just as we currently do for Markdown and Wiki format content in Fossil.

If you still do not understand my position after that explanation, please give up trying to persuade me. We're just going to keep talking past each other.

I've written manuals the way you suggest before, compiling a document to static HTML or PDF output, then uploading that somewhere. I don't want to do that any more. Period. Fossil is as much CMS as VCS. I want to work to its strengths, not treat it like a static web server.

That settles it: even the most inclusive flavor of Markdown still doesn't have an "include" directive, so you must write the whole document in a single huge file.

That's not true for Pandoc, which understands CommomMark

I looked at the CommonMark spec and couldn't find an include directive. So, point me at the section in the spec where it shows how I can do this.

If I can achieve my desired end simply by extending the existing Fossil Markdown processor, I might very well choose that option instead. I like Markdown just fine. It just isn't as powerful as something like DocBook, and I believe I need some of that power.

The open sourced Data Journalism Handbook, linked before, was produced without starting from a huge monolitic file.

While composing my reply to your prior post, I looked at your project's files view, and saw two Markdown files, one of which is very large and which I assume is the entire contents of the book. None of the subdirectories appear to contain the same material in chunks.

Either I'm missing something big, or your argument appears to be, "Trust me, it can be done." How about you show me how it can be done instead. Give me a simple example demonstrating the feature. I'm not going to pore through a book-length Markdown file to figure out what you're trying to get across here.

Okay, let me simplify it, then:

I want to add a third more powerful markup language to Fossil, because I believe I need some features for this user manual project that the current Fossil Markdown and Wiki formatters do not have.

I will then convert the existing manual to that markup language, then break it apart into useful pieces, then update the pieces. This will allow Fossil users to refer someone to a URL like https://fossil-scm.org/userman/doc/trunk/merge.rst so they can read just the chapter on merging in Fossil. It will also make searching the user manual more useful.

I want to do all of this inside Fossil so that we can use Fossil as our CMS for delivering this manual, with the "source code" to the manual being generated on the fly to HTML, just as we currently do for Markdown and Wiki format content in Fossil.

If you still do not understand my position after that explanation, please give up trying to persuade me. We're just going to keep talking past each other.

I think that we can overcome the personal and confrontational tone about what one is trying to do to the other ;-). I understand you better now, but, again, I'm not trying to persuade you about changing anything that you want to do by yourself in a solo endeavor (or with the people that share your approach). I'm trying to be part of a community conversation about how the Fossil Book can be updated, but (again) even if some follow a particular approach, I don't care about incompatibilities, because I have Pandoc to convert from that to the approach I'm suggesting, as I have done before.

I just think that is good to have an overview of the approaches others have follow while dealing with a similar problem: the limitations of plain Markdown for producing complex documents, like books. Some abandon plain Markdown for more complete but different formats (reST, AsciiDoc), some extend Markdown (CommonMark, Pandoc), keeping the learning of the basics that is already in the community, while making it more powerful and complete. That is the approach I have followed,

Now I have a set of documents made in Pandoc's Markdown and I would like to improve support for them in Fossil. My approach for making it is the opposite as yours, I want to be as far away as possible from C and I want to keep Markdown knowledge (while extending it) instead of switching markup languages completely. My idea is to extend Fossil with Lua via some javascript library like Startlight (as has be done to extend support for syntax highlighting recently using highlight.js) and use Lunamark to support an extended and extensible Markup syntax. In that way document rendering and traversing can use Pandoc syntax without relying on heavy weight libraries (not even the Pandoc binary or its Haskell toolkit) and without changing a lot Fossil internals or already supported markup languages (at some point I would like to have better Fossil+Lua integration, but that's another project).

While composing my reply to your prior post, I looked at your project's files view, and saw two Markdown files, one of which is very large and which I assume is the entire contents of the book. None of the subdirectories appear to contain the same material in chunks.

Yes, but that huge Markdown file was produced by reverse engineering source individual HTML files, convert them into individual files Markdown and, for that particular project, recombine them into a single file, just because we work with an outliner and it give us an unified view of the whole and the parts. That doesn't mean that you can not recombine individual files in other ways (see below).

I looked at the CommonMark spec and couldn't find an include directive. So, point me at the section in the spec where it shows how I can do this.

[...]

Either I'm missing something big, or your argument appears to be, "Trust me, it can be done." How about you show me how it can be done instead. Give me a simple example demonstrating the feature. I'm not going to pore through a book-length Markdown file to figure out what you're trying to get across here.

Here you have a particular example on how to create a book from individual files using Pandoc:

http://pandoc.org/epub.html#a-real-book

Notice that talking about CommonMark was done while mentioning CodiMD as a real time community environment for documentation (in case that approach would be interesing for this community), but the proposed toolkit would use Pandoc (and its Markdown variant) or a lighter version of it (Lunamark).

Anyway this two explorations about how to extend Fossil documenting capabilities can be in parallel as I'm not persuading you of anything and our approaches, and toolkits and timelines are pretty different. I just thought that the community would benefit for knowing that such alternatives exist.

Cheers,

Offray

I'm not trying to persuade you about changing anything that you want to do by yourself in a solo endeavor (or with the people that share your approach). I'm trying to be part of a community conversation

FOSS is a do-ocracy: he who does the work makes the rules. I've volunteered to do this project, and I've proposed to do it using approach X. You prefer approach Y, so you tried to persuade me to switch to that approach, but failed. That leaves you with only one choice: you can take this user manual project away from me, in which case you get to choose the approach.

So, do you want the job? You're welcome to it. I'm happy to go back to being primarily just a user of Fossil.

Here you have a particular example on how to create a book from individual files using Pandoc:

CommonMark is not combining the documents, pandoc is, through simple concatenation.

Includes are not the same thing: they allow arbitrary insertion points.

Let's take a look at another chapter from the MySQL++ User Manual. Search that page for the 10 uses of the <xi:include ...> XInclude directive. Notice that they're scattered throughout the document. How would you do that in Markdown, with or without pandoc?

I think you'd have to strap a preprocessor to it, like M4.

The question then is, how would I extend Fossil's Markdown processor to do the same thing? I think I'd end up creating yet another nonstandard Markdown extension, which is how we got into the mess the lead us to need CommonMark in the first place.

FOSS is a do-ocracy: he who does the work makes the rules. I've volunteered to do this project, and I've proposed to do it using approach X. You prefer approach Y, so you tried to persuade me to switch to that approach, but failed. That leaves you with only one choice: you can take this user manual project away from me, in which case you get to choose the approach.

So, do you want the job? You're welcome to it. I'm happy to go back to being primarily just a user of Fossil.

Thanks for the FLOSS 101, but I understand how FLOSS works. As I have told you several times, giving a panoramic view of how a problem can be solved doesn't mean I'm trying to do anything to you. You see FLOSS as (personal?) confrontation with people trying and (mostly) failing or (some times) succeeding, particularly while talking in response to your messages. I don't. I think that other people reading the thread has faced Markdown limitations in the context of Fossil (and other projects) and could be interested in tracing ways to overcome this, for example in the context of tackling the Fossil Book project. A conversation about this is not always about who gets the job. Please ignore my comments and follow your path whatever it is. I will try to not answer to you, but finding other places in the thread to document alternatives approaches without going into the "Macho" FLOSS programmer/user confrontation with victories, failures and fights for jobs.

CommonMark is not combining the documents, pandoc is, through simple concatenation.

I was showing that not only monolithic huge documents are the only way to create complex documents, because that was implied in the thread. Simple concatenation is powerful because is simple. I already tried to clarify the context of mentioning CommonMark in the thread and mentioning that Pandoc is the core of the proposal, using it as a whole or some light compatible versions like Lunamark. I don't think this conversation will continue, but at least hopefully others will see the differences and stop insisting in the completeness of CommonMark format over the Pandoc based toolkit.

Includes are not the same thing: they allow arbitrary insertion points.

There are several options for includes in Pandoc.

The question then is, how would I extend Fossil's Markdown processor to do the same thing? I think I'd end up creating yet another nonstandard Markdown extension, which is how we got into the mess the lead us to need CommonMark in the first place.

But the fact is that we already have CommonMark, Pandoc and Lunamark, so is possible to have things like includes in a light weight way without creating another nonstandard Markdown extension. But please (seriously) feel free to ignore that. I don't want you to consider anything different of whatever you come only by yourself and serve your purposes and workflows. I'm not trying to "take your job", nor trying to participate in any piss contest, and I don't want us having such nonconstructive attitudes and talks.

Offray

I would say that CommonMark and/or Pandoc (and its Markdown extension) is also good for book size writings, while keeping pretty good readability.

Cheers,

Offray

I personally prefer ConTeXt when needing to create manuals, etc. But that is primarily for PDF output, which is a basic requirement for what I am doing. I find ConTeXt to be much easier to get a book out of, since ConTeXt is meant for books, while LaTeX was meant for articles and journals. ConTeXt also allows for inclusion of separate files for chapters/appendices/etc.

However, the argument for direct rendering to HTML is a valid one, and not easily done from any TeX based language. While Markdown is nice and lightweight it is probably too lightweight for higher-end documentation. However I prefer Textile to Markdown and would suggest looking at it as well.

I think Pandoc's Markdown (and maybe CommonMark) beat AsciiDoc when support for admonitions is enabled, mainly because of its readability in comparison with AsciiDoc, as explained here.

Yes, plain Markdown is too lightweight for more demanding documentation task, but CommonMark and Pandoc's Markdown support complex document requirements. Lunamark is a light Markdown extension that supports several Markdown features and is pretty embeddable to extend plain Markup beyond its original features.

I don't want to add fuel to the fire in this discussion, but may I throw in another possibility I stumbled upon some time ago (haven't tested it myself so far, to be honest):

Markdeep

It's a superset of CommonMark markdown, renders to HTML, allows inclusion of other documents, is BSD-licensed, and is "self contained" without the overhead of some of the bigger alternatives mentioned so far (it requires a JavaScript file which would need to be included into the repository).

This might allow to maintain a small book as documentation within the Fossil sources, written in plain text (resp. Markdown), but still offering quite nice HTML output.

Markdeep would be excellent. I've used it a bit, very nice to have the rendering handled from a single javascript file.

Markdeep seems pretty interesting and minimal. Glad to know that a Markdown superset can be implemented easily without reinventing the wheel or changing totally the supported documentation format.

Cheers,

Offray

Earlier:

I want to add a third more powerful markup language to Fossil, because I believe I need some features for this user manual project that the current Fossil Markdown and Wiki formatters do not have.

Later:

FOSS is a do-ocracy: he who does the work makes the rules.

I just have two cents, as a single solitary Fossil fan and user.

1st $0.01: I’m personally kind of dismayed at the thought of Fossil being extended with any more markup languages, just so it can include support for rendering a structured, book-length piece of documentation. As someone who does web and print book publishing, and is familiar with pandoc and asciidoc and the 87 flavors of markdown, I think that job is just better left to tools that are dedicated to it.

2nd $0.01: A possible slight problem with the do-ocracy mindset here is that there is no "work" a person can do that will "make the rule" that Fossil would be better off without a particular feature.

That's all, corrections/criticism welcome.

Although I'm not a fan of Markdown and its derivatives, I do like the idea that markdeep is implemented as a single Javascript file. Also, I like the idea of not having to learn yet another "mark up language".

(Out of curiosity, I searched for a javascript implementation of AsciiDoc and found "asciidoctor.js". Unfortunately, it seems to be dependant on "node.js" (the official installation instructions only talk about using npm to install it).)

I’m personally kind of dismayed at the thought of Fossil being extended with any more markup languages, just so it can include support for rendering a structured, book-length piece of documentation.

Not "just". Others will want to store book-length documents in their Fossil repos, too. Me, for one: I'm proud of having produced the PDF version of the MySQL++ manual, but I could see myself switching to this new system once it's working.

Since all of the above went down, I've since learned that GitHub already supports AsciiDoc and reStructuredText. So we're behind, oh noes! :)

I've begun work on a reStructuredText to HTML renderer, but I don't have anything worth sharing yet.

I think [document production] is just better left to tools that are dedicated to it.

As someone who has done many projects in DocBook and is proud of his hard-won skills with that toolchain, I will only be a little sad to see it in my rearview mirror.

Of course all of this is based on the assumption that this project will succeed. It could crater. We'll see.

there is no "work" a person can do that will "make the rule" that Fossil would be better off without a particular feature.

Sure there is: you just did it, which is one reason we have a forum.

Another reason we have a forum, though, is that someone (drh) decided to do the work and let the work speak for itself. It's too early for my project, but that day will come before the code is pushed into Fossil.

I'm developing the first version of this feature in a separate code repository, which I'll eventually make public, if it's at all usable. The idea is then to produce an amalgamation version of the library, strap a Fossil Blob-based HTML generator to it, and then offer it to drh for inclusion into Fossil, rather than check it in unsolicited.

If it turns out that I get something working, but it sucks, that may be as far as the project goes.

It's really way too early to be worrying about what this will end up doing to Fossil.

+1 for markdeep.

It also integrates nicely in doxygen.

Cheers Marc

I disagree that the book and its tools need to be embedded into Fossil. Its a nice tight, small executable now.

No one said the Schimpf book contents would be embedded into the Fossil executable. The idea is to put it into the Fossil source code repository, right along with the contents of its www subdirectory where Fossil's embedded documentation is already. That is, the book would just be more embedded documentation.

As for "its tools," the rendering feature would be no different from the present lightweight markup to HTML rendering features of Fossil, wiki and Markdown. If those languages could be pressed into service as-is, we wouldn't need more powerful languages. I've already made my case for more powerful markup languages in this thread, so I won't repeat those arguments here.

Bloat is bad.

Define "bloat."

If the definition of "bloat" is "stuff I don't want," I struggle to see how any software would exist at all, since that would mean the only existing software we keep and the only new software we write is that which we can get 100% buy-in for.

Whatever language(s) get implemented to achieve this end, if any at all (!), they'll likely be optional, so the only thing getting larger is the Fossil source tree.

Trying to embed a SMTP is bad too.

While I'm personally in favor of using an external MTA with Fossil, there are good arguments in favor of an internal MTA:

1. It becomes practical to allow email forum submissions by email, while still blocking spammers. A forum subscriber could contribute to a thread simply by replying to an email alert, so that a subscriber never needs to re-visit the web forum after signing up, just as with old mailing lists.

2. Dealing with bounces changes from a multi-MTA integration problem into an inherent feature of the new software.

3. As the first step of setting up Fossil alerts, "Just set up an MTA" passes a lot of complexity to the users of Fossil. drh wants to make the process of setting up mail service for a project easier. Given his track record, he may well succeed.

once the toolset is determined.

Yes, well, now that the release is out, maybe I can get back to that.

Have you considered Creole? There's a BSD licensed ISO-C implementation. FWIW, Creole 1.0 is the default syntax in Bitbucket wikis, which also support some Creole 1.0 additions.

I've just learned that reST has an include directive

Not sure if Creole supports an include directive of any sort, but perhaps that could be handled with an addition that Creole allows for.

As far as I can tell, Creole is just an attempt to standardize wiki markup language. It's roughly equivalent to what we have already.

The only advantage I can see in supporting it is just as a lingua franca...or a creole, if you will. ;)

Hah, look what I just found while browsing the Fossil project's branch list:

https://fossil-scm.org/fossil/timeline?r=creole

Yah, I found that as well via Extending Fossil's Wiki. I wonder why that was abandoned?

dunno why the <<creole>> and <<fossil>> macro approach was chosen there. Treating the .creole file extension on equal footing with .wiki and .md makes more sense IMHO. On the other hand, having fossil's built in wiki use creole markup is nice too.

It's been nearly a year and a half since this thread died. Any progress since? Could we use markdeep within Fossil now?

Markdeep will never be embedded in fossil but it can be used as-is with fossil. Here's an example:

https://fossil.wanderinghorse.net/r/morbad-card-tools-s2cgi/doc/ckout/doc/index.html