Here's some proof of concept for defining a repo's site menu via the db. The idea is that the repo defines its site menu in a table then modifies their skin(s) to read the menu list from there instead of hard-coding the menu into every skin.

First the code/demo:

Schema

$ echo '.dump fx_menu' | fossil sql
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE fx_menu (
  anycap TEXT DEFAULT NULL, -- value for use with th1 anycap function
  seq INTEGER DEFAULT 0, -- relative order of this entry in the menu
  label TEXT NOT NULL,
  uri TEXT NOT NULL
);
INSERT INTO fx_menu VALUES(NULL,1,'Timeline','timeline');
INSERT INTO fx_menu VALUES('sa',2,'Admin','setup');
COMMIT;

Test Script

A test script representing what would go into a skin:

$ cat mtest.th1
repository 1
set home /
# ^^^ normally set via TH1 style environment
proc menulink {url name} {
  upvar home home
  html "<a href='$home$url'>$name</a>\n"
}
query {
  SELECT anycap, label, uri FROM fx_menu ORDER BY seq
} {
  if {"" eq $anycap || [anycap $anycap]} {
    menulink $uri $label
  }
}

Output

# Guest user:
$ fossil test-th-source mtest.th1 --no-print-result
<a href='/timeline'>Timeline</a>

# Setup user:
$ TH1_TEST_USER_CAPS=s fossil test-th-source mtest.th1 --no-print-result --set-user-caps
<a href='/timeline'>Timeline</a>
<a href='/setup'>Admin</a>

Caveats

• The menu table does not sync, nor can it be easily administered via the web interface. It "can" be done via /admin_sql, but that's a bit tedious. This means that the menu would not work for any clones unless/until they somehow obtained a copy of the fx_menu table.

• The menu generation code probably needs to be slightly different for some skins.

• Probably something else.

Hypothetical Use Cases

• A repository which offers access to the same repo via multiple skins, e.g. https://fossil-scm.org/skins, use this to define the main menu in a single place.

• Hypothetical support for offering users the option of selecting (but not editing) skins, noting that the list of options would/should be limited to those skins which the admin has configured to work with the repository (e.g. edited to include any required custom scripts/CSS/etc).

Having the site menu in the db would be a first step towards simplifying maintenance of such multi-skin setups. However, the inability to sync the menu is a bit of a show-stopper.

This isn't a rabbit hole i'm currently prepared to go down, but maybe it will inspire some ideas for others.

I think we've been struggling with the same question, which is:

How Can We Make Skinning And Site Configuration Easier?

This is a worthy goal, I believe, and definitely worth looking into for the next release cycle. I don't have a solution yet. Let's keep this idea of an fx_menu table under consideration.

How To Sync The FX_MENU Table

I've been wondering if I shouldn't enhance the sync protocol to make it easier to keep specific SQL tables in sync, using the "fossil config pull/push/sync" command. This could be accomplished, perhaps, by creating an in-memory SQLite database file with the information to be transferred, then sending that entire database as a single card in the sync message. I've been wanting this capability so that I can transfer the CHAT table, for example. The same infrastructure could be leveraged to move the FX_MENU table, or whatever other FX_ tables the administrator wants to move.

One concern is how to deal with the case where the client and server are different versions of Fossil that are using slightly different schemas for the tables being moved....

Other Skin Simplification Ideas

Just this past weekend, I've been contemplating the idea of combining all of the skin information into a single text file. This "*.skin" file would be a human-readable and editable file that contains all of the information needed to generate:

• Header
• Header menu (and/or FX_MENU table)
• Footer
• CSS color choices
• CSS font choices
• CSS layout and spacing
• Graph drawing options
• Pikchr rendering options
• Javascript appended to the end of every page
• Ticket schema
• Ticket options and forms

When Fossil loads a skin-file for the first time, it parses it up into its various components and stores the components separately, just as it does now, so that they can be rendered quickly for each HTTP request. But the original skin-file is preserved for future edits. The goal is to make it easy to pass around and try out skin-files to see how they look.

Most people, when they want a custom skin, just want to tweak the colors and the header menu. We should have specific sections of the skin-file the specifically and completely specify the colors and the header menu. That way, people can take an existing skin and easily make the adjustments they want, without having to get tangled up in a lot of complex CSS and/or TH1 scripts. Complete control of the CSS and/or TH1 is available to experts or people designing completely new skins, but for the common case of some color tweaks, knowledge of CSS/TH1 is no longer necessary.

Perhaps has part of this effort, we could also rationalize and simplify the DOM class names and the corresponding CSS?

Fossil Version 3?

Extensive changes to the skin mechanism might require people who operate servers with custom skins to redo their customizations when upgrading. Does this justify changing the version numbers from 2.x to 3.x?

This could be accomplished, perhaps, by creating an in-memory SQLite database file with the information to be transferred, then sending that entire database as a single card in the sync message. I've been wanting this capability so that I can transfer the CHAT table

One potential headache with that, which came up when the idea of sending self-contained dbs like this came up before (a few years ago, but i don't remember the context), is that it's fine for small tables (menu) but won't work with large amounts of data on memory-constrained environments (e.g. shared hosters may have quotas in place). The chat table can hold an arbitrary number of large images and files, and could easily be a 50MB+ payload.

This "*.skin" file would be a human-readable and editable file that contains all of the information needed to generate:

Having everything in one file would probably (ironically) make editing more difficult for many (most?) users by breaking things like syntax highlighting and automatic indentation. emacs would have a fit with such a bundle. My emacs is configured to treat css.txt as CSS and js.txt as JS code, so highlighting and indentation work nicely.

The goal is to make it easy to pass around and try out skin-files to see how they look.

Having each skin in its own directory serves that purpose well, IMO. We could possibly expand support to reading a single skin from a sqlar file instead of a directory, so that fossil ui --skin foo.sqlar would use whatever skin is in the top of the archive file. That could be done within the current skin framework. Reading from a zip file would be more user-friendly, but then we'd need new code for reading a zip file, whereas the sqlar is something we already have code for reading.

Most people, when they want a custom skin, just want to tweak the colors and the header menu.

Guilty as charged, plus i sometimes add a couple 3rd-party CSS imports and a JS script for adding syntax highlighting to file/wiki/embedded doc content.

Perhaps has part of this effort, we could also rationalize and simplify the DOM class names and the corresponding CSS?

That's a topic near and dear to my heart, alongside documenting the CSS classes which are intended for public use.

Does this justify changing the version numbers from 2.x to 3.x?

It would ostensibly require changing every skin out there, so arguably yes. There's no particular hurry/pressure, though.

reading a single skin from a sqlar file instead of a directory

Please consider also adding a setting that specifies a particular check-in that holds the appropriate skin's files within a well-known (or maybe configurable) directory.

combining all of the skin information into a single text file

Much worse than the syntax highlighting issues Stephan brought up, I think you'll end up battling differing interpretations of the languages' syntax in edge cases. When you have two different "escape" characters, three different quoting styles, and four different sets of whitespace rules in a single file, a single misplaced punctuation character could asplode things you prefer did not asplode.

Your idea of a SQLar file is better. It's not easily editable, but that's an easier problem to solve than the quoting, escaping, and whitespace problem.

Proposed design:

fossil conf export all               # no FILENAME, so yields config.sqlar
fossil conf export chat chat.db      # clone of `chat` table
fossil conf export all --expand      # yields css.txt, js.txt, chat.db, user.db...
fossil conf import foo-config.sqlar  # sees SQLar, expands it, and shards it
fossil conf import all *.txt         # infers AREA from passed files' names

Thus the config interface would use text files where that makes sense (CSS, JS, etc.) but SQLite tables where that's better, such as the chat and user tables in my example. The all-inclusive SQLar file would combine both: archived text files and direct copies of the copied tables. That is, it would avoid storing SQLite-in-SQLar, there being no point to that.

Most people, when they want a custom skin, just want to tweak the colors

Thus Inskinerator.

You're welcome to rebuild the idea in a more Fossil-native way. I only used Perl and made it external because of the SCSS layer: I didn't want to write a SCSS translator in C just to put this into Fossil proper.

...and the header menu

I have only the vaguest of ideas of how to extend Inskinerator to cover the non-CSS aspects of the skin. But yes, it would be good if it could generate details.txt, header.txt, etc. as well as css.txt.

rationalize and simplify the DOM class names and the corresponding CSS?

I've been doing that as part of the Inskinerator project, upstreaming as much of it as I could. It's a good place to do it because it's only as you study the multi-part skinning infrastructure that you can see where there's stuff that's duplicated, misordered, and abandoned.

You can see that most clearly in my common/css.scss file. The incompleteness of the project is documented by the extent to which there are still hard-coded colors in there rather than SCSS variables. (Search for dollar signs to make the unhandled cases stand out.)

Does this justify changing the version numbers from 2.x to 3.x?

As I've laid it out above, I don't see the need, but I expect that you see further than I do.

won't work with large amounts of data on memory-constrained environments (e.g. shared hosters may have quotas in place)

I've most commonly seen it with large unversioned files when Fossil is behind a proxy, since the limit on an HTTP reply's body is around a meg in common HTTP proxy servers. The sync protocol doesn't have a problem with that, being able to break things up to get under the limit, but /uv accesses send large files in a single message.

It would ostensibly require changing every skin out there, so arguably yes.

So far, my rationalizations of CSS have been limited to either src/default.css or to harmless cases in skin/default/css.txt, which lets us stick with "2.x".

Only if we do something that invalidates all downstream custom skins do we have a real problem, and even then, I'm not certain we must go to 3.x. We've broken downstream skins with upstream changes before, requiring that those with custom skins adjust them when they upgrade. I recall the 2.5 to 2.7 era requiring a lot of this.

I suppose it comes down to how extensive the "creative destruction" is.

it would be good if it could generate details.txt, header.txt, etc. as well as css.txt.

And now that I’ve thought more about it, it would be good if Inskinerator could generate a skin.sqlar file and load it directly into a “draft” slot for immediate preview via Fossil UI.

An Alternative Idea For Consideration:

1. Add another skin configuration file "menu.txt" that, like header.txt and footer.txt, contains TH1 code. The menu.txt file contains TH1 code to generate the menu.

2. Add a new built-in TH1 command, perhaps called "generateMainMenu", that runs the TH1 in "menu.txt"

3. Modify the various built-in skins such that instead of generating the main menu directly, they invoke the generateMainMenu command.

4. The techniques for specifying an alternative skin (the --skin option to "fossil ui", or the "skin:" entry in the CGI script) continue to use the default "menu.txt", not the alternative menu.txt, unless the name is followed by "!".

In this way, you can run alternative skins on a repository yet easily keep your customized menu bar. I think this technique is backwards compatible to legacy skins that do not invoke the generateMainMenu command.

Complications due to the hamburger menu

The hamburger menu complicates this, but only slightly. I propose to promote the /skins/default/js.txt file to be another built-in JS file and provide a new TH1 command to cause that file to be loaded. Then, any menu.txt that wants to include a hamburger menu merely has to include an <a> with id='hbbtn' and an <div> with id='hbdrop' and invoke the new TH1 command to cause the JS to be inserted.
A menu.txt that does not want to use the default hamburger JS can insert their own, as they have access to the $nonce.

Modify the various built-in skins such that instead of generating the main menu directly, they invoke the generateMainMenu command.

There's one catch with that: some of the skins generate the menu items slightly differently. e.g. some explicitly mark an entry as the current page if the page URI matches the menu's, but others don't. That said, there's no harm in always marking those with CSS, as skins are under no obligation to make use of that marker class.

It's also conceivable, but i haven't confirmed whether this is currently the case in any skins, that a skin may create menu entries as a list of LI elements, rather than a collection of anchor tags.

The techniques for specifying an alternative skin (the --skin option to "fossil ui", or the "skin:" entry in the CGI script) continue to use the default "menu.txt", not the alternative menu.txt, unless the name is followed by "!".

How about: if a skin has its own menu.txt, use that one, else fall back to the one in the default skin (if any)?

How about: if a skin has its own menu.txt, use that one, else fall back to the one in the default skin (if any)?

That doesn't seem to solve the problem I'm trying to solve, which is, how can I look at my current setting (with my customized menu) but using one of the other built-in skins?

how can I look at my current setting (with my customized menu) but using one of the other built-in skins?

That would seem to argue for menu.txt being a repo config option, rather than (or in addition to) a skin option. If repo-level menu.txt is set, use it, otherwise use the skin's (if available), otherwise ignore it (or fall back to the default skin's)?

That assumes that the menu.txt setting is compatible with all skins, which might not be the case but would certainly be the ideal, giving us centralized control over the menu.

Patches inspired by this idea, though implemented somewhat differently, are now on trunk:

• There is a new setting named "mainmenu" that is a big multi-line string that describes the content of the main menu as a TCL list. A repository can edit this one string to customize the main menu. Use the /Setup/Configuration setup page to modify the "mainmenu" setting.

• All of the built-in skins are revised to use the "mainmenu" setting to generate their main menus.

• As most skin customization (at least in my experience) is about changing the main menu, this means that most repositories can now run with a default skin.

• It also means that the skin demos (including the forum skin demos) now all have the correct menu bar.

Additional recent changes include:

• The customization of the /sitemap page is now done with a single setting, rather than a cluster of settings. This provides additional flexibility in customizing the /sitemap page. As /sitemap is often used as the hamburger menu, this adds new customization oppportunities to the hamburger menu.

• Hamburger menus have been added to the ardoise and plain_gray. skins. The operation of the hamburger menu has been simplified somewhat so that it should be easy to add it to other built-in skins, if desired. Feedback is requested.

Please try out the new changes and post a follow-up if you encounter any problems or see any areas in need of improvement.

Patches inspired by this idea, though implemented somewhat differently, are now on trunk:

i've been watching that closely and like what i see :). That consolidation of the TH1 caps features was a definite improvement - having 3 different functions/semantics for that was a hurdle/problem i punted on in my proof of concept.

The next step, i think: the skins mostly have the same foreach loop for generating the menu. We "could" move the code for that one menu-rendering option into a th1 function so the the skins simply need to call renderSkinMenu, without any arguments, to run that loop. There will always be outlier skins which need custom rendering, and they can implement their own foreach loop.

i will dig through that and see if it would give us any benefit to consolidate those as soon as my left hand allows me to use emacs again.

Please try out the new changes and post a follow-up if you encounter any problems or see any areas in need of improvement.

i have a slight hiccup here which i'm not sure how to solve...

The menu (currently set in the header.txt of a local skin):

  set mainmenu {
   Home      /home        *              {}
   Timeline  /timeline    {o r j}        {}
   {s2 docs}  /doc/ckout/s2/      {o r j} {}
   Files     /dir?ci=tip  oh             desktoponly
   Wiki      /wiki        j              wideonly
   Setup     /setup       s              {}
  }

The rendering code is taken directly from the default skin:

foreach {name url expr class} $mainmenu {
  if {![capexpr $expr]} continue
  if {[string match /* $url]} {
    if {[string match /$current_page* $url]} {
      set class "active $class"
    }
    set url $home$url
  }
  html "<a href='$url' class='$class'>$name</a>\n"
}

The problem is that 3rd menu entry: it is the only one of those entries for which the "is-active" check does not pass when that is the current page. My initial thought was because it ends with a slash, but the behavior is the same if it instead ends with /index.md.

Do you have a suggestion for how to rephrase the is-active check so that it will catch such URLs?

Please try out the new changes and post a follow-up if you encounter any problems or see any areas in need of improvement.

It's now in place on all of my active repos over at https://fossil.wanderinghorse.net. Aside from the "active" selection quirk, which isn't visible there because that skin doesn't currently do anything with that CSS class, all is well. This certainly saves me some grey hairs when rolling out changes to my custom skin across repos, as the menu differs significantly for a couple of them.

Here's a hint for those wanting to script the setting of the menu value:

fossil sql -R repo.fossil
sqlite> replace into config(name,value,mtime)
   ...> values('mainmenu', readfile('/path/to/the-menu.th'), now());

I'm not able to repro this problem. Or, perhaps I'm not understanding the problem.

In a local Fossil repository, I added:

Extra /doc/ckout/www/ * {}

And in the skin I wrote:

foreach {name url expr class} $mainmenu {
  if {![capexpr $expr]} continue
  html "<!-- url=$url current_page=$current_page match: [string match /* $url] [string match /$current_page* $url] -->\n"
  if {[string match /* $url]} {
    if {[string match /$current_page* $url]} {
      set class "active $class"
    }
    set url $home$url
  }
  html "<a href='$url' class='$class'>$name</a>\n"
}

When I visit the "Extra" tab, it shows both matches as 1 and the class is "active".

I'm not able to repro this problem.

Weird. i'll take a closer look at after and see if i can find the culprit. It's not a big deal, in any case, as i'm not currently using that CSS class for anything.