Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

200 most recent check-ins by user wyoung

Fixed a typo in an error message ... (check-in: f25da6ec user: wyoung tags: trunk)
Improved the documentation for quote parsing in globs. ... (check-in: 609bf98b user: wyoung tags: trunk)
Fix to the prior commit: editing lost a closing "*/" in a comment, yet it compiled with only a warning, and I missed the warning. ... (check-in: c5d287b6 user: wyoung tags: trunk)
Updated all of "fossil help *-glob" output to refer to the syntax docs instead of repeating the parsing rules in each setting, incorrectly. (If URLs in help output are verboten, then we need to repeat the rules *correctly* in each setting's help.) ... (check-in: 7dc0433b user: wyoung tags: trunk)
Added a few named anchors to the globs doc, added a mention of the new empty-dirs handling, and improved the internal MD markup style. ... (check-in: b293b265 user: wyoung tags: trunk)
Brought the globs doc back into line with the underlying implementation, and to mention the new treatement of the empty-dirs setting's value. ... (check-in: 2c0b70eb user: wyoung tags: trunk)
Switched the empty-dirs setting handler from a one-off parser to glob_create() both so we get consistent behavior across the settings and because glob_create() allows quoted whitespace, needed when asking it to create directories with spaces in them. Addresses this forum report. ... (check-in: f12609fc user: wyoung tags: trunk)
Updated the function comment on glob_create() to match what it actually does. ... (check-in: c43205d7 user: wyoung tags: trunk)
Replaced a complicated bit of logic with something slighlty less complicated, having the same effect. The glob parser used a mix of second-clause for-loop testing and internal break and continue checks without any other internal processing inside the loop. Combining all of this into a single expression requires the line to wrap (bad for clarity) but it does make clear all of the conditions required for this loop to continue iterating. I think it's a net improvement in clarity, though the margin is admittedly small. Testing shows no regression in functionality, limiting this non-functional change to a style improvement. ... (check-in: b8789239 user: wyoung tags: trunk)
Reworked the prior commit's changelog entry to be more accurate and to match preexisting style. ... (check-in: 8ce70b4c user: wyoung tags: trunk)
Fixed a symmetry problem in the vfile.chnged implementation: it had an explicit check for when a file used to be a regular file and was replaced by a symlink pointing to the same content, but it didn't then check for the inverse case. Since the size and hash checks become nonsense when both files aren't of the same type (whether symlinks or regular files) we have to check for this case, too, else the only way to get the "UNLINK" output from "fossil status" required trickery.

See this forum post for a test case and the trick previously needed to get Fossil to report "UNLINK" as a file's change status. ... (check-in: 4cca9033 user: wyoung tags: trunk)

Moved the "sync is all-or-nothing" section of the gitusers doc above the autosync section. The order of those two isn't important, and it's better that autosync flow immediately into the new "reset" section without this as a digression. ... (check-in: b47b96a0 user: wyoung tags: trunk)
Clarity pass on the new material in the Autosync section of the gitusers doc. ... (check-in: cc55044b user: wyoung tags: trunk)
Added a link from the new "git reset" example in the intro of the gitusers doc to the new section where we address the matter. ... (check-in: 84955715 user: wyoung tags: trunk)
Clarity pass on the autosync section of the gitusers doc ... (check-in: 80963ffe user: wyoung tags: trunk)
Added a new section "Resetting the Repository" to the gitusers doc, following and relying on the two recently-updated sections. ... (check-in: bd7b1099 user: wyoung tags: trunk)
Added a motivating example to the "Autosync" section of the gitusers doc, showing why it's better to let the software system resolve conflicts asynchronously than to force Alice and Bob to fight it out, moving the synchronization point out into the human-time workflow. ... (check-in: 8975e8da user: wyoung tags: trunk)
Clarified the "Sync Is All-or-Nothing" section of the gitusers doc, adding a reference to the cap-theorem doc and dropping the "eventually-consistent" buzzword to explain *why* it's all-or-nothing. ... (check-in: 35b2307f user: wyoung tags: trunk)
Made the msmtp mention in the alerts doc more concrete now that we have a verified tester. ... (check-in: ea74c83a user: wyoung tags: trunk)
Removed all of the XML-style " />" tag closing markers on empty tags such as "hr", "br", and "input" to placate modern HTML5 validators. That's the doctype we declare, not XHTML, so we should conform. (Besides which, the XHTML dream is dead.) ... (check-in: f5482a0a user: wyoung tags: trunk)
Small fix to previous ... (check-in: 5537d9c2 user: wyoung tags: trunk)
Replaced use of the obsolete (because presentational) "center" tag in the docs with CSS equivalents. ... (check-in: 4d21ec88 user: wyoung tags: trunk)
Added an alt attribute on the home page's Fossil logo ... (check-in: ccf117fe user: wyoung tags: trunk)
Fixed a manual section numbering duplication in the delta encoding algorithm doc. ... (check-in: d83f2490 user: wyoung tags: trunk)
A prior edit to the quickstart doc left a vestigial version of the "Making and Committing Changes" section behind, resulting in duplicate information and a second h2 section with the same fragment ID. Some differences existed between the two presentations, so merged them rather than nuke the smaller (presumably older) one outright. ... (check-in: 39beee4c user: wyoung tags: trunk)
In an effort to make www/* source documents read as cleanly as their rendered counterparts, replaced nearly all use of HTML "p" tags, relying instead on the Wiki and Markdown markup features to achieve the same appearance. The only uses remaining are:
  • in Markdown nested lists, where blank lines should render the list items as separate paragraphs just as at the list's top level; since it does not, if you want a line break, you either have to wrap the item in "p" tags or do the double-br hack.
  • in Wiki where blank lines within a list give you a separate list in the HTML output; this is fine for bullet lists, but with numbered lists it causes the numbering to restart unless you do the same sort of manual HTML workaround as with the prior item
  • in plain HTML docs and wiki docs between "nowiki" tags

In many places, this cleanup gets rid of pointless stray "p" tags, placating HTML verifiers. ... (check-in: 1fd407f6 user: wyoung tags: trunk)

Stray character deleted ... (check-in: 0cb4f1fc user: wyoung tags: trunk)
Replaced use of HTML "p" tags in the doc with hard line breaks, letting the wiki language add the same "p" tags on output. Replaced some of them with "tt" tags instead to make command text stand out from the prose. Added "tt" tags elsewhere for consistency. ... (check-in: 262eb510 user: wyoung tags: trunk)
Added explicit SIGINT handler for the rootful container (PID 1) case, so you can Ctrl-C it when "run -it". ... (check-in: a0b0d8f9 user: wyoung tags: trunk)
Set permissions on /bin/fossil inside the container to 755, not 700. Docker doesn't care since it runs everything as virtual-root, but Podman does. Also, it complicates the extraction of a static binary since you probably didn't want it set to 700 root:root out on the host, too. ... (check-in: 260efac1 user: wyoung tags: trunk)
Assorted cleanups to the Dockerfile: disentangled use of /tmp for building and creation of the eventual contents of /etc, dropped an unnecessary directory hierarchy level, removed an unused variable, whitespace fixes, etc. No functional change. ... (check-in: f9bd6273 user: wyoung tags: trunk)
Typo squish in the server doc section's index page ... (check-in: 4158e5e2 user: wyoung tags: trunk)
Made the URL matcher more specific in the nginx doc when showing how to get long expiration times on URLs known to yield unchanging content so we can drop the warning about it affecting URLs more likley to change. Added a second example for images, CSS, JS and such to show how you might handle these differently, addressing the core issue by example than by prose warning. ... (check-in: f77ab197 user: wyoung tags: trunk)
Added a bit more advice about caching to previous, and fixed a whitespace problem. ... (check-in: 4d5a6477 user: wyoung tags: trunk)
Improved the caching configuration recommended in the nginx doc to show the option for immutable content service for /artifact, /doc, /file, and /raw URLs, which are likely to contain hashes that make the URLs unique identifiers for particular versions of the referenced content. ... (check-in: 1d0d6986 user: wyoung tags: trunk)
Tiny tweaks to the systemd doc ... (check-in: 0835ec45 user: wyoung tags: trunk)
Added a paragraph to the unvers doc on the "y" capability. ... (check-in: faffd271 user: wyoung tags: trunk)
Modernized the URL format in the unvers doc: using an actual (reserved) domain name instead of the vague "domain", showing it as "https" since "http" is widely deprecated, and using "tt" tags around it to make it stand out from the surrounding sans-serif text and to prevent it from breaking across word-wrap boundaries. ... (check-in: d1a3dda6 user: wyoung tags: trunk)
Edit pass on the doc, primarily to replace passive voice phrasing with active voice. Made a few clarifications and grammar tweaks atop that. ... (check-in: fef0ade9 user: wyoung tags: trunk)
Converted the hand-crafted footnotes in the "Image Format vs Fossil Repo Size" doc to use the new Markdown affordance. ... (check-in: 389e3fb9 user: wyoung tags: trunk)
Improved the systemd guide's points about `podman generate systemd`. ... (check-in: ac30c2d9 user: wyoung tags: trunk)
A few more small tweaks to the containers doc ... (check-in: 117cf4ff user: wyoung tags: trunk)
Reordered a few sections in the container customization section to flow better and reduce redundancy. ... (check-in: bb189d17 user: wyoung tags: trunk)
Moved the sectio about elaborating the container runtime layer down into the section of the doc where we talk about other customizations. Its prior location was because it was a tangent off a prior point, but it's just as easy to jump down via hyperlink. Assorted other small improvements while in there. ... (check-in: 301d4f21 user: wyoung tags: trunk)
Added the CENGINE abstraction between `docker` and `podman` in the Makefile's container convenience targets. ... (check-in: 72d82404 user: wyoung tags: trunk)
Removed all the "BBXVER" stuff in the containers doc now that we aren't fetching the BusyBox sources and building a custom version to install outside the jail. ... (check-in: b1416554 user: wyoung tags: trunk)
Updated the Podman docs to no longer talk about all the "sudo" stuff we used to have to do to get it to build and run. There are no more mknod calls to fail in that rootless environment. ... (check-in: 779cb8fd user: wyoung tags: trunk)
Updated the container size info in the Fossil v Git doc to track recent developments. ... (check-in: 283b1a42 user: wyoung tags: trunk)
Commit [cda5d6a7] invalidated the BusyBox steps in the Tcl and Python examples in §3.2 of the containers doc. ... (check-in: ddcdc6f3 user: wyoung tags: trunk)
Using "FROM busybox" in the second stage of the Dockerfile instead of "FROM scratch" plus a copy of the busybox.static binary installed via APK in the first stage. We're throwing this layer away in the third stage, so the difference is immaterial. This simplifies things without losing anything we care about. ... (check-in: cda5d6a7 user: wyoung tags: trunk)
Post-sleep edit pass on the new material in §3.2 of the containers doc. ... (check-in: d21fb267 user: wyoung tags: trunk)
Refined the Tcl and Python examples in the new §3.2 of the container doc. ... (check-in: 9baa4423 user: wyoung tags: trunk)
Switched to a split ENTRYPOINT/CMD scheme for launching the Fossil server in the container. The immediate need is so we override lower-level ENTRYPOINTs provided by mix-in layers, but it's more correct generally. ENTRYPOINT says this is the hard-coded purpose of the container, and CMD gives the arguments to that command. The split is therefore between the mandatory parts and the parts the user might want to override without needing to write their own Dockerfile. ... (check-in: deb99e22 user: wyoung tags: trunk)
The container now uses BusyBox only in the build and setup stages, leaving just the static Fossil binary in the final stage, plus absolute necessities like a /tmp directory.

This removes the justification for the custom BusyBox configuration, which then means we can use Alpine's busybox-static package in the second stage, saving a bunch of network I/O and build time.

That in turn means we no longer have any justification for jailing the Fossil binary, since there's nothing extra left inside the container for it to play with. Doing this required bumping the Dockerfile syntax back up from 1.0 to 1.3 to get the "COPY --chmod" feature; tested it in Podman, which has had it for two years now.

Doing all of this simplifies the Dockerfile and its documentation considerably. As a bonus, it builds quicker, and it's nearly a meg lighter in compressed image form. Especially for the case of using the container as a static "fossil" binary builder, this is nothing but win. ... (check-in: 79ac06a5 user: wyoung tags: trunk)

Comment and whitespace tweaks ... (check-in: 81c30ab9 user: wyoung tags: trunk)
Switched from a Dockerfile "ADD" command to wget for the BusyBox source tarball because, surprisingly, BuildKit pulls the URL unconditionally under the logic that it can't know whether to cache the pulled data until it has a copy to compare against! This not only means you pull the BusyBox source tarball for each container build even though it's tagged and thus cannot possibly change, it puts a load on GitHub which then causes it to begin throttling each pull, making your local builds slower and slower when iterating on a change set, as in the prior set of commits. By pushing the URL down into a wget command, we cause BuildKit to see an unchanging shell script line (assuming $BBXURL keeps its default) so it *does* cache the pulled layer. ... (check-in: ac955594 user: wyoung tags: trunk)
Another fixup to the nojail patch to track the previous. (Can't reliably create these patches without having a committed version to diff against, alas.) ... (check-in: c9e4b3d2 user: wyoung tags: trunk)
Dropped our canned /etc/os-release file entirely, recommending instead that those who need a VM-like container image switch the second stage from "scratch" to one of Google's "distroless" images, which provide that and more. That in turn gets rid of the need for the dummied up /usr/bin and /run, which simplifies the mainstream case. ... (check-in: d778a023 user: wyoung tags: trunk)
Updated the nojail patch so it applies cleanly atop all these recent Dockerfile changes. No functional change; merely tracks changes in the context parts of the diff. ... (check-in: 2bdd5819 user: wyoung tags: trunk)
Tiny clarity tweaks to the Dockerfile. No functional change. ... (check-in: 591e3eb9 user: wyoung tags: trunk)
Removed a reference to /etc/os-release from stage 2 of the Dockerfile. Commit [4cb5c03e] took care of stage 1 only. ... (check-in: 4b41a7f8 user: wyoung tags: trunk)
Switched from "adduser" and "addgroup" commands for setting up the "fossil" user to direct echo-into-output, same as we already do for the root user. We had to to it for root since the BusyBox implementation of adduser/addgroup won't create these files if they're missing, but that meant we had two different ways of creating users and groups. This not only removes a weak dependency, it's more consistent. ... (check-in: fff11fc6 user: wyoung tags: trunk)
Added the interactive debugging shell command to the Quick Start section of the containers doc for easy cut-and-paste. ... (check-in: 2f014407 user: wyoung tags: trunk)
URL and whitespace fixes to previous. ... (check-in: 9e73519c user: wyoung tags: trunk)
The /etc/os-release workaround for nspawn's pickiness has caused the feature to go into negative ROI territory. Ripped it out of the mainstream process and made it a manual step for those who need it, in the hopes that this will cause fewer ongoing problems than leaving it as it is. ... (check-in: 4cb5c03e user: wyoung tags: trunk)
Dropped declaration of Dockerfile syntax version from 1.4 to 1.0. Put it at 1.4 when we were using heredocs, a feature that went from experimental to stable at that version, then failed to drop it back when we replaced the use of heredocs with externally generated files to regain Podman compatibility. ... (check-in: 5b62bfe1 user: wyoung tags: trunk)
Linked to the Dockerfile from the top of the containers doc. ... (check-in: 2210c15d user: wyoung tags: trunk)
Renamed the new "Capabilities" glossary entry to "Capability" since we shouldn't be using a plural top-word entry even though they're defined, transported, stored, and otherwise treated as a group. Also replaced a use of this word in its own definition. ... (check-in: d3f45814 user: wyoung tags: trunk)
Moved the "snapshot" term in the glossary down into a footnote because it's got assorted problems, making it a much worse overall synonym for "version" even than "UUID". ... (check-in: 733ef88a user: wyoung tags: trunk)
Expanded the "version/revision/UUID/snapshot" discussion in the glossary into a separate term since these aren't strict synonyms for "check-in", the definition which previously hosted this topic. ... (check-in: 58030a78 user: wyoung tags: trunk)
Added a new glossary item "Capabilities" to introduce the term and distinguish it from "Permissions", and made several changes to the referenced document to reinforce this distinction and explain why we bother to make it. ... (check-in: 23b91f37 user: wyoung tags: trunk)
The recommendation to configure Fossil with the --static flag is semi-obsolete, and the following advice to look further down in the same document for the Docker workaround was wholly obsolete since moving all of this into the dedicated doc. Fixed all this up, and linked to the "why" answers on Stack Overflow about all of this in a few more places. ... (check-in: d282e42c user: wyoung tags: trunk)
The /zip and /tarball built-in help now makes clear that the VERSION/ part of the URL is optional to help avoid confusions like we're seeing in the forum post that sparked this sequence of improvements. ... (check-in: 4717db33 user: wyoung tags: trunk)
Clarified one of the 2.21 changelog entries. (Started as fixing a typo.) ... (check-in: 86c4e6bf user: wyoung tags: trunk)
Reverted half of commit [4ad86dd5]: it incorrectly moved a CSS style instead of copying it to where it also needed to be. The user-visible effect was that centered Pikchrs varied in size according to their size and complexity, which meant that elements that should've been the same size weren't. ... (check-in: 5ad62aba user: wyoung tags: trunk)
Cleaned up a few inconsistencies in the Pikchrs in the branching doc in an attempt to fix the smaller-and-smaller diagram size problem currently occuring in this doc. ... (check-in: 239fb5b1 user: wyoung tags: trunk)
Updated the PBKDF2 recommendations in the backup doc to track recent changes in best practice due to all these GPU computing fleets coming online. Added a few paragraphs explaining the limits to all of this and why we chose the passphrase lengths we did as examples. ... (check-in: 6a3d6fa6 user: wyoung tags: trunk)
Wrapping a few calls to vfile_check_signature() from the new local diff code in unprotect/pop call pairs to squish a DB protection error. ... (check-in: 1b3ef05e user: wyoung tags: ui-local-diff)
Brought the ui-local-diff branch up to date relative to trunk. It isn't a simple merge, primarily due to all the changes to /vdiff and /fdiff made over the past 2 years. It seems to work as well as it originally did, but it isn't ready to merge down to trunk as-is. ... (check-in: 76fa1657 user: wyoung tags: ui-local-diff)
Replaced a standalone "diffFlags" variable in the /fdiff handler with use of the new DiffConfig.diffFlags member. No functional change, just a code cleanup found while working on another branch. Making it on trunk to keep that branch's diffs minimal. ... (check-in: 65d97f23 user: wyoung tags: trunk)
Small fix to the nojail patch; accidentally lost the [80faedbc] change in the shuffle. ... (check-in: 7a6cf9dd user: wyoung tags: trunk)
Removed the two "mknod" calls from the Dockerfile in the nojail patch used by Podman rootless containers. Not only is the build user not allowed to run mknod in that case, there will be a /dev tree mapped into the container, causing the commands to fail due to these two basic dev nodes preexisting. ... (check-in: d97a8fb1 user: wyoung tags: trunk)
No longer running "fossil" with a relative path ("bin/fossil") at the end of the Dockerfile, but instead relying on the hard-coded PATH defined a few sections prior. This allows the same command to work for both the rootful and rootless cases since moving the binary into /usr/bin/fossil to placate nspawn. Before, it was /jail/bin vs /bin, so the difference netted out to nothing. ... (check-in: 80faedbc user: wyoung tags: trunk)
Fixed a copy-paste error in the Podman sections of the container doc: was using "docker" commands instead of "podman" in a few places. That'll work for people who aliased them, but it's confusing. ... (check-in: 6eefa9b0 user: wyoung tags: trunk)
Removed use of UPX in the container build process. It complicates the build for a tiny gain while breaking ARM builds. We worked around the ARM-on-ARM case earlier, but it also breaks x86 cross-compilation on ARM. Images are already compressed, and while `upx -9` is stronger compression than whatever Docker Engine is using, it's a small advantage. This does mean the static executable isn't compressed any more on x86, but if you want that, you can UPX it afterward. ... (check-in: da545c9e user: wyoung tags: trunk)
Generating the /etc/os-release file for the OCI container using autosetup at configure time rather than from a build arg in the Dockerfile at image creation time. This lets us back out the use of heredocs in the Dockerfile, which isn't supported in Podman at all as of this writing and under Docker requires use of BuildKit rather than the legacy "docker build" mechanism.

The primary consequence of doing it this way is that the Fossil version number in that generated file becomes the configure-time version, unconditionally. The old way let you override the FSLVER variable at image build time and have that value put into the os-release file. Under this new scheme, you now have to run "/jail/bin/fossil version" to find out what version of Fossil got baked into the image. ... (check-in: ec8ef573 user: wyoung tags: trunk)

Modernized several old URLs, changing "http" to "https" where absolute URLs are necessary, and using site-relative URLs otherwise. Also found and fixed a reference to, which doesn't seem to resolve any more. ... (check-in: 143f1db7 user: wyoung tags: trunk)
Removed pointless "udc=1" parameters from a few Fossil file links from the docs. ... (check-in: 40d912ae user: wyoung tags: trunk)
Added named anchors to the "Image Format vs Fossil Repo Size" doc so I can refer to one in particular. ... (check-in: 7de2410f user: wyoung tags: trunk)
Updated the macOS sidebar in the doc to cover Ventura. ... (check-in: a55042a0 user: wyoung tags: trunk)
Grammar and spelling fix pass on the new nspawn material in the containers doc. ... (check-in: 5405aa57 user: wyoung tags: trunk)
Typo fixes ... (check-in: 00e4d91e user: wyoung tags: trunk)
Assorted prose polishing in the new systemd-container section at the end of the containers doc. ... (check-in: 120a2076 user: wyoung tags: trunk)
Added a few more "container-*" targets to the main makefile to simplify the examples in the containers doc and make the resulting images and containers easier to manage. ... (check-in: b7edb5f1 user: wyoung tags: trunk)
Merged two redundant discussions of the consequences of disabling private network virtualization under systemd-container infrastructure, then added better reasons why the reader might care. ... (check-in: 70554336 user: wyoung tags: trunk)
Updates to the systemd service doc, primarily to refer the reader to the new containerized runner methods, but also to add other tips. ... (check-in: ad09d3ee user: wyoung tags: trunk)
Updated the nojail patch so it'll apply atop the new Dockerfile changes. ... (check-in: 45e0475c user: wyoung tags: trunk)
Worked out how to get systemd-container (a.k.a. nspawn + machinectl) working with the stock Fossil container. Following the above commits, it's pure documentation. Removed the runc and crun docs at the same time since this is as small as crun while being more functional; there's zero reaon to push through all the additional complexity of those even lower-level tools now that this method is debugged and documented. ... (check-in: 930a655a user: wyoung tags: trunk)
Added empty /tmp and /run directories to the "OS image" layer of the stock container in case someone is mounting the base layer read-only with tmpfs mounted atop these points. (Seen with "systemd-nspawn --read-only" but might affect other runtimes.) ... (check-in: 0733be50 user: wyoung tags: trunk)
Container build changes to allow systemd-nspawn to recognize it as an "OS tree:"
  • Added a dummied-up /etc/os-release file
  • Moved several programs from /bin to /usr/bin, since existence of /usr is how it decides if the rootfs you point it at contains an OS image. Bogus, but that's how it is.

Had to switch to buildx to make this work, so I could use heredocs in the first step. ... (check-in: f74ddbce user: wyoung tags: trunk)

Added "container-clean" target to cleanup after the other container-* targets. ... (check-in: e119d598 user: wyoung tags: trunk)
Tried to get "--with-tcl=1" working in the containerized build, but failed, so I documented the reason why it isn't going to work given our current design goals and pointed at an alternative with different tradeoffs. ... (check-in: fb1bfce1 user: wyoung tags: trunk)
Added the FSLCFG Dockerfile build arg and showed how to use it in the containers doc, plus other improvements to the doc while in there. ... (check-in: e2277aad user: wyoung tags: trunk)
Put a "sleep 1" into "make container-run" before the step that shows the container logs to ensure we show everything it says on startup. Added this on seeing just the first line of output due to a race condition, so I missed the generated admin password. ... (check-in: 4429e10f user: wyoung tags: trunk)
The "container-run" target now runs "container-image" conditionally, building it only if it wasn't created in a prior step. This allows custom image builds followed by a one-command way of running that built image. Without this, the custom image gets stomped on. ... (check-in: a9e862b8 user: wyoung tags: trunk)
Reverted the build hack to strip out all but the default and darkmode skins in the stock Dockerfile. That was done to cater to a wish for extremely small ARM builds, for fun, not for any practical reason. It conflicts with a key philosophy behind this container project, to create stock Fossil builds by default. "make container-image" should get you a functionally identical binary inside the container as "./configure && make" does outside it. ... (check-in: 3e95d945 user: wyoung tags: trunk)
Prefixing each shell script section in the Dockerfile with "set -x" broke the checks to prevent running UPX on ARM builds. You can still get release container builds on ARM by copying this fixed Dockerfile to your release checkout. ... (check-in: b4c3d9a1 user: wyoung tags: trunk)
Also documented the new "clone -u -v" feature. ... (check-in: 0d61fd23 user: wyoung tags: trunk)
Since it seems my clone -u fixes are going to stick, documented them in the changelog. ... (check-in: 02631e35 user: wyoung tags: trunk)
The check for whether to continue during sync due to outstanding "uvgimme" requests was being skipped in clone -u mode due to misordered tests at the end of the client side of the sync protocol. ... (check-in: 52648d03 user: wyoung tags: trunk)
Since "fossil uv sync -v" turns on UV trace mode, made "fossil clone -u -v" enable that mode as well, since otherwise there's no way to get into UV trace mode during clone. (e.g. There is no global "--uvtrace" option.) ... (check-in: cdd58b1f user: wyoung tags: trunk)
Consolidated two related tests in the sync protocol to avoid re-testing a flag twice and to bring related code closer together. ... (check-in: 6293b282 user: wyoung tags: trunk)
Corrected a difference in the case of a SQLite table name. The DBMS doesn't care, but it risks missing relevant references to this table when searching with a case-sensitive text editor. ... (check-in: 1b1887cb user: wyoung tags: trunk)
Typo fix in the 2.20 changelog ... (check-in: c3012508 user: wyoung tags: trunk)
Grammar fix ... (check-in: 658547aa user: wyoung tags: trunk)
Assorted fixes and improvements to the doc ... (check-in: 27458ef7 user: wyoung tags: trunk)
Updated the debian/ doc for Ubuntu 22.04. The biggie is simplifying the TLS configuration, since the manual method we used to have no longer seems to be required with current versions of Certbot. ... (check-in: 716ae7c0 user: wyoung tags: trunk)
Fixed a few references to the obsolete doc. (It became part of the overall server doc long ago.) ... (check-in: 780b58bc user: wyoung tags: trunk)
Assorted updates surrounding my fslsrv wrapper:
  • Reflected improvements from the version into this simpler alternative. Although we don't generally recommend use of this script any more, preferring systemd to get autostart on boot and autorestart on crash, www/server/any/ still refers to this script, and it feels like a regression to remove it. If someone is interested in simple-as-possible SCGI service, fslsrv is a fit companion.
  • Removed direct reference to fslsrv from www/server/debian/ since the indirect reference via the SCGI doc suffices.
  • The full-strength nginx doc now refers to both of these fslsrv variants in a handwavy way, since it's outside the scope of that doc to care how you get your background SCGI servers running.
... (check-in: 1cbcb38c user: wyoung tags: trunk)
Added hyperlinks to the new changelog entries referencing the files in question. ... (check-in: 2c127ba7 user: wyoung tags: trunk)
Closing off the containers project: added the doc to the permuted index, noted the changes in the changelog, and removed all the hedging about WAL mode in the doc, having failed to make WAL fail in this scenario. ... (check-in: 92982dc4 user: wyoung tags: trunk)
Replaced most of the speculation in the walmode section of the containers doc with a link to the walbanger project, where we'll be answering this question. ... (check-in: 96633067 user: wyoung tags: trunk)
Mentioned containerd+nerdctl in place of runc in the containers doc. A tightened-up version of the prior runc and crun sections are now collected below the Podman section. This gives a better flow: each successive option is smaller than the last, excepting only nspawn, which is a bit bigger than crun. (We leave nspawn last because we can't get it to work!) ... (check-in: 457c14a4 user: wyoung tags: trunk)
Updated the "nojail" patch for our Dockerfile to track the recent changes: rename back from and the layer refactoring. It does essentially the same thing as before. ... (check-in: 19abf0ac user: wyoung tags: trunk)
Broke the Dockerfile up into more layers to allow better local caching at build time. Further optimized build time by producing the Fossil source tarball from the local repo instead of hitting the home site if you use the container-image target, since we can be reasonably certain you're working from a repo checkout and thus have all the info available here locally already. ... (check-in: 1da464ee user: wyoung tags: trunk)
Expanded the paragraph on WAL mode interactions in the container doc into a full section, placed higher up, immediately after the first use of Docker's "--volume" flag, to explain why we don't map just the repo DB file, but the whole directory it sits in. Even if we later convince ourselves WAL is safe under this scenario, it'll be conditional at best, so some remnant of this section must remain, no matter which way the experiments go. ... (check-in: 698587d4 user: wyoung tags: trunk)
Renamed back to Dockerfile so it can be used as-is on non-autosetup systems. Realized that we can pass the Fossil checkin hash prefix in as a build arg instead of regenerating the file on disk from auto.def. If you use the Dockerfile as-shipped, you get a "trunk" build, which risks a stale cache — it thinks it already has a tarball by that name and helpfully refuses to pull it again — but at least Windows users get *something* without hand-hacking the file. ... (check-in: b0c9c26a user: wyoung tags: trunk)
Added a /jail/log directory to the container so someone can pass --errorlog and such to the Fossil instance and have a place to put it. It also acts as a mountpoint for appending to a log out on the host. ... (check-in: ed50ceee user: wyoung tags: trunk)
/dev permissions were too tight in the container. They're still tighter than on a stock Ubuntu box, but they should suffice for Fossil's needs. ... (check-in: 8eeb95e1 user: wyoung tags: trunk)
Restricted the container listeners to localhost in section 6 of the containers doc, and mentioned a few other items related to reverse proxying with nginx. ... (check-in: c9ab736f user: wyoung tags: trunk)
Folded info from an exchange with the Podman devs into the container doc. ... (check-in: 80f4a1dd user: wyoung tags: trunk)
Added section numbers to the containers doc (it was getting confusing) and added a few internal fragment IDs. ... (check-in: 4d51d524 user: wyoung tags: trunk)
Finished all the new topics planned for the new containers doc, adding sections on rootful Podman containers and on building via Docker but running via Podman, using Docker Hub as an intermediary to avoid building on the remote host. ... (check-in: 9c96e499 user: wyoung tags: trunk)
Sanitized a local port number out of previous ... (check-in: 3dfa4581 user: wyoung tags: trunk)
Added my sad tale of failure and woe with systemd-nspawn to the container docs, both as a warning to those who follow, and as a cry for help to someone who can make this work. I can't be bothered to spend more time on it, but there's no point throwing the work away. ... (check-in: 1e8c6655 user: wyoung tags: trunk)
Documented another cause to modify the "m" variable in the runc examples in the container docs. ... (check-in: bf503088 user: wyoung tags: trunk)
Added more jq filters to the runc examples to remove further problematic things left in the automatic conversion from the Docker container configuration file to the one we provide to runc. ... (check-in: 4e8c7479 user: wyoung tags: trunk)
Worked through some difficulties here in applying the runc method on remote systems, then documented what I learned in the containers doc. ... (check-in: 56f4e2ce user: wyoung tags: trunk)
Small fix to previous ... (check-in: d5695c8e user: wyoung tags: trunk)
Expanded the runc section of the container doc to cover "bundle" terminology and to show a method for rsyncing the bundle across to a remote host. Also explained why this is a bad idea unless you've got a rather constrained use case, lest people avoid using podman/docker in places where they could provide real value. ... (check-in: f9f13ce7 user: wyoung tags: trunk)
Documented the runc and crun options for running a container, including the cryptic method for exporting an OCI bundle from Docker, allowing you to use both together: Docker Desktop on your big dev box in the office, then one of the two lightweight runtimes out in the cloud. ... (check-in: c9431ef4 user: wyoung tags: trunk)
Added explicit instructions for patching the Dockerfile for the nojail/podman method and for mapping a single Fossil repo into the container rather than a directory. Also included my best current advice on using WAL mode in these contexts. ... (check-in: 87a23d2a user: wyoung tags: trunk)
Removed a TODO-based section of the new containers doc that wasn't meant to be checked in yet. Made a few improvements to the new Podman material as well. ... (check-in: 5adf6c40 user: wyoung tags: trunk)
Added the "Lightweight Alternatives to Docker" section to the new containers doc, currently limited to a tutorial on converting the stock Dockerfile to work under Podman in its default mode, creating a rootless container. This brings in the second container-related file at the root of the repo, the patch file for this, so we don't have to maintain two nearly-parallel Dockerfiles. As a bonus, it allows us to point to the patch from the prose, making explicit what we had to change. ... (check-in: f0399ea9 user: wyoung tags: trunk)
Moved the busybox-config file from tools/ into a new containers/ subdirectory. We were using that as a junk-drawer directory, for lack of a better place to put it. Now that we're about to have a second container-related file in the repo, that weak excuse is wearing thin. ... (check-in: b08e2bb7 user: wyoung tags: trunk)
Referencing the new file from so we can remove a big redundant block comment from it. While in there, made a few style tweaks that will help the ongoing container document expansion. ... (check-in: be8f721d user: wyoung tags: trunk)
Extracted the Docker containers material from www/ and moved it into a new document dedicated to the topic, It was already pushing the bounds of how much info we want to provide in a single section of that doc, and it's about to get bigger.

As part of the conversion from wiki format to Markdown, did another edit pass on the doc, improving a few things along the way.

Dropped the "docker-" prefix from all internal IDs, as we no longer need them to disambiguate references to other parts of the build doc. ... (check-in: 7129dc98 user: wyoung tags: trunk)

Embroidered the "make container-run" target to make it more convenient. ... (check-in: bc09e28a user: wyoung tags: trunk)
The container doc bit on raw sockets now covers the other three Busybox utilities we left out previously. Today's removal of ping and traceroute merely completes the set; it wasn't complete in itself. ... (check-in: b429bd71 user: wyoung tags: trunk)
Clarified the points in §5.2.1 of the Docker container build doc regarding the reason why the server parent process runs as root. ... (check-in: c2eaa60d user: wyoung tags: trunk)
Researched, tested, and documented the set of "docker create --cap-drop" options we can add to strip away unnecessary root privileges inside the container without harming normal operation. Belt-and-suspenders: if any bad actor ever got into the container with root privileges, this would help prevent them from affecting anything outside the container. Added that set to the "make container-run" target so they get applied by default in the easy case. ... (check-in: f715add9 user: wyoung tags: trunk)
Removed ping and traceroute commands from the Docker container. They require raw sockets support, which means if anyone broke into the container and managed a root privilege escalation, they could do a wide array of bad things on any network the container is bound to. ... (check-in: f00a88f8 user: wyoung tags: trunk)
Polishing pass on §5.2 of the container build doc, "Why Chroot?" ... (check-in: e9860314 user: wyoung tags: trunk)
Clarified the parent process user ID vs the child process in the explanation of how the chroot feature interacts with the custom user feature of the Docker container. ... (check-in: f9ddd38e user: wyoung tags: trunk)
Made a better distinction between bind mounts and Docker volumes in the new Docker section of the build doc. ... (check-in: 958a6af9 user: wyoung tags: trunk)
Removed a digression in the gitusers doc about Fossil's new clone-and-open mechanisms. That got moved to the ckout-workflows doc quite some time back, and we already point to it from that same section. There's no reason for the redundancy. Also cleaned up some grammar and typos while in there. ... (check-in: f43eaf01 user: wyoung tags: trunk)
Changed the "fossil server --user" flag's argument back to "admin" from "fossil" for the container: I was confusing the Unix user name with the default Fossil repo user name. The new "adduser fossil" stuff doesn't help here; we still want it to be called "admin". ... (check-in: 72d820f3 user: wyoung tags: trunk)
ARM build fixes for the container:
  • QEMU couldn't cope with "make -j" on the BusyBox step (too many processes) so I changed it to -j11
  • Made the new executable compression step conditional, since there is no upx package in Alpine for either ARM flavor. There's a long bug thread for it on GitHub, which doesn't look to be getting resolved any time soon.
... (check-in: 8849abb7 user: wyoung tags: trunk)
Minor fixes to the Docker container build process ... (check-in: 454397b0 user: wyoung tags: trunk)
URL fix necessitated from the rename ... (check-in: 2f67bf94 user: wyoung tags: trunk)
Carved the Docker container image size down still further by stripping out all but two of the stock skins (d* so we get default and darkmode) and packing Fossil and BusyBox with UPX. ... (check-in: e20d044c user: wyoung tags: trunk)
Fixed an Obi Wan error in the new Fossil version prefix stuff in auto.def: it was extracting the first 13 characters of the hash, not the first 12. ... (check-in: 7ecd23e0 user: wyoung tags: trunk)
Added the container-image and container-run top-level build targets to manage dependencies better and to auto-version the build products. ... (check-in: 67386c75 user: wyoung tags: trunk)
Put the "--user fossil" bit back into the fossil server command for the container. Just ran into a situations where it's still needed. ... (check-in: 4c8cc804 user: wyoung tags: trunk)
Polishing pass on the container repo storage section of the build docs. ... (check-in: 3e332637 user: wyoung tags: trunk)
Changed several of the Docker environment variables to build arguments so the user an override them at build time rather than container creation time, and documented them in Using this new mechanism to pull the Fossil source tarball in such a way that we can use the Docker artifact cache without getting stale builds. You can now pass one of the new build args to force the old behavior if you want it. This required generating Dockerfile from at configure time, to inject the current Fossil checkin ID. (This busts the Docker cache when the source tree changes.) ... (check-in: f9384383 user: wyoung tags: trunk)
Adding the BusyBox tarball to the container image with an ADD command rather than wget to avoid triggering GitHub throttling. Unlike the Fossil repo URL, it has a version number baked into it, so it's safe to give it over to Docker's caching behavior. ... (check-in: d06d7c46 user: wyoung tags: trunk)
Noted the container size shrinkage in the fossil-v-git doc ... (check-in: f21de33e user: wyoung tags: trunk)
The container now builds Busybox from source so we can remove utilities that are unhelpful inside the container. We leave a lot behind for expansion (e.g. the runit init system, crond, inetd…) but we remove things that have no possible justification, such as modprobe. We remove everything from /bin that's a shell builtin (echo, printf, test…) and we replace a few BusyBox commands (sha[13]sum) with wrapper shell scripts that call Fossil builtins. We cap that off by adding a "sqlite3" wrapper that calls "fossil sqlite3 --no-repository", just for fun. All together, this trims about a meg of fat. ... (check-in: 953f367e user: wyoung tags: trunk)
The chown -R bit added to the Dockerfile touches /jail/bin/fossil, which causes "docker build" to promote it back into a new layer, nearly doubling the container size. Doing a chown now only on two directories, restoring it to its sub-9M size. ... (check-in: 00cc9c3e user: wyoung tags: trunk)
Fossil's chroot feature drops root permissions based on file ownership, but since the container was built with everything-root, its HTTP hit handling children would run as whatever host-side UID/GID pair you used for file ownership. What happened next was complex.

If you let the container create the repo internally, it would be owned as root, so it would drop root permissions for…root! This isn't super-bad, since Fossil is presumed secure and is double-jailed besides. The risk is, if anyone works out an RCE for Fossil, they might be able to get it to create raw sockets or do various other types of escapes despite the double-jail dance.

Attaching a Docker volume brings external permisssions into the container. We were recommending a "chown 0" command on the shared volume to make it similar to the in-container case, but that opens you to the same risks above. If you ignored this and used host-side UID/GID pairs, Fossil would then be left running under IDs that didn't exist internally, which could cause assorted weirdness.

We're now creating an explicit "fossil" user/group pair inside the container and recommending that Docker volumes use these IDs for copied-in files to batten down something that shouldn't've been left flapping.

Updated to cover all this. ... (check-in: ba21bc0b user: wyoung tags: trunk)

Moved the SIGTERM handler up before the "fossil server" HTTP hit handler. We had it clustered with the other signal() calls, but those are to handle signals intended to occur only during CGI processing. This one will normally occur while we're blocked, waiting for the HTTP hit to occur, so it had no useful effect where it was. ... (check-in: d3c55fe0 user: wyoung tags: trunk)
Changed previous to call fossil_exit() instead of exit(3) so we close our databases before dying. ... (check-in: 7c857d22 user: wyoung tags: trunk)
The parent process now handles SIGTERM with an explicit exit(3) call when its PID is 1, as when it's running as "fossil server" in a Docker container. Without this, the container host's shutdown process takes a long time because it's waiting on PID 1 to die and eventually has to time out and kill it. ... (check-in: 1d09e607 user: wyoung tags: trunk)
Markup fix ... (check-in: cf149787 user: wyoung tags: trunk)
Clarified the fact that the "docker cp" command is changing the name of the repository DB file. ... (check-in: f0b15a37 user: wyoung tags: trunk)
Slight emphasis fix in previous ... (check-in: 1441c2e6 user: wyoung tags: trunk)
Edit pass on §5.1 of, fixing a number of unclear bits, particularly with regard to images vs containers. ... (check-in: e2b9114b user: wyoung tags: trunk)
Using the preceding --chroot fixes to make the Docker container serve the repo from /jail/museum/repo.fossil rather than from the chroot dir, /jail. This then allows us to mount a Docker volume at /jail/museum, which has an independent persistence from the container proper, so we can now rebuild the container without destroying the presumably precious repo. Updated to track this change and document the lessons gleaned from doing all of this. ... (check-in: f76e762f user: wyoung tags: trunk)
Moved the chdir() call within enter_chroot_jail() down below the new repo name canonicalization code to allow use of relative path names. Before, you had to give an absolute path to the repo, since we'd cd'd away from that directory before we started to validate the path. ... (check-in: e9462118 user: wyoung tags: trunk)
Moved the setting of g.fJail flag into the repo = "/" case since it exists only to communicate the chroot status to --repolist mode. (This confirms the speculation in the prior commit's comment: the prior behavior existed to serve repolist mode only.) ... (check-in: 324d232c user: wyoung tags: trunk)
Fixed the --chroot flag to "fossil server" and "fossil http" to allow it to work in conjunction with the single-repository case. Before, it blindly assumed --repolist mode. ... (check-in: 6f92ad99 user: wyoung tags: trunk)
Fixed pointless use of interwiki link in the new section 2.2 material of fossil-v-git. ... (check-in: 73c95307 user: wyoung tags: trunk)
Fixed a few stray parens in the new material in the fossil-v-git doc, left behind from a prior edit. ... (check-in: ea13701c user: wyoung tags: trunk)
Typo fix ... (check-in: b628a883 user: wyoung tags: trunk)
Fixed a problem in image naming in the new Docker container doc in reported on the forum. ... (check-in: 509447a2 user: wyoung tags: trunk)
Did away with the temporary src.tar.gz file in the new Docker container by streaming the output of wget straight into tar's stdin. This cuts the build time by about five seconds, presumably due to the saving from unnecessary file I/O. Also replaced the explicit "cd src" afterward with an out-of-tree build configuration, since it doesn't matter if we clutter the first stage's /tmp dir. ... (check-in: 289c9b50 user: wyoung tags: trunk)
The build docs for "./configure --static" now reference the section further down on Docker, since you may need to use this indirection to get --static to produce something suitable. ... (check-in: 7bfd7413 user: wyoung tags: trunk)
Replaced Jan Nijtman's Dockerfile with a new one that does a 2-stage build. The first stage runs atop Alpine Linux instead of Fedora, reducing the initial build from ~635 MiB to about 16.

Rather than stop there, I then made it multi-stage, copying two key static binaries — Fossil and Busybox — over from the first stage into a fresh-from-scratch container and set it up to run the former jailed away from the latter.

The result is under 9 MiB, and it's as secure as one can hope, given that it starts up in "PUBLIC" mode. The new build doesn't have all the extra features turned on that the old one did, but it seems right to build the container with Fossil in its default configuration. If you want something else, copy the Dockerfile, hack it, and make it do what you want instead.

Having done all this, I replaced the one-off Dockerfile inline in section 5.0 of the build doc with a reference to this new Dockerfile and rewrote the section in terms of the new capabilities.

Finally, this lets us brag on how small the container can be, as compared to the Gitlab-CE container. Before, we were comparing a standalone binary to the container, which wan't entirely fair. (The desire to produce such a container was the spark that kicked this project off.) ... (check-in: 77d603c6 user: wyoung tags: trunk)

Assorted improvements to the first few sections of the fossil-v-git doc, mainly in updating them to track changes to world facts and to clarify the presentation. ... (check-in: c7afd68b user: wyoung tags: trunk)
Still moer grammar fixes in fossil-v-git ... (check-in: e28c25e4 user: wyoung tags: trunk)
More grammar fixes ... (check-in: 9f135f2f user: wyoung tags: trunk)
Grammar fix to the fossil-v-git doc ... (check-in: f36fb951 user: wyoung tags: trunk)
Changed a number of "a" articles followed by vowels in docs and comments to "an", per a forum post. ... (check-in: 99a319bd user: wyoung tags: trunk)
Mentioned "fnc stash" at the end of the section of gitusers where it talks about alternatives to "git add -p" and such. ... (check-in: b3b2c1ab user: wyoung tags: trunk)
Continued the edit pass on the main body of the gitusers doc, shy of the case studies, mainly doing minor style tweaks. Biggest substantial change is to rewrite the colorized diff section to cover the changes in Fossil 2.17, and to present the alternatives in a more logical order. ... (check-in: c026fb9a user: wyoung tags: trunk)
Fix to the fix. :( ... (check-in: e3f9584e user: wyoung tags: trunk)
Fixed a broken internal link in the gitusers doc resulting from moving the museum tree pikchr into the glossary. ... (check-in: 116d8c75 user: wyoung tags: trunk)
Rewrote the login-groups doc, making it both more clear and more detailed. This started out as clarifying a confusion brought up on the forum, but experimentation kept bringing up new and interesting restrictions and interactions that I felt were worth documenting. ... (check-in: 697cf6fb user: wyoung tags: trunk)
Brought the "Fossil grep vs POSIX grep" doc up to date relative to the merged grep-enhancements branch. ... (check-in: caba4b01 user: wyoung tags: trunk)