Fossil

Check-in [77d603c6]
Login

Check-in [77d603c6]

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Replaced Jan Nijtman's Dockerfile with a new one that does a 2-stage build. The first stage runs atop Alpine Linux instead of Fedora, reducing the initial build from ~635 MiB to about 16.

Rather than stop there, I then made it multi-stage, copying two key static binaries — Fossil and Busybox — over from the first stage into a fresh-from-scratch container and set it up to run the former jailed away from the latter.

The result is under 9 MiB, and it's as secure as one can hope, given that it starts up in "PUBLIC" mode. The new build doesn't have all the extra features turned on that the old one did, but it seems right to build the container with Fossil in its default configuration. If you want something else, copy the Dockerfile, hack it, and make it do what you want instead.

Having done all this, I replaced the one-off Dockerfile inline in section 5.0 of the build doc with a reference to this new Dockerfile and rewrote the section in terms of the new capabilities.

Finally, this lets us brag on how small the container can be, as compared to the Gitlab-CE container. Before, we were comparing a standalone binary to the container, which wan't entirely fair. (The desire to produce such a container was the spark that kicked this project off.)

Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 77d603c6a194a8941a028a5d43f38122b86114ca6525a5dd2658f3483c99a7a8
User & Date: wyoung 2022-08-06 04:24:33
Context
2022-08-06
19:34
The build docs for "./configure --static" now reference the section further down on Docker, since you may need to use this indirection to get --static to produce something suitable. ... (check-in: 7bfd7413 user: wyoung tags: trunk)
04:24
Replaced Jan Nijtman's Dockerfile with a new one that does a 2-stage build. The first stage runs atop Alpine Linux instead of Fedora, reducing the initial build from ~635 MiB to about 16.

Rather than stop there, I then made it multi-stage, copying two key static binaries — Fossil and Busybox — over from the first stage into a fresh-from-scratch container and set it up to run the former jailed away from the latter.

The result is under 9 MiB, and it's as secure as one can hope, given that it starts up in "PUBLIC" mode. The new build doesn't have all the extra features turned on that the old one did, but it seems right to build the container with Fossil in its default configuration. If you want something else, copy the Dockerfile, hack it, and make it do what you want instead.

Having done all this, I replaced the one-off Dockerfile inline in section 5.0 of the build doc with a reference to this new Dockerfile and rewrote the section in terms of the new capabilities.

Finally, this lets us brag on how small the container can be, as compared to the Gitlab-CE container. Before, we were comparing a standalone binary to the container, which wan't entirely fair. (The desire to produce such a container was the spark that kicked this project off.) ... (check-in: 77d603c6 user: wyoung tags: trunk)

2022-08-05
12:05
Assorted improvements to the first few sections of the fossil-v-git doc, mainly in updating them to track changes to world facts and to clarify the presentation. ... (check-in: c7afd68b user: wyoung tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to Dockerfile.

1
2
3
4
5



6
7

8
9
10
11







12


13
14
15

16
17
18

19



20





21
22


23
24
25
26




###
#   Dockerfile for Fossil
###
FROM fedora:29




### Now install some additional parts we will need for the build
RUN dnf update -y && dnf install -y gcc make tcl tcl-devel zlib-devel openssl-devel tar && dnf clean all && groupadd -r fossil -g 433 && useradd -u 431 -r -g fossil -d /opt/fossil -s /sbin/nologin -c "Fossil user" fossil


### If you want to build "trunk", change the next line accordingly.
ENV FOSSIL_INSTALL_VERSION release








RUN curl "https://fossil-scm.org/home/tarball/fossil-src.tar.gz?name=fossil-src&uuid=${FOSSIL_INSTALL_VERSION}" | tar zx


RUN cd fossil-src && ./configure --disable-fusefs --json --with-th1-docs --with-th1-hooks --with-tcl=1 --with-tcl-stubs --with-tcl-private-stubs
RUN cd fossil-src/src && mv main.c main.c.orig && sed s/\"now\"/0/ <main.c.orig >main.c
RUN cd fossil-src && make && strip fossil && cp fossil /usr/bin && cd .. && rm -rf fossil-src && chmod a+rx /usr/bin/fossil && mkdir -p /opt/fossil && chown fossil:fossil /opt/fossil


### Build is done, remove modules no longer needed
RUN dnf remove -y gcc make zlib-devel tcl-devel openssl-devel tar && dnf clean all





USER fossil






ENV HOME /opt/fossil



EXPOSE 8080

CMD ["/usr/bin/fossil", "server", "--create", "--user", "admin", "/opt/fossil/repository.fossil"]




<
|
<
<

>
>
>
|
|
>

|
<
|
>
>
>
>
>
>
>
|
>
>
|
<
<
>

<
<
>

>
>
>
|
>
>
>
>
>

<
>
>

|
|
|
>
>
>
>

1


2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
18
19
20
21
22


23
24


25
26
27
28
29
30
31
32
33
34
35
36

37
38
39
40
41
42
43
44
45
46

# STAGE 1: Build a static Fossil binary atop Alpine Linux



# Avoid the temptation to swap the wget call below out for an ADD URL
# directive.  The URL is fixed for a given release tag, which triggers
# Docker's caching behavior, causing it to reuse that version as long
# as it remains in the cache.  We prefer to rely on the caching of the
# server instance on fossil-scm.org, which will keep these trunk
# tarballs around until the next trunk commit.

FROM alpine:latest AS builder

WORKDIR /tmp
RUN apk update                                               \
     && apk upgrade --no-cache                               \
     && apk add --no-cache                                   \
         busybox-static gcc make                             \
         musl-dev                                            \
         openssl-dev openssl-libs-static                     \
         zlib-dev zlib-static                                \
     && wget https://fossil-scm.org/home/tarball/src.tar.gz  \
     && tar -xf src.tar.gz                                   \
     && cd src                                               \
     && ./configure --static CFLAGS='-Os -s'                 \


     && make -j



# STAGE 2: Pare that back to the bare essentials.

FROM scratch
ENV JAIL=/jail
WORKDIR ${JAIL}
COPY --from=builder /tmp/src/fossil ${JAIL}/bin/
COPY --from=builder /bin/busybox.static /bin/busybox
RUN [ "/bin/busybox", "--install", "/bin" ]
RUN mkdir -m 700 dev                   \
    && mknod -m 600 dev/null    c 1 3  \
    && mknod -m 600 dev/urandom c 1 9


# Now we can run the stripped-down environment in a chroot jail, while
# leaving open the option to debug it live via the Busybox shell.

EXPOSE 8080/tcp
CMD [ \
    "bin/fossil", "server", \
    "--create",             \
    "--jsmode", "bundled",  \
    "--user", "admin",      \
    "repo.fossil"]

Changes to www/build.wiki.

246
247
248
249
250
251
252
253
254
255
256
257
258

259
260
261
262

263
264

265




266
267




268


269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308


309
310
311
312
313
314
315
316
317
318
319
320


321
322
323
324
325
326
327

328
329
330
331
332
333
334
  TCC += -D_BSD_SOURCE
  TCC += -DWITHOUT_ICONV
  TCC += -Dsocketlen_t=int
  TCC += -DSQLITE_MAX_MMAP_SIZE=0
</pre></blockquote>
</ul>

<h2>5.0 Building a Static Binary on Linux using Docker</h2>

Building a static binary on Linux is not as straightforward as it
could be because the GNU C library requires that certain components be
dynamically loadable. That can be worked around by building against a
different C library, which is simplest to do by way of a container

environment like [https://www.docker.com/ | Docker].

The following instructions for building fossil using Docker
were adapted from [https://fossil-scm.org/forum/forumpost/5dd2d61e5f | forumpost/5dd2d61e5f].

These instructions assume that docker is installed and that the user running
these instructions has permission to do so (i.e., they are <tt>root</tt> or

are a member of the <tt>docker</tt> group).





First, create a file named <tt>Dockerfile</tt> with the following contents:







<pre><code>
FROM    alpine:edge
RUN     apk update                                                                                      \
        && apk upgrade                                                                                  \
        && apk add --no-cache                                                                           \
        curl gcc make tcl                                                                               \
        musl-dev                                                                                        \
        openssl-dev zlib-dev                                                                            \
        openssl-libs-static zlib-static                                                                 \
        && curl                                                                                         \
        "https://fossil-scm.org/home/tarball/fossil-src.tar.gz?name=fossil-src&uuid=trunk"    \
        -o fossil-src.tar.gz                                                                            \
        && tar xf fossil-src.tar.gz                                                                     \
        && cd fossil-src                                                                                \
        && ./configure                                                                                  \
        --static                                                                                        \
        --disable-fusefs                                                                                \
        --with-th1-docs                                                                                 \
        --with-th1-hooks                                                                                \
        && make
</code></pre>

Be sure to modify the <tt>configure</tt> flags, if desired. e.g., add <tt>--json</tt>
for JSON support.

From the directory containing that file, build it with docker:

<pre><code># docker build -t fossil_static .</code></pre>

If you get permissions errors when running that as a non-root user,
be sure to add the user to the <tt>docker</tt> group before trying
again.

That creates a docker image and builds a static fossil binary inside
it. That step will take several minutes or more, depending on the
speed of the build environment.

Next, create a docker container to host the image we just created:

<pre><code># docker create --name fossil fossil_static</code></pre>



Then copy the fossil binary from that container:

<pre><code># docker cp fossil:/fossil-src/fossil fossil</code></pre>

The resulting binary will be <em>huge</em> because it is built with
debug info. To strip that information, reducing the size greatly:

<pre><code># strip fossil</code></pre>

To delete the Docker container and image (if desired), run:



<pre><code># docker container rm fossil
# docker image ls
</code></pre>

Note the IDs of the images named <tt>fossil_static</tt> and <tt>alpine</tt>, then:

<pre><code>docker image rm THE_FOSSIL_ID THE_ALPINE_ID</code></pre>



<h2>6.0 Building on/for Android</h2>

<h3>6.1 Cross-compiling from Linux</h3>

The following instructions for building Fossil for Android via Linux,







<

|
|
<
<
>
|

|
|
>
|
|
>
|
>
>
>
>

<
>
>
>
>

>
>
|
|
<
<
<
<
<
<
|
|
|
|
|
|
|
|
|
<
<
|
<

|
|

|
|
|
|
|
|
|
|
<
<
|

<
|
<
>
>

|
|
|

|
|
|
|
|
|

>
>
|
|


<
|
<
>







246
247
248
249
250
251
252

253
254
255


256
257
258
259
260
261
262
263
264
265
266
267
268
269
270

271
272
273
274
275
276
277
278
279






280
281
282
283
284
285
286
287
288


289

290
291
292
293
294
295
296
297
298
299
300
301


302
303

304

305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324

325

326
327
328
329
330
331
332
333
  TCC += -D_BSD_SOURCE
  TCC += -DWITHOUT_ICONV
  TCC += -Dsocketlen_t=int
  TCC += -DSQLITE_MAX_MMAP_SIZE=0
</pre></blockquote>
</ul>



<h2 id="docker">5.0 Building a Docker Container</h2>



Fossil ships a <tt>Dockerfile</tt> at the top of its source tree which
you can build like so:

<pre><code>  $ docker build -t fossil --no-cache .</code></pre>

If the image built successfully, you can create a container from it and
test that it runs:

<pre><code>  $ docker run --name fossil -p 9999:8080/tcp fossil</code></pre>

This shows us remapping the internal TCP listening port as 9999 on the
host. As a rule, there's little point to using the "<tt>fossil server
--port</tt>" feature inside a Docker container. Let it default to 8080
internally, then remap it to wherever you want it on the host instead.


Our stock <tt>Dockerfile</tt> configures Fossil with the default feature
set, so you may wish to modify the <tt>Dockerfile</tt> to add
configuration options, add APK packages to support those options, and so
forth.

It builds tip-of-trunk. To get a release version instead, append
"<tt>?r=release</tt>" to the URL in the <tt>Dockerfile</tt>, then
(re)build it.







You may wish to direct Docker to copy an existing repo into the image at
build time, rather than let it create a blank one automatically. Simply
add this to the <tt>Dockerfile</tt> before the first "RUN" directive:

<pre><code>  COPY /local/path/to/my-project.fossil /jail/repo.fossil</code></pre>

A potentially surprising feature of this container is that it runs
Fossil as root, which causes [./chroot.md | Fossil's chroot jail
feature] to kick in. Since a Docker container is a type of über-jail


already, you may be wondering why we don't either:


  #  run <tt>fossil server --nojail</tt> to skip the internal chroot; or
  #  create a non-root user and force Docker to use that instead

The reason is, although this container is quite stripped-down by today's
standards, it's based on the [https://www.busybox.net/BusyBox.html |
surprisingly powerful Busybox project].  (This author made a living for
years in the early 1990s using Unix systems that were less powerful than
this container.) If someone ever figured out how to make a Fossil binary
execute arbitrary commands on the host or to open up a remote shell,
they'd likely be able to island-hop from there into the rest of your
network. We need this cute double-jail dance to keep the Fossil instance


from accessing the Busybox features.


We deem this risk low since a) it's never happened, that we know of;

and b) we've turned off all of the risky features like TH1 docs.
Nevertheless, we believe in defense-in-depth.

Our 2-stage build process uses Alpine Linux only as a build host. Once
we've got everything reduced to the two key static binaries — Fossil and
Busybox — we throw all the rest of it away.

A secondary benefit falls out of this process for free: it's arguably
the easiest way to build a purely static Fossil binary for Linux. Most
modern Linux distros make this surprisingly difficult, but Alpine's
back-to-basics nature makes static builds work the way they used to,
back in the day. If that's what you're after, you can skip the "run"
command above and extract the executable from the image instead:

<pre><code>  $ docker create --name fossil_static fossil_static
  $ docker cp fossil_static:/jail/bin/fossil .
  $ docker container rm fossil_static
  $ docker image rm fossil_static
</code></pre>


The resulting binary is the single largest file inside that container,

at about 6 MiB. (It's built stripped.)


<h2>6.0 Building on/for Android</h2>

<h3>6.1 Cross-compiling from Linux</h3>

The following instructions for building Fossil for Android via Linux,

Changes to www/fossil-v-git.wiki.

178
179
180
181
182
183
184
185
186



187
188
189
190
191
192
193
precompiled binaries], unpack the executable from the archive and put it
somewhere in your <tt>PATH</tt>. To uninstall it, delete the executable.

This policy is particularly useful when running Fossil inside a
restrictive container, anything from [./chroot.md | classic chroot
jails] to modern [https://en.wikipedia.org/wiki/OS-level_virtualization
| OS-level virtualization mechanisms] such as
[https://en.wikipedia.org/wiki/Docker_(software) | Docker].
([src:/file?name=Dockerfile&ci=trunk | We ship a <tt>Dockerfile</tt>].)




Modern Linux systems tend to make full static linking
[https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead
| difficult], but our official executables do statically link to OpenSSL
to remove a version dependency, resulting in an executable that's around
6 MiB, depending on the platform. ([Release Build How-To | Details].)
The result is dependent only upon widespread platform libraries with







|
|
>
>
>







178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
precompiled binaries], unpack the executable from the archive and put it
somewhere in your <tt>PATH</tt>. To uninstall it, delete the executable.

This policy is particularly useful when running Fossil inside a
restrictive container, anything from [./chroot.md | classic chroot
jails] to modern [https://en.wikipedia.org/wiki/OS-level_virtualization
| OS-level virtualization mechanisms] such as
[https://en.wikipedia.org/wiki/Docker_(software) | Docker]. Our
([src:/file?name=Dockerfile&ci=trunk | stock <tt>Dockerfile</tt>]
creates a container that's under 9 MiB on 64-bit Linux, including
a capable [https://www.busybox.net/ | Busybox] environment for live
debugging of the container's innards.)

Modern Linux systems tend to make full static linking
[https://stackoverflow.com/questions/3430400/linux-static-linking-is-dead
| difficult], but our official executables do statically link to OpenSSL
to remove a version dependency, resulting in an executable that's around
6 MiB, depending on the platform. ([Release Build How-To | Details].)
The result is dependent only upon widespread platform libraries with