Fossil Forum

Forum
Login

Fossil compile fails when static compile flags are used.

(2) By SMAH on 2019-11-21 11:12:19 as edited by drh on 2019-11-21 11:25:19 [link]

I have tried to compile fossil version a9027e7d (20th Nov 2019) with the following configure arguments on Zorin Linux 15:

./configure --static --disable-fusefs --with-th1-docs --with-th1-hooks --with-openssl=/usr/lib/x86_64-linux-gnu

This fails. The config.log file is included below. The libcrypto.a, libcrypt.a, and libc.a libraries are in the directory /usr/lib/x86_64-linux-gnu long with their dynamic .so counterparts. When I omit the --static flag, the compile against the dynamic libraries is successful. The same is true for the release version 2.10 (the dynamic compile works and the static compile fails. Can someone tell me whether some other required libraries are missing or whether for the static compilation it is necessary to compile against a specific version of libc library. There is no guidance on this in the documentation.

config.log:

------------------------------------------------------------------------------
Invoked as: ./configure --static --disable-fusefs --with-th1-docs --with-th1-hooks --with-openssl=/usr/lib/x86_64-linux-gnu
Tclsh: /usr/bin/tclsh
Failed: cc -g -O2 conftest__.c -o conftest__
/tmp/ccDZQ0MQ.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `inflateEnd'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void inflateEnd(void);
int main(void) {
inflateEnd();
return 0;
}
============
Failed: cc -g -O2 -c conftest__.c -o conftest__.o
conftest__.c:2:10: fatal error: bind/resolv.h: No such file or directory
 #include <bind/resolv.h>
          ^~~~~~~~~~~~~~~
compilation terminated.
child process exited abnormally
============
The failed code was:
#include <netinet/in.h>
#include <bind/resolv.h>
int main(void) {

return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lssl -lcrypto -lz
/tmp/ccwVJnbf.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `dn_expand'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void dn_expand(void);
int main(void) {
dn_expand();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
/tmp/ccYfBd0h.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `dn_expand'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void dn_expand(void);
int main(void) {
dn_expand();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lssl -lcrypto -lz
/tmp/ccCcynEj.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `ns_name_uncompress'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void ns_name_uncompress(void);
int main(void) {
ns_name_uncompress();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lbind -lssl -lcrypto -lz
/usr/bin/ld: cannot find -lbind
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void ns_name_uncompress(void);
int main(void) {
ns_name_uncompress();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
/tmp/ccDbHuJv.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `res_query'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void res_query(void);
int main(void) {
res_query();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lbind -lresolv -lssl -lcrypto -lz
/usr/bin/ld: cannot find -lbind
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void res_query(void);
int main(void) {
res_query();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lresolv -lssl -lcrypto -lz
/tmp/cc9mF3uE.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `res_query'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void res_query(void);
int main(void) {
res_query();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
/tmp/cczMwAhL.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `res_9_ns_initparse'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void res_9_ns_initparse(void);
int main(void) {
res_9_ns_initparse();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lresolv -lssl -lcrypto -lz
/tmp/ccENe9OM.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `res_9_ns_initparse'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void res_9_ns_initparse(void);
int main(void) {
res_9_ns_initparse();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
/tmp/ccBcpM5Q.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `pledge'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void pledge(void);
int main(void) {
pledge();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
/tmp/ccbovJI0.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `getpassphrase'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void getpassphrase(void);
int main(void) {
getpassphrase();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lresolv -lssl -lcrypto -lz
conftest__.c:1:13: warning: conflicting types for built-in function ‘sin’ [-Wbuiltin-declaration-mismatch]
 extern void sin(void);
             ^~~
/tmp/ccEe3bT9.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `sin'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void sin(void);
int main(void) {
sin();
return 0;
}
============
Failed: cc -g -O2 conftest__.c -o conftest__ -lm -lresolv -lssl -lcrypto -lz
/tmp/cc6Uphph.o: In function `main':
/home/sm/FOSSIL/Static-Fossil-a9027e7d/conftest__.c:3: undefined reference to `dlopen'
collect2: error: ld returned 1 exit status
child process exited abnormally
============
The failed code was:
extern void dlopen(void);
int main(void) {
dlopen();
return 0;
}
============
-----------------------------------------------------------------------------

(3) By stephan on 2019-11-21 11:39:39 [link] in reply to 1

Not to second-guess your motivations here, but are you certain you want a static binary on Linux? i ask because...

(line-breaks added for readability)

...warning: Using 'getpwuid' in statically linked applications requires at
runtime the shared libraries from the glibc version used for linking
bld/http_socket.o: In function `socket_open':
...warning: Using 'getaddrinfo' in statically linked applications requires at
runtime the shared libraries from the glibc version used for linking
...warning: Using 'gethostbyname' in statically linked applications requires at
runtime the shared libraries from the glibc version used for linking
...
...warning: Using 'getpwuid' in statically linked applications requires at
runtime the shared libraries from the glibc version used for linking

(That was from trying to build fossil with ./configure --json --static, but it failed because of missing pthread-related definitions when linking.)

To the best of my knowledge, it's no longer possible to make "proper" static binaries on Linux if they use the system networking libraries.

That said, the fossil binaries on the site's download page are static, so maybe Richard can tell us what flags he uses to build those with.

(4) By juef on 2019-11-21 12:04:11 [link] in reply to 3

To the best of my knowledge, it's no longer possible to make "proper" static binaries on Linux if they use the system networking libraries.

with musl libc, I build like this:

export CC="clang"
export CC_FOR_BUILD="$CC"
export CFLAGS="-DNDEBUG -O2 -pipe -static"
export LDFLAGS="-static"

./configure			\
  --static			\
  --with-mman			\
  --disable-fusefs		\
  --disable-json		\
  && make			\
  && strip -s fossil

and it works just fine.

To ease build process you can use some musl-based Linux distributives, for example, Alpine Linux, Void Linux.

(10) By SMAH on 2019-11-24 15:06:06 [link] in reply to 4

I have tried musl-gcc on Zorin linux, and I still get some compile errors, and missing header files. There are probably library version, file location issues.

At the moment I am trying to evaluate the th1 hooks and th1 docs features, so if I can't get it to compile against the static libraries, I will compile against the dynamic libraries and use that for now.

Can you tell me which OS and version you compiled on and any additional libraries and versions installed.

Thanks.

(12) By juef on 2019-11-24 20:08:09 in reply to 10

Nothing special, alpine linux, "edge" repos; zlib 1.2.11; OpenSSL 1.1.1d

You can build fossil using this example Dockerfile:

FROM    alpine:edge

RUN     apk update                                                                                      \
        && apk upgrade                                                                                  \
                                                                                                        \
        && apk add --no-cache                                                                           \
        curl gcc make tcl                                                                               \
        musl-dev                                                                                        \
        openssl-dev zlib-dev                                                                            \
        openssl-libs-static zlib-static                                                                 \
                                                                                                        \
        && curl                                                                                         \
        "https://www.fossil-scm.org/index.html/tarball/fossil-src.tar.gz?name=fossil-src&uuid=trunk"    \
        -o fossil-src.tar.gz                                                                            \
                                                                                                        \
        && tar xf fossil-src.tar.gz                                                                     \
        && cd fossil-src                                                                                \
                                                                                                        \
        && ./configure                                                                                  \
        --static                                                                                        \
        --disable-fusefs                                                                                \
        --with-th1-docs                                                                                 \
        --with-th1-hooks                                                                                \
                                                                                                        \
        && make

and build it via:

docker build -t static_trunk_fossil .

then create a container via:

docker create --name fossil static_trunk_fossil

and finally copy a binary file from that container:

docker cp fossil:/fossil-src/fossil fossil

(13) By SMAH on 2019-11-25 13:59:26 [link] in reply to 12

Thanks for that - works like a charm. Alpine in Docker is so clean and easy compared to trying to do the same in a full distro, and the executables with static linking are fully portable. I am a convert.

Thanks again.

(14) By juef on 2019-11-25 14:06:40 [link] in reply to 13

My pleasure.

(16) By stephan on 2019-11-28 07:58:40 and edited on 2019-11-28 08:00:07 [link] in reply to 12

FYI: i have added those static/Docker build steps to a new section in:

https://fossil-scm.org/fossil/doc/trunk/www/build.wiki

(Section 5.0, as of this writing.)

Note that i am not a docker expert, so if any of the terminology used there is wrong or misleading, please suggest changes (or, if you've got commit access, feel free to edit it directly: www/build.wiki).

(5) By SMAH on 2019-11-21 13:47:34 [link] in reply to 3

The reason is because I can put it in a chroot environment or container without having to bother too much about associated libraries or changes to those libraries.

(7) By wyetr on 2019-11-21 15:06:29 [link] in reply to 5

How do you square a wish for chroots with a wish to statically link to libraries that may be receiving security updates? Do you rebuild Fossil on each OS update?

(8) By drh on 2019-11-21 15:15:10 [link] in reply to 7

I cannot answer for the OP, however the Fossil website uses statically linked Fossil binaries in a chroot jail for added security. The idea is that even if there is a vulnerability in the app that allows an attacker to penetrate into the machine, the chroot jail is so sparsely populated (no shared libraries, no shell, no /proc, /dev empty except for /dev/null and /dev/random, etc) that it will be more difficult for them to do any damage.

(9) By wyetr on 2019-11-21 15:56:48 [link] in reply to 8

In your case, you are rebuilding Fossil regularly, so you're reasonably safe.

As for risk of damage, once inside the chroot, you have access to the user table, which then gets you full Setup user access if you want it.

The Fossil binary nearest at hand here is dynamically linked to 18 OS-provided libraries. I suspect if I checked into it, I'd find that several of them have been updated in the last 6-12 months, some probably for security flaws.

(6) By drh on 2019-11-21 14:12:11 [link] in reply to 3

the fossil binaries on the site's download page are static, so maybe Richard can tell us what flags he uses to build those with.

See https://fossil-scm.org/fossil/wiki?name=Release+Build+How-To

(11) By SMAH on 2019-11-24 15:09:05 [link] in reply to 6

I did all of this. It is probably library package/version differences on the linux distro I am using.