Fossil Forum

Undefined reference to __res_query
Login

Undefined reference to __res_query

Undefined reference to __res_query

(1.1) By andygoth on 2018-09-22 00:20:59 edited from 1.0 [source]

I'm trying to compile Fossil on an older Linux system, but smtp.c apparently is trying to use library functions that aren't available.

$ f info
checkout: bd2bf81c352f67940f1cfbb8556ff94c65858680 2018-08-31 13:05:36 UTC
$ make
cc -o fossil ... -lm /.../fossil/compat/zlib/libz.a -ldl
bld/smtp.o: In function `smtp_mx_host':
/.../fossil/src/smtp.c:59: undefined reference to `__res_query'
/.../fossil/src/smtp.c:62: undefined reference to `__ns_initparse'
/.../fossil/src/smtp.c:69: undefined reference to `__ns_parserr'
/.../fossil/src/smtp.c:82: undefined reference to `__ns_name_uncompress'
collect2: ld returned 1 exit status
make: *** [fossil] Error 1
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.8 (Tikanga)
CentOS release 5.8 (Final)

(2) By anonymous on 2018-09-22 00:12:50 in reply to 1.0 [link] [source]

Check to see if there is a libresolv. If so maybe adding -lresolv to the link line after -ldl may help.

(3) By andygoth on 2018-09-22 00:35:10 in reply to 2 [link] [source]

Yes, there is a libresolv. No, -lresolv doesn't help. I tried it, but I get the same error as I originally reported.

If I misspell -lresolv (e.g. -lresolve), then I get /usr/bin/ld: cannot find -lresolve, as expected. It's always good to include a negative test to make sure what I type is being honored!

(4) By Warren Young (wyoung) on 2018-09-22 01:34:45 in reply to 3 [link] [source]

Adding libresolv does fix the res_query() link error, but it doesn't solve the three ns_*() link errors.

I believe this is because those three functions were internal support functions, not made visible until glibc 2.9, but CentOS 5 ships glibc 2.5.

You can try changing the code to use the hidden versions with the leading underscores, but I think it's probably better to just skip the features dependent on this when those libresolv features are missing. The smtp.c module isn't really ready for use at the moment anyway.

(5) By anonymous on 2018-09-22 02:06:00 in reply to 3 [link] [source]

Maybe libbind ?

(6) By Warren Young (wyoung) on 2018-09-22 07:06:10 in reply to 5 [link] [source]

Good call. I believe I've got the problem fixed on branch libbind-ns-alternative. So far, I've tested it on CentOS 5, CentOS 7, Debian 9, and macOS 10.13.

As a bonus, this change sequence allows the fancy DNS MX resolver stuff in src/smtp.c to work on macOS for the first time, provided you install libbind from Homebrew first. If you don't, it falls back to the previous "skip fancy DNS stuff" behavior.

I have it on good authority that Fossil 2.7 is imminent, so this not only needs to be tested before it can be safely merged to trunk, it needs to be tested quickly if it's going to make it into the release.

I'm up past my bedtime, so I can't do any more testing on other platforms tonight.

(7) By sean (jungleboogie) on 2018-09-22 07:30:54 in reply to 6 [link] [source]

On first pass, no issues compiling c7640b3895 on openBSD -current: Compiled on Sep 22 2018 00:23:31 using clang-6.0.0 (tags/RELEASE_600/final) (64-bit)

(8) By Warren Young (wyoung) on 2018-09-22 07:32:33 in reply to 7 [link] [source]

Does it actually compile a body for smtp_mx_host in src/smtp.c now? When I said that this change set allows it to build on macOS now, that's me being parochial: it potentially builds on any non-Linux OS as long as it can find suitable libraries and headers.

(9) By sean (jungleboogie) on 2018-09-22 07:38:28 in reply to 8 [link] [source]

Can you please give a hint on how to test that? Would I use test-smtp-probe?

(10) By Warren Young (wyoung) on 2018-09-22 07:40:03 in reply to 9 [link] [source]

Insert #error hi at line 58 in that file and rebuild. :)

(11) By Warren Young (wyoung) on 2018-09-22 07:56:45 in reply to 10 [link] [source]

I've done just that on an old OpenBSD 5.8 VM I happen to have on hand, and it's skipping the MX lookup stuff on it because it can't find ns_name_uncompress(). Oh well, it couldn't do that before, either. It tells you this during the configure run, right after the checks for ns_name_uncompress().

It's not enough to hold off on merging this branch to trunk, but maybe I or someone else will figure out a way to find that on OpenBSD before this branch is merged.

(12) By anonymous on 2018-09-22 14:30:25 in reply to 11 [link] [source]

The libbind-ns-alternative branch is worse on OpenBSD than on trunk.  It refuses to even make SMTP now:

Checking for bind/resolv.h...not found
Checking for resolv.h...not found
Checking libs for ns_name_uncompress...no
Checking libs for __ns_name_uncompress...no
WARNING: SMTP feature will not be able to look up local MX.


Why not simply change from using ns_name_compress to dn_expand?

I did this in trunk and it compiles fine (not sure how it works because I don't have a Fossil Forum anywhere):

$ ./fossil diff
Index: auto.def
==================================================================
--- auto.def
+++ auto.def
@@ -513,11 +513,11 @@
     # Last resort, may be Windows
     if {[is_mingw]} {
         define-append LIBS -lwsock32
     }
 }
-cc-check-function-in-lib ns_name_uncompress resolv
+cc-check-function-in-lib dn_expand resolv
 cc-check-functions utime
 cc-check-functions usleep
 cc-check-functions strchrnul
 cc-check-functions pledge
 cc-check-functions backtrace

Index: src/smtp.c
==================================================================
--- src/smtp.c
+++ src/smtp.c
@@ -77,11 +77,11 @@
         iBestPriority = priority;
       }
     }
   }
   if( pBest ){
-    ns_name_uncompress(aDns, aDns+nDns, pBest+2,
+    dn_expand(aDns, aDns+nDns, pBest+2,
                        zHostname, sizeof(zHostname));
     return fossil_strdup(zHostname);
   }
   return 0;
 #elif defined(FOSSIL_WINDOWS_STYLE_DNS)


Here's the relevant parts of configure:

Checking libs for gethostbyname...none needed
Checking libs for socket...none needed
Checking libs for dn_expand...none needed

Now, I don't know if this provides the same interface as ns_name_uncompress (I couldn't even find a man page for ns_name_compress), perhaps dn_expand is more standard?

(13) By Warren Young (wyoung) on 2018-09-22 16:47:44 in reply to 12 [link] [source]

The libbind-ns-alternative branch is worse on OpenBSD than on trunk. It refuses to even make SMTP now:

You've misunderstood what was previously happening on trunk: the trunk code only builds the Fossil SMTP server's MX record lookup code on Linux. All you're pointing out is the message I added to warn you that this code currently has no effect on your platform. Try my #error hi test on trunk if you think this was working on OpenBSD before.

Also, keep in mind that smtp.c is a half-finished SMTP server, not yet ready for anyone to make use of. Eventually it'll be an option that allows tight integration between the Forum and SMTP, so you don't have to tie Fossil to a third-party SMTP server in order to handle bounce messages, forum message submissions via email, etc. Until all of those features are finished, you aren't actually missing much on OpenBSD relative to Linux, because we can't do much with the existing feature set on Linux, either.

All of that is why this is just a warning, not a fatal configuration error.

I don't know if this provides the same interface as ns_name_uncompress

It appears to, but only testing will show if it's functionally equivalent.

perhaps dn_expand is more standard?

I like that it's shorter, so I've switched to that name on that branch, with the old alternatives as fallbacks.

However, this change now causes the build to fail outright on OpenBSD because it now can't find ns_parserr(). I lack the time right now to chase down a method of placating it. Maybe you can finish the detective work?

(14) By Warren Young (wyoung) on 2018-09-22 17:16:36 in reply to 13 [link] [source]

Until all of those features are finished, you aren't actually missing much on OpenBSD relative to Linux, because we can't do much with the existing feature set on Linux, either.

I've just realized a key implication of that fact: the only thing that's really needed before the 2.7 release is the workaround for pre-glibc-2.9 Linux flavors, and that only because Fossil trunk is testing for platforms here, not features. (My branch now does feature tests only.)

Therefore, I've just added another autosetup test for ns_parserr, which if missing, causes this experimental code to not be compiled, just as on trunk. Now it builds on OpenBSD again. It's no more functional than a trunk build, but at least it doesn't cause a compiler error.

If by chance one of you does come up with a method to let this work, perhaps that can slip into 2.7, but there's not really much point in achieving that goal until the SMTP server is much nearer a usable state.

(15.1) By andygoth on 2018-09-26 01:39:21 edited from 15.0 in reply to 1.1 [link] [source]

I tried again using Fossil 2.7, this time using a cross compiler targeting Red Hat 9. Very old, I know, but I have to support it. It fails due to smtp.c and also the lack of zlib when I try to use --with-miniz.

$ ./configure --with-miniz --with-openssl=none --host=i386-redhat-linux
$ make
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/smtp.o: in function `smtp_mx_host':
/home/andy/fossil-2.7/./src/smtp.c:59: undefined reference to `__res_query'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/smtp.c:62: undefined reference to `__ns_initparse'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/smtp.c:69: undefined reference to `__ns_parserr'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/smtp.c:82: undefined reference to `__ns_name_uncompress'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `zipfileDeflate':
/home/andy/fossil-2.7/./src/shell.c:5003: undefined reference to `compressBound'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:5019: undefined reference to `deflateInit2_'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:5020: undefined reference to `deflate'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:5030: undefined reference to `deflateEnd'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `sqlarUncompressFunc':
/home/andy/fossil-2.7/./src/shell.c:6291: undefined reference to `uncompress'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `zipfileInflate':
/home/andy/fossil-2.7/./src/shell.c:4969: undefined reference to `inflateInit2_'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:4982: undefined reference to `inflateEnd'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:4973: undefined reference to `inflate'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `zipfileStep':
/home/andy/fossil-2.7/./src/shell.c:6017: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:6017: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:6017: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `sqlarCompressFunc':
/home/andy/fossil-2.7/./src/shell.c:6243: undefined reference to `compressBound'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:6251: undefined reference to `compress'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `zipfileUpdate':
/home/andy/fossil-2.7/./src/shell.c:5628: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:5628: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: /home/andy/fossil-2.7/./src/shell.c:5628: undefined reference to `crc32'
/opt/cross/lib/gcc/i386-redhat-linux/8.2.0/../../../../i386-redhat-linux/bin/ld: bld/shell.o: in function `do_meta_command':
/home/andy/fossil-2.7/./src/shell.c:15309: undefined reference to `zlibVersion'
collect2: error: ld returned 1 exit status
src/main.mk:681: recipe for target 'fossil' failed
make: *** [fossil] Error 1

See also: zlib.h required even when using --with-miniz.

Update: The workaround in the above-linked thread solved the zlib issue, so don't worry about that for now.

(16) By Warren Young (wyoung) on 2018-09-26 03:15:44 in reply to 15.0 [link] [source]

Red Hat 9. Very old, I know, but I have to support it.

Then you should provide the patches to make that happen. Else, the statement should be, "but you have to support it," and no, we do not. :)

undefined reference to __res_query

Are you using my libbind-ns-alternative branch? It should not attempt to link to res_query if it can't find that at configure time.

undefined reference to compressBound

When I run nm -C bld/miniz.o here, I get a bunch of mz_* prefixed names, not the zlib-compatible names implied by the MINIZ_NO_ZLIB_COMPATIBLE_NAMES macro's name. Unless I'm missing something, someone went and added all those prefixes, then didn't change the sense of that macro to remove the prefixes. Maybe there's an upstream version of miniz with the fix already done. If not, I'd remove all the mz_ prefixes, as appears to have been originally intended in that code.

Fixing that will address all of your other build errors above. I'd do it myself, but I've got to sign off soon, else bad things will happen to me tomorrow.

(17) By andygoth on 2018-09-26 03:36:08 in reply to 16 [link] [source]

No, I didn't try libbind-ns-alternative. Sorry I missed that earlier. I've been away from this issue for a while. Turns out this fixes the issue, thank you! Can we merge to trunk?

Also, I'm annoyed that Markdown supports _ as a way to italicize and/or bold text. :/

(18) By Warren Young (wyoung) on 2018-09-26 04:24:19 in reply to 17 [link] [source]

Can we merge to trunk?

I'm just hoping to do some broader platform testing first. If someone else merges it before I get around to doing that, that wouldn't break my heart. Further extensions can be done onesey twosy atop the base I've created here.

(19) By andygoth on 2018-09-26 17:05:21 in reply to 1.1 [link] [source]

Here is another resolution (haha) from another thread.

(20) By Warren Young (wyoung) on 2018-09-26 18:26:33 in reply to 19 [link] [source]

Yes: what my branch does is define that for you when it sees that your build platform can't provide the necessary headers or libraries.