linux 静态链接 mysql glibc 库的悲催过程 mango的日志 网易博客

linux 静态链接 mysql glibc 库的悲催过程 - mango的日志 - 网易博客

linux 静态链接 mysql glibc 库的悲催过程   

2012-12-29 19:09:31|  分类： 默认分类 |  标签：linux  静态链接  static  linking  glibc   |字号 大中小 订阅

这两天需要把项目做成静态链接版本，便于在各公司的各种linux平台上运行。

我想这还不简单，链接参数加一个-static不就行了，但接下来解决一系列问题的时间远远超出我的意料

开发环境：

CentOS release 5.8 (Final)

gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-52)

问题是由于静态链接mysql库引起的，我链接mysql库的参数是这样写的：

LIB_MYSQL     = $(shell mysql_config --libs)

发现静态链接悲剧了，好几百个链接错误，都是undefined，需要krb5库。链接加上krb5库：

LIB_KBR5     = $(shell krb5-config --libs)

最终部分链接参数是这样的：$(LIB_MYSQL) $(LIB_KBR5) $(LIB_THREAD) -static

结果链接器还是报错：

/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_get_principal':
(.text+0xfaa): undefined reference to `keyctl_read_alloc'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_getkeycount':
(.text+0x1129): undefined reference to `keyctl_read'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_clearcache':
(.text+0x1194): undefined reference to `keyctl_clear'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_clearcache':
(.text+0x11c1): undefined reference to `keyctl_clear'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x1692): undefined reference to `request_key'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x16bb): undefined reference to `keyctl_read'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x170c): undefined reference to `keyctl_search'
/usr/lib64/libkrb5.a(cc_keyring.o): In function `krb5_krcc_resolve':
(.text+0x1733): undefined reference to `add_key'

......

接下来就是排查是不是还缺什么库啊，缺什么链接时就加什么，结果都没能把undefined完全消灭。很郁闷，就静态链接mysql库，这都搞不定？

折腾了很久，还是把问题描述给一个朋友听，经他提醒，才想到是不是我用yum自动安装mysql的问题，要不自己下载mysql源码包，手动编译安装试试？下载了mysql-5.5.29.tar.gz，经过tar -xvf，cmake，make，make install，并且把链接mysql的链接参数手动指定了一下：
LIB_MYSQL     = /usr/local/mysql/lib/libmysqlclient.a

哈哈，的确不需要链接krb5的库了，但是有几个新的undefined，但是还好，只有几个，而且清晰：

/usr/local/mysql/lib/libmysqlclient.a(client_plugin.c.o): In function `add_plugin':
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:178: undefined reference to `dlclose'
/usr/local/mysql/lib/libmysqlclient.a(client_plugin.c.o): In function `mysql_load_plugin_v':
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:354: undefined reference to `dlopen'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:382: undefined reference to `dlsym'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:374: undefined reference to `dlerror'
/home/rsm/mysql-5.5.29/sql-common/client_plugin.c:385: undefined reference to `dlclose'

于是，链接的库增加libdl.a(属于glibc里的)，部分链接参数如下：

-lcrypt -L/usr/local/mysql/lib/ -lmysqlclient -ldl -lpthread -static

哦也，终于链接过了！虽然有几个让人讨厌的warning:

/usr/local/mysql/lib//libmysqlclient.a(client_plugin.c.o): In function `mysql_load_plugin_v':
/home/win/mysql-5.5.29/sql-common/client_plugin.c:354: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(mf_pack.c.o): In function `expand_tilde':
/home/win/mysql-5.5.29/mysys/mf_pack.c:387: warning: Using 'getpwnam' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(libmysql.c.o): In function `read_user_name':
/home/win/mysql-5.5.29/libmysql/libmysql.c:422: warning: Using 'getpwuid' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(mf_pack.c.o): In function `expand_tilde':
/home/win/mysql-5.5.29/mysys/mf_pack.c:389: warning: Using 'endpwent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/local/mysql/lib//libmysqlclient.a(client.c.o): In function `mysql_real_connect':
/home/win/mysql-5.5.29/sql-common/client.c:3247: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-redhat-linux/4.4.6/../../../../lib64/libpthread.a(libpthread.o): In function `sem_open':
(.text+0x764d): warning: the use of `mktemp' is dangerous, better use `mkstemp'
/usr/local/mysql/lib//libmysqlclient.a(libmysql.c.o): In function `mysql_server_init':
/home/win/mysql-5.5.29/libmysql/libmysql.c:153: warning: Using 'getservbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

于是很激动的启进程，刚起来，就宕掉了！！！！用GDB起来，发现是宕在调用libdl的库里。

其实上面的7个warning中，后面6个我认识的：在以前的另一个项目中(环境是ubuntu server)，也是静态链接，一直都有后面这6个warning，但是这几个函数没有用到，所以进程没有问题，当时项目进度又异常的紧，大家就一直没有花心思去研究消掉这几个warning。现在新增加的第一个warning，其中dlopen会调用到，会造成宕机，这下无法逃避，所以我接下来的精力就花在消除这几个warning上。

就是这么一个问题：Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking，这是一个很常见的问题：

static linking issue with glibc

但是我各种关键字组合google搜索，发现问这个问题的人不少，讨论也很热烈，但是都没有人给出解决问题的方案，下面两个连接是比较明确的：

http://bytes.com/topic/c/answers/877929-lnk-error-using-dlopen-statically-linked-apps-requires-runtime-shared-lib

https://bugzilla.redhat.com/show_bug.cgi?id=111298

http://www.qtcentre.org/threads/32521-Problems-with-static-linking-gt-segmentation-fault

其中有个老外说了下面一段：

Jakub Jelinek 2003-12-01 17:49:41 EST

If a RHL9 built statically linked binary using NSS works on older  glibcs, then you are just lucky, nothing else.  It certainly will not  work on all older glibcs, only on some and only under some circumstances.  Statically linked programs which are not self-contained and depend  on the installed glibc (be it because of using NSS or iconv (which  both dlopen glibc internally), using dlopen directly or even using  locale support) are really the least portable thing you can build.    Dynamically linking is uncomparably more portable.  You can link some libraries statically using -Bstatic -lfoo -lbar -Bdynamic,  yet link against the crucial system libraries dynamically.    The warnings were added so that everyone is aware of the problems  with -static.  For statically linked binaries using NSS/iconv/dlopen  the dynamic linker needs to be compiled into the statically linked  program, yet the shared libraries it loads come from a different  glibc if the statically linked binary is moved onto a different system.  This means the glibc private interface between the dynamic linker  and libc.so/libpthread.so is exposed, but this interface is constantly  changing (and has been changing for years in the past as well).    So, the only supported way of using NSS/iconv/dlopen in statically  linked programs is if they are compiled/linked against the same glibc  as they are run with.

 搜索可以打开这个页面：

http://www.google.com.hk/search?hl=en&source=hp&q=Using+'gethostbyname'+in+statically+linked+applications+requires+at+runtime+the+shared+libraries+from+the+glibc+version+used+for+linking&aq=f&oq=&aqi=

不过都没能解决问题，warning依然存在。朋友还提醒，是不是系统版本装的有问题？或者开发环境装的有问题？好吧，我已经没有更好的方法了，还让IT的同事重新找电脑装了新环境CentOS6.3 DVD版，折腾了很久，编译，链接，结果还是那7个warning，它们就死死的杵在那里！

最终，这个问题都没能解决。但是项目得继续，所以最终采用了一个折中方案：

mysql和另一些自有库静态链接，glibc,libstdc++等一些常见的基础库动态链接。

-Wl,-dn 其它自己的静态库 -L/usr/local/mysql/lib/ -lmysqlclient -Wl,-dy -lpthread -ldl -lcrypt

据说，这才是标准做法~~ 因为网上有人反问：为什么要静态链接glibc库？

当然，也可以直接在编译阶段将静态库编进项目里，例如：

g++ main.cpp libmysqlclient.a