Linux下共享库嵌套依赖问题 （转载）

转自：http://my.oschina.net/moooofly/blog/506466

问题场景：

1. 动态库 librabbitmq_r.so 内部依赖动态库 libevent_core.so 和 libevent_pthreads.so ；
2. 可执行程序 sa 依赖动态库 librabbitmq_r.so ;
3. 在链接生成 sa 的时候希望只指定 librabbitmq_r.so 而不指定 libevent_core.so 和 libevent_pthreads.so 。

错误信息：

...
g++ ../source/authorisecfg.o ../source/bmcinst.o ../source/config.o ../source/lgsinst.o ../source/logicsrv.o ../source/logicsrvinst.o ../source/logicsrvmodulelistcfg.o ../source/main.o ../source/moduleinst.o ../source/print.o ../source/routingkeycfg.o ../source/sautils.o ../source/structself.o ../source/../../../common/source/bossutils.o 
../source/../../../common/source/bossversion.o -o sa -m32 -L../../../../10-common/lib/release/linux -lrt -lwatchdogclient -lfiletransfer -losp -lkprop -ljsonconvert -ljsoncpp -ldeploycfg -lnosectionini -lrabbitmq_r -lmqwrapper -lreadwritelock -lcaptureexception -lnetconfig

/usr/bin/ld: warning: libevent_core.so, needed by ../../../../10-common/lib/release/linux/librabbitmq_r.so, not found (try using -rpath or -rpath-link)
/usr/bin/ld: warning: libevent_pthreads.so, needed by ../../../../10-common/lib/release/linux/librabbitmq_r.so, not found (try using -rpath or -rpath-link)

../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_base_free'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `evthread_use_pthreads'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_assign'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_base_dispatch'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_base_loopbreak'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_del'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_add'
../../../../10-common/lib/release/linux/librabbitmq_r.so: undefined reference to `event_base_new'
collect2: ld returned 1 exit status
make: *** [sa] Error 1

      由错误信息可以看出，未找到的符号均属于 libevent_core.so 和 libevent_pthreads.so 内部。但这两个库确实存在于 -L../../../../10-common/lib/release/linux 路径下，但为什么链接器仍旧无法找到对应的库和符号呢？

下面的文章讨论了这个问题（下面给出部分摘录）

《Why does ld need -rpath-link when linking an executable against a so that needs another so?》

...
You system, through ld.so.conf, ld.so.conf.d, and the system environment, LD_LIBRARY_PATH, etc.., provides the system-wide library search paths which are supplemented by installed libraries through pkg-config information and the like when you build against standard libraries. 
...
There is no standard run-time library search path for custom shared libraries you create yourself. You specify the search path to your libraries through the -L/path/to/lib designation during compile and link. For libraries in non-standard locations, the library search path can be optionally placed in the header of your executable (ELF header) at compile-time so that your executable can find the needed libraries.
...
rpath provides a way of embedding your custom run-time library search path in the ELF header so that your custom libraries can be found as well without having to specify the search path each time it is used. This applies to libraries that depend on libraries as well. As you have found, not only is the order you specify the libraries on the command line important, you also must provide the run-time library search path, or rpath, information for each dependent library you are linking against as well so that the header contains the location of all libraries needed to run.
...
back to the semantics of ld. In order to produce a "good link", ld must be able to locate all dependent libraries. ld cannot insure a good link otherwise. The runtime linker must find and load, not just to find the shared libraries needed by a program. ld cannot guarantee that will happen unless ld itself can locate all needed shared libraries at the time the progam is linked.
...

结论就是，像这种 a.so 依赖 b.so ，而 c 依赖 a.so 的情况，在链接过程中需要通过 -rpath-link 指定所需 .so 的位置，而不能仅仅使用 -L 指定。

示例如下：

[root@Betty include_test]# ll
总用量 20
-rw-r--r-- 1 root root 149 9月  14 16:19 main.c
-rw-r--r-- 1 root root 123 9月  14 16:40 say_hello.c
-rw-r--r-- 1 root root  20 9月  14 15:58 say_hello.h
-rw-r--r-- 1 root root 416 9月  14 16:39 time_print.c
-rw-r--r-- 1 root root  69 9月  14 15:00 time_print.h
[root@Betty include_test]#

【say_hello.c】

// g++ -o say_hello.so -fpic -shared say_hello.c

#include <stdio.h>
void say_hello()
{
    printf( "Hello World!
" );
}

【say_hello.h】

void say_hello();

【time_print.c】

// g++ -o time_print.so -fpic -shared -I. -L. time_print.c say_hello.so 

#include <time_print.h>
#include <stdio.h>
#include <say_hello.h>

int time_print( time_t tmp )
{
    int off = 0;
    time_t t;
    char buf[64] = {0};

    t = time( NULL );
    off = strftime( buf, sizeof(buf), "%d %b %H:%M:%S", localtime( &t ) );
    fprintf( stderr, "current timestamp = %s
", buf );

    say_hello();

    return 0;
}

【time_print.h】

#include <time.h>
int time_print( time_t t );

【main.c 】

// g++ -o main main.c time_print.so -I. -Wl,-rpath-link,.

#include <time_print.h>

int main()
{
    time_t t;
    time_print( t );
    return 0;
}

生成两个 .so 库

[root@Betty include_test]# g++ -o say_hello.so -fpic -shared say_hello.c
[root@Betty include_test]# ll
总用量 28
-rw-r--r-- 1 root root  149 9月  14 16:19 main.c
-rw-r--r-- 1 root root  123 9月  14 16:40 say_hello.c
-rw-r--r-- 1 root root   20 9月  14 15:58 say_hello.h
-rwxr-xr-x 1 root root 6286 9月  15 19:31 say_hello.so
-rw-r--r-- 1 root root  416 9月  14 16:39 time_print.c
-rw-r--r-- 1 root root   69 9月  14 15:00 time_print.h
[root@Betty include_test]# 
[root@Betty include_test]# g++ -o time_print.so -fpic -shared -I. -L. time_print.c say_hello.so
[root@Betty include_test]# ll
总用量 36
-rw-r--r-- 1 root root  149 9月  14 16:19 main.c
-rw-r--r-- 1 root root  123 9月  14 16:40 say_hello.c
-rw-r--r-- 1 root root   20 9月  14 15:58 say_hello.h
-rwxr-xr-x 1 root root 6286 9月  15 19:31 say_hello.so
-rw-r--r-- 1 root root  416 9月  14 16:39 time_print.c
-rw-r--r-- 1 root root   69 9月  14 15:00 time_print.h
-rwxr-xr-x 1 root root 7117 9月  15 19:31 time_print.so
[root@Betty include_test]#

若不指定 -rpath-link 选项，则链接失败

[root@Betty include_test]# g++ -o main main.c time_print.so -I. -L.
/usr/bin/ld: warning: say_hello.so, needed by time_print.so, not found (try using -rpath or -rpath-link)
time_print.so: undefined reference to `say_hello()'
collect2: ld 返回 1
[root@Betty include_test]#

若指定 -rpath-link 选项，则可以成功链接

[root@Betty include_test]# g++ -o main main.c time_print.so -I. -L. -Wl,-rpath-link,.
[root@Betty include_test]# ll
总用量 44
-rwxr-xr-x 1 root root 6999 9月  15 19:37 main
-rw-r--r-- 1 root root  149 9月  14 16:19 main.c
-rw-r--r-- 1 root root  123 9月  14 16:40 say_hello.c
-rw-r--r-- 1 root root   20 9月  14 15:58 say_hello.h
-rwxr-xr-x 1 root root 6286 9月  15 19:31 say_hello.so
-rw-r--r-- 1 root root  416 9月  14 16:39 time_print.c
-rw-r--r-- 1 root root   69 9月  14 15:00 time_print.h
-rwxr-xr-x 1 root root 7117 9月  15 19:31 time_print.so
[root@Betty include_test]#

查看一下共享库依赖关系

[root@Betty include_test]# ldd say_hello.so 
        linux-vdso.so.1 =>  (0x00007fffb53ff000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f56915d4000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f5691350000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5691139000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f5690da5000)
        /lib64/ld-linux-x86-64.so.2 (0x000000388c400000)
[root@Betty include_test]# 
[root@Betty include_test]# ldd time_print.so 
        linux-vdso.so.1 =>  (0x00007fff50cfa000)
        say_hello.so => not found
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007ff0f7829000)
        libm.so.6 => /lib64/libm.so.6 (0x00007ff0f75a5000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff0f738f000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff0f6ffa000)
        /lib64/ld-linux-x86-64.so.2 (0x000000388c400000)
[root@Betty include_test]# 
[root@Betty include_test]# ldd main
        linux-vdso.so.1 =>  (0x00007fffc55ff000)
        time_print.so => not found
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003899800000)
        libm.so.6 => /lib64/libm.so.6 (0x000000388dc00000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003898000000)
        libc.so.6 => /lib64/libc.so.6 (0x000000388c800000)
        /lib64/ld-linux-x86-64.so.2 (0x000000388c400000)
[root@Betty include_test]#

执行程序

[root@Betty include_test]# ./main
./main: error while loading shared libraries: time_print.so: cannot open shared object file: No such file or directory
[root@Betty include_test]# 
[root@Betty include_test]# LD_LIBRARY_PATH=. ./main
current timestamp = 15 Sep 19:41:00
Hello World!
[root@Betty include_test]#

最后给出一个 rpath 递归问题的讨论： 《Resursive linking with rpath》