目录
1. 引言 2. C/C++运行库 3. 静态Glibc && 可执行文件 入口/终止函数 4. 动态Glibc && 可执行文件 入口/终止函数 5. 静态Glibc && 共享库 入口/终止函数 6. 动态Glibc && 共享库 入口/终止函数 7. 静态库/共享库->编译/使用、动态加载 8. 和静态库/动态库相关的辅助工具
1. 引言
0x1: glibc
Any Unix-like operating system needs a C library: the library which defines the 'system calls' and other basic facilities such as open, malloc, printf, exit... The GNU C Library is used as the C library in the GNU systems and most systems with the Linux kernel. The GNU C Library is primarily designed to be a portable and high performance C library. It follows all relevant standards including ISO C11 and POSIX.1-2008. It is also internationalized and has one of the most complete internationalization interfaces known.
glibc是GNU发布的libc库，即c运行库。glibc是linux系统中最底层的api，几乎其它任何运行库都会依赖于glibc。glibc除了封装linux操作系统所提供的系统服务外，它本身也提供了许多其它一些必要功能服务的实现。由于 glibc 囊括了几乎所有的 UNIX 通行的标准，其内容包罗万象。而就像其他的 UNIX 系统一样，其内含的档案群分散于系统的树状目录结构中，像一个支架一般撑起整个操作系统
Glibc主要实现的功能如下
1. string: 字符串处理 2. signal: 信号处理 3. dlfcn: 管理共享库的动态加载 4. direct: 文件目录操作 5. elf: 共享库的动态加载器，也即interpreter 6. iconv： 不同字符集的编码转换 7. inet： socket接口的实现 8. intl: 国际化，也即gettext的实现 9. io 10. linuxthreads 11. locale: 本地化 12. login: 虚拟终端设备的管理，及系统的安全访问 13. malloc: 动态内存的分配与管理 14. nis 15. stdlib: 其它基本功能
0x2: 初始化代码和终止代码(Initialization and Termination code)
要明白的是，对于可执行文件、和共享对象来说，它们本质上都是可执行代码的集合体，区别仅仅在于
1. 可执行ELF文件可以独立运行可执行代码，并且可以载入外部的动态共享库ELF中的代码进行执行 2. 共享对象自身仅仅是可执行代码的一个"集合体"，需要被载入到可执行程序中得以执行
不管是可执行文件的执行、还是共享对象被载入到可执行文件中，它们都有对应的"初始化代码"和"终止代码"
1. 可执行文件 1) 初始化代码: 初始化代码在用户程序开始执行前执行，即在main()函数之前执行 2) 终止代码: 终止代码则在进程退出时执行，即程序return、exit返回的时候 //可执行文件的初始化代码和终止代码由"Glibc初始化入口函数"负责执行 2. 共享对象 1) 共享对象的初始化代码(_init()函数)在可共享对象文件获得控制权之前执行 2) 共享对象的终止代码(_fint()函数)在共享对象被卸载之后执行 //共享对象的初始化代码和终止代码由"动态连接器(ld-linux.so.2)"负责执行
ELF的初始化工作和终止工作都统一由Glibc运行库负责
0x3: Linux程序运行的基本流程
1. 操作系统运行用户程序时，将ELF文件映射到内存中 2. 当它看到可执行文件中的"PT_INERP"段时，操作系统将"PT_INTERP"段指定的"动态共享库加载器(ld.linux.so.2)"映射进内存，并通过栈向其传递它所需要的参数，并跳到"动态共享库加载器(ls-linux.so.2)"的入口处开始执行，将控制权交给"动态共享库加载器(ld.linux.so.2)" 3. "动态共享库加载器(ld.linux.so.2)"开始自举(Bootstrap) 1) 动态链接器入口地址就是自举代码的入口，当操作系统将进程控制权交给动态链接器时，动态链接器的自举代码即开始执行 2) 自举代码会找到自己的GOT。而GOT的第一个入口保存的即是".dynamic"段的偏移地址，由此获得了动态链接器本身的".dynamic"段 3) 通过".dynamic"段中的信息，自举代码便可以获得动态链接器本身的重定位表和符号表等，从而得到动态链接器本身的重定位入口，先将它们全部重定位 4) 从这一步开始动态链接器代码中才可以开始使用自己的全局变量和静态变量 4. 自举完成后，动态链接器根据可执行文件".dynamic"段中的"DT_NEEDED"元素开始依次加载(广度优先)依赖的共享对象，并加入它的符号表。如果这个共享对象依赖其它的共享对象，动态链接器也会加载它们。当这个过程结束时，所有需要的共享对象都已加载进内存，动态链接器也具有了程序和所有共享库的符号表 5. 当完成动态链接器的装载、普通共享对象的装载之后，链接器开始重新遍历可执行文件和每个共享对象的重定位表，将它们的GOT/PLT中的每个需要重定位的位置进行修正 要注意的是，链接器对共享库的重定位顺序是遵循"图的后序遍历顺序"依赖进行的，即如果A对象依赖B对象，则先处理B对象再处理A对象，加载时重定位包括： 1) 对数据的引用(在.rel.dyn段中)，需要初始化一个 GOT(在.got中)项为一个全局符号的地址 2) 对代码的引用(在.rel.plt段中)，需要初始化一个 GOT(在.got.plt)项为PLT表中第二条指令的地址(Procedure Linkage Table) 6. 重定位完成后就，如果某个共享对象有".init"段，那么动态链接器会执行".init"段中的代码，用以实现共享对象特有的初始化过程，例如 1) 共享对象中的C++全局/静态对象的构造就是通过"init"段来初始化 当完成了重定位和初始化后，所有的准备工作就宣告完成了，所需要的共享对象也都已经装载并且链接完成了，程序执行逻辑准备开始运行可执行程序本身了，值得注意的是，为了能在程序退出时让动态链接器有机会调用共享对象的终止代码，动态链接器会传递 一个"终止函数"(用以调用共享对象的终止代码)给用户程序 7. Glibc运行库负责在可执行程序main()函数执行前对运行库和程序的运行环境进行必要的初始化工作，包括 1) 堆、栈 2) I/O 3) 线程 4) 全局变量构造 8. 用户程序开始执行 1) 注册动态链接器的终止函数 2) 注册已加载的共享对象的终止函数 3) 注册可执行程序自身的终止函数 然后调用用户程序的初始化代码(和共享对象一样，可执行程序也有_init()函数) 9. 调用用户定义的main()函数，正式开始执行程序主体部分，这已部分是程序员可以控制的部分 10. main()函数返回后，返回到Glibc运行库入口函数，Glibc运行库入口函数进行清理工作，，包括 1) 全局变量析构 2) 堆的销毁 3) 关闭I/O Glibc的入口函数进行销毁操作，以注册的相反顺序调用终止函数 1) 调用用户程序的终止函数 2) 调用已加载的共享对象的终止函数 3) 调用动态链接器的终止函数 最后调用_exit()系统调用退出进程
Relevant Link:
https://www.gnu.org/software/libc/index.html http://baike.baidu.com/view/1323132.htm http://blog.csdn.net/tigerscorpio/article/details/6227730 http://dbp-consulting.com/tutorials/debugging/linuxProgramStartup.htm
Glibc的启动过程在不同的情况下差别很大，静态/动态的glibc、用于可执行文件/共享库文件的差别。可以组合出4种情况
2. C/C++运行库
运行时库(Runtime Library 运行期库)，在计算机程序设计领域中，是指一种被编译器用来实现编程语言内置函数以提供该语言程序运行时(执行)支持的一种特殊的计算机程序库。这种库一般包括基本的输入输出或是内存管理等支持。它是一群支持正在运行程序的函数，与操作系统合作提供诸如数学运算、输入输出等功能，让程序员不需要"重新发明轮子"，并善用操作系统提供的功能
运行时库由编译器决定，以面向编程语言，提供其最基本的执行时需要
1. Windows Visual Basic Visual Basic Virtual Machine (5.1) Visual Basic Virtual Machine (6.0) 2. Windwos C/C++ Microsoft C Runtime Library (7.0) Microsoft C Runtime Library (7.10) Microsoft Visual C++ 2005 SP1 (8.0.59193) Microsoft Visual C++ 2008 SP1 (9.0.30729) Microsoft Visual C++ 2008 ATL Update kb973924 (9.0.30729.4148) Microsoft Visual C++ 2010 (10.0.40219) 3. Windows .NET/C# .NET Framework 4. Java JVM(Java Virtual Machine) 5. Linux C/C++ Glibc Runtime Library
0x1: C语言运行库
任何一个C程序，它的背后都有一套庞大的代码库来支撑，以使得程序能够正常运行，这样的代码集合库称之为"运行时库(Runtime Library)"，而其中C语言的运行库，即被称为C运行库(CRT)，这套代码库包括
1. 启动与退出: 1) 入口函数 2) 入口函数所依赖的其他函数 3) 终止函数 2. 标准函数: 由C语言标准规定的C语言标准库所拥有的函数实现 3. I/O: I/O功能的封装和实现 4. 堆: 堆的封装和实现 5. 语言实现: 语言中一些特殊功能的实现 6. 调试: 实现调试功能的代码
这些运行库的组成成分中，C语言标准库占据了主要地位。C语言标准库是C语言标准化的基础函数库，例如printf、exit等都是标准库中的一部分。标准库定义了C语言中普遍存在的函数集合，程序员可以直接使用标准库中规定的函数而不用担心将代码移植到别的平台时对应的平台上不提供这个函数
ANSI C的标准库由24个C头文件组成，与许多其他语言(如java)的标准库不同，C语言的标准库非常轻量，它仅仅包含了数学函数、字符/字符串处理、I/O等基本方面，例如
1. 标准输入输出: stdio.h 2. 文件操作: stdio.h 3. 字符操作: ctype.h 4. 字符串操作: string.h 5. 数学函数: math.h 6. 资源管理: stdlib.h 7. 格式转换: stdlib.h 8. 时间/日期: time.h 9. 断言: assert.h 10. 各种类型上的常数: limits.h & float.h 11. 变长参数: stdarg.h 12. 非局部跳转: setjmp.h
0x2: glibc && MSVC CRT
运行库是和平台相关的，它和操作系统结合得非常紧密。C语言的运行库从某种程序上讲是C语言和不同操作系统平台之间的抽象层，它将不同的操作API抽象成相同的库函数，例如程序员可以在不同平台下使用fread来读取文件，而不用考虑在操作系统层面的实现是不同的。虽然各个平台下的C语言运行库提供了很多功能，但很多时候它们毕竟有限，例如用户的权限控制、操作系统线程创建等操作都不属于标准的C语言运行库，在这种情况下，我们不得不绕过C语言运行库直接调用操作系统的系统调用API或使用其他的库。也造成了在程序中可能会出现C运行库API和系统调用API混用的情况
Glibc(GNU C Library)是Linux平台下的主要运行库，MSVCRT(Microsoft Visual C Run-time)
Relevant Link:
http://zh.wikipedia.org/wiki/%E8%BF%90%E8%A1%8C%E6%97%B6%E5%BA%93
3. 静态Glibc && 可执行文件 入口/终止函数
在静态Glibc编译进可执行文件这种情况下，glibc的程序入口为_start()，这个入口由ld链接器默认的链接脚本所指定的。_start()由汇编实现，并且和平台相关
glibc-2.18sysdepsi386start.S
_start: /* 让ebp寄存器清零，这样做的目的是表明当前是程序的最外层函数 */ xorl %ebp, %ebp /* Extract the arguments as encoded on the stack and set up the arguments for main(argc, argv). envp will be determined later in __libc_start_main. 在调用_start前，装载器会把用户的参数和环境变量压入栈中，按照压栈的方法，栈顶的元素是argc、紧接的就是argv、env环境变量的数组(栈是向高地址生长的，所以最后入栈的参数位于栈的最高位置) */ // Pop the argument count. popl %esi // argv starts just at the current stack top. %ecx指向argv和env环境变量数组 movl %esp, %ecx /* Before pushing the arguments align the stack to a 16-byte (SSE needs 16-byte alignment) boundary to avoid penalties from misaligned accesses. T */ andl $0xfffffff0, %esp pushl %eax /* Push garbage because we allocate 28 more bytes. */ /* Provide the highest stack address to the user code (for stacks which grow downwards). */ pushl %esp /* Push address of the shared library termination function. */ pushl %edx #ifdef SHARED /* Load PIC register. */ call 1f addl $_GLOBAL_OFFSET_TABLE_, %ebx /* Push address of our own entry points to .fini and .init. */ leal __libc_csu_fini@GOTOFF(%ebx), %eax pushl %eax leal __libc_csu_init@GOTOFF(%ebx), %eax pushl %eax /* Push second argument: argv. */ pushl %ecx /* Push first argument: argc. */ pushl %esi pushl main@GOT(%ebx) /* Call the user's main function, and exit with its value. But let the libc call main. */ call __libc_start_main@PLT #else /* Push address of our own entry points to .fini and .init. */ pushl $__libc_csu_fini pushl $__libc_csu_init /* Push second argument: argv. */ pushl %ecx /* Push first argument: argc. */ pushl %esi pushl $main /* Call the user's main function, and exit with its value. But let the libc call main. */ call __libc_start_main #endif /* Crash if somehow `exit' does return. */ hlt
综上分析，我们可以把_start改写为一段更具有可读性的伪代码
void _start() { %ebp = 0; int argc = pop from stack; char** argv = top of stack; __libc_start_main( main, argc, argv, __libc_csu_init, __libc_csu_finit, edx, top of stack ); } //argv除了指向参数表外，还隐含紧接着环境变量表，这个环境变量表要在__libc_start_main里从argv内提取出来
实际执行代码的函数是__libc_start_main
glibc-2.18csulibc-start.c
/* Note: the fini parameter is ignored here for shared library. It is registered with __cxa_atexit. This had the disadvantage that finalizers were called in more than one place. 这是__libc_start_main函数的头部，和_start函数里的调用一致 */ STATIC int LIBC_START_MAIN ( int (*main) (int, char **, char ** MAIN_AUXVEC_DECL), int argc, char **argv, #ifdef LIBC_START_MAIN_AUXVEC_ARG ElfW(auxv_t) *auxvec, #endif //init: main调用前的初始化工作 __typeof (main) init, //fini: main结束后的收尾工作 void (*fini) (void), //rtld_fini: 和动态加载有关的收尾工作，rtld_fini即runtime loader void (*rtld_fini) (void), //stack_end: 标明了栈底的地址，即最高的栈地址 void *stack_end) { /* Result of the 'main' function. */ int result; __libc_multiple_libcs = &_dl_starting_up && !_dl_starting_up; #ifndef SHARED //根据当前栈的布局获取紧跟在argv数组之后的环境变量的地址 char **ev = &argv[argc + 1]; __environ = ev; /* Store the lowest stack address. This is done in ld.so if this is he code for the DSO. */ __libc_stack_end = stack_end; # ifdef HAVE_AUX_VECTOR /* First process the auxiliary vector since we need to find the program header to locate an eventually present PT_TLS entry. */ # ifndef LIBC_START_MAIN_AUXVEC_ARG ElfW(auxv_t) *auxvec; { char **evp = ev; while (*evp++ != NULL) ; auxvec = (ElfW(auxv_t) *) evp; } # endif _dl_aux_init (auxvec); if (GL(dl_phdr) == NULL) # endif { /* Starting from binutils-2.23, the linker will define the magic symbol __ehdr_start to point to our own ELF header if it is visible in a segment that also includes the phdrs. So we can set up _dl_phdr and _dl_phnum even without any information from auxv. 获取程序入口点地址 */ extern const ElfW(Ehdr) __ehdr_start __attribute__ ((weak, visibility ("hidden"))); if (&__ehdr_start != NULL) { assert (__ehdr_start.e_phentsize == sizeof *GL(dl_phdr)); GL(dl_phdr) = (const void *) &__ehdr_start + __ehdr_start.e_phoff; GL(dl_phnum) = __ehdr_start.e_phnum; } } # ifdef DL_SYSDEP_OSCHECK if (!__libc_multiple_libcs) { /* This needs to run to initiliaze _dl_osversion before TLS setup might check it. */ DL_SYSDEP_OSCHECK (__libc_fatal); } # endif /* Perform IREL{,A} relocations. */ apply_irel (); /* Initialize the thread library at least a bit since the libgcc functions are using thread functions if these are available and we need to setup errno. */ __pthread_initialize_minimal (); /* Set up the stack checker's canary. */ uintptr_t stack_chk_guard = _dl_setup_stack_chk_guard (_dl_random); # ifdef THREAD_SET_STACK_GUARD THREAD_SET_STACK_GUARD (stack_chk_guard); # else __stack_chk_guard = stack_chk_guard; # endif #endif /* Register the destructor of the dynamic linker if there is any. __cxa_atexit函数是glibc的内部函数，等同于atexit，用于将参数指定的函数在main结束之后调用 */ if (__builtin_expect (rtld_fini != NULL, 1)) __cxa_atexit ((void (*) (void *)) rtld_fini, NULL, NULL); #ifndef SHARED /* Call the initializer of the libc. This is only needed here if we are compiling for the static library in which case we haven't run the constructors in `_dl_start_user'. 调用共享库的_init()函数 */ __libc_init_first (argc, argv, __environ); /* Register the destructor of the program, if any. __cxa_atexit函数是glibc的内部函数，等同于atexit，用于将参数指定的函数在main结束之后调用 */ if (fini) __cxa_atexit ((void (*) (void *)) fini, NULL, NULL); /* Some security at this point. Prevent starting a SUID binary where the standard file descriptors are not opened. We have to do this only for statically linked applications since otherwise the dynamic loader did the work already. */ if (__builtin_expect (__libc_enable_secure, 0)) __libc_check_standard_fds (); #endif /* Call the initializer of the program, if any. */ #ifdef SHARED if (__builtin_expect (GLRO(dl_debug_mask) & DL_DEBUG_IMPCALLS, 0)) GLRO(dl_debug_printf) (" initialize program: %s ", argv[0]); #endif if (init) (*init) (argc, argv, __environ MAIN_AUXVEC_PARAM); #ifdef SHARED /* Auditing checkpoint: we have a new object. */ if (__builtin_expect (GLRO(dl_naudit) > 0, 0)) { struct audit_ifaces *afct = GLRO(dl_audit); struct link_map *head = GL(dl_ns)[LM_ID_BASE]._ns_loaded; for (unsigned int cnt = 0; cnt < GLRO(dl_naudit); ++cnt) { if (afct->preinit != NULL) afct->preinit (&head->l_audit[cnt].cookie); afct = afct->next; } } #endif #ifdef SHARED if (__builtin_expect (GLRO(dl_debug_mask) & DL_DEBUG_IMPCALLS, 0)) GLRO(dl_debug_printf) (" transferring control: %s ", argv[0]); #endif #ifdef HAVE_CLEANUP_JMP_BUF /* Memory for the cancellation buffer. */ struct pthread_unwind_buf unwind_buf; int not_first_call; not_first_call = setjmp ((struct __jmp_buf_tag *) unwind_buf.cancel_jmp_buf); if (__builtin_expect (! not_first_call, 1)) { struct pthread *self = THREAD_SELF; /* Store old info. */ unwind_buf.priv.data.prev = THREAD_GETMEM (self, cleanup_jmp_buf); unwind_buf.priv.data.cleanup = THREAD_GETMEM (self, cleanup); /* Store the new cleanup handler info. */ THREAD_SETMEM (self, cleanup_jmp_buf, &unwind_buf); /* Run the program. */ result = main (argc, argv, __environ MAIN_AUXVEC_PARAM); } else { /* Remove the thread-local data. */ # ifdef SHARED PTHFCT_CALL (ptr__nptl_deallocate_tsd, ()); # else extern void __nptl_deallocate_tsd (void) __attribute ((weak)); __nptl_deallocate_tsd (); # endif /* One less thread. Decrement the counter. If it is zero we terminate the entire process. */ result = 0; # ifdef SHARED unsigned int *ptr = __libc_pthread_functions.ptr_nthreads; # ifdef PTR_DEMANGLE PTR_DEMANGLE (ptr); # endif # else extern unsigned int __nptl_nthreads __attribute ((weak)); unsigned int *const ptr = &__nptl_nthreads; # endif if (! atomic_decrement_and_test (ptr)) /* Not much left to do but to exit the thread, not the process. */ __exit_thread (0); } #else /* Nothing fancy, just call the function. 调用主程序main函数 */ result = main (argc, argv, __environ MAIN_AUXVEC_PARAM); #endif exit (result); }
最后的_exit函数由汇编实现，且与平台相关，可能是80中断陷入，也可能是sysenter调用，可见，_exit()的作用仅仅是调用了exit这个系统调用，_exit调用后，进程就会直接结束，程序正常结束有2种情况
1. main函数正常返回 2. 程序中用exit退出 //exit是进程正常退出的必经之路
值得注意的是，_start和_exit的末尾都有一个hlt指令。这是因为在linux中，进程必须使用eixt系统调用结束，一旦exit被调用，程序的运行就会终止。
_exit里的hlt指令是为了检测exit系统调用是否成功，如果失败，程序就不会终止，hlt指令的的作用就是强行把程序停止下来。而_start里的hlt的作用也是如此。为了预防某种没用exit就回到了_start的情况(例如被误删了__libc_main_start末尾的exit)
Relevant Link:
http://dbp-consulting.com/tutorials/debugging/linuxProgramStartup.html http://blog.csdn.net/langchibi_zhou/article/details/5744922
4. 动态Glibc && 可执行文件 入口/终止函数
5. 静态Glibc && 共享库 入口/终止函数
6. 动态Glibc && 共享库 入口/终止函数
我们知道，可执行ELF程序的初始化工作是由Glibc来完成，而动态共享库的加载和初始化工作由"动态加载器(ld-linux-so.2)"完成
动态连接器的入口是_start， 在glibc/sysdeps/i386/dl-machine.h中的RTLD_START宏中定义。它首先调 用_dl_start()
#define RTLD_START asm (" .text .align 16 0: movl (%esp), %ebx ret .align 16 .globl _start .globl _dl_start_user _start: # Note that _dl_start gets the parameter in %eax. movl %esp, %eax call _dl_start ...
glibc-2.18elf tld.c
static ElfW(Addr) __attribute_used__ internal_function _dl_start (void *arg) { #ifdef DONT_USE_BOOTSTRAP_MAP # define bootstrap_map GL(dl_rtld_map) #else struct dl_start_final_info info; # define bootstrap_map info.l #endif /* This #define produces dynamic linking inline functions for bootstrap relocation instead of general-purpose relocation. Since ld.so must not have any undefined symbols the result is trivial: always the map of ld.so itself. */ #define RTLD_BOOTSTRAP #define RESOLVE_MAP(sym, version, flags) (&bootstrap_map) #include "dynamic-link.h" if (HP_TIMING_INLINE && HP_TIMING_AVAIL) #ifdef DONT_USE_BOOTSTRAP_MAP HP_TIMING_NOW (start_time); #else HP_TIMING_NOW (info.start_time); #endif /* Partly clean the `bootstrap_map' structure up. Don't use `memset' since it might not be built in or inlined and we cannot make function calls at this point. Use '__builtin_memset' if we know it is available. We do not have to clear the memory if we do not have to use the temporary bootstrap_map. Global variables are initialized to zero by default. */ #ifndef DONT_USE_BOOTSTRAP_MAP # ifdef HAVE_BUILTIN_MEMSET __builtin_memset (bootstrap_map.l_info, ' ', sizeof (bootstrap_map.l_info)); # else for (size_t cnt = 0; cnt < sizeof (bootstrap_map.l_info) / sizeof (bootstrap_map.l_info[0]); ++cnt) bootstrap_map.l_info[cnt] = 0; # endif # if USE___THREAD bootstrap_map.l_tls_modid = 0; # endif #endif /* Figure out the run-time load address of the dynamic linker itself. */ bootstrap_map.l_addr = elf_machine_load_address (); /* Read our own dynamic section and fill in the info array. */ bootstrap_map.l_ld = (void *) bootstrap_map.l_addr + elf_machine_dynamic (); elf_get_dynamic_info (&bootstrap_map, NULL); #if NO_TLS_OFFSET != 0 bootstrap_map.l_tls_offset = NO_TLS_OFFSET; #endif /* Get the dynamic linker's own program header. First we need the ELF file header. The `_begin' symbol created by the linker script points to it. When we have something like GOTOFF relocs, we can use a plain reference to find the runtime address. Without that, we have to rely on the `l_addr' value, which is not the value we want when prelinked. */ #if USE___THREAD dtv_t initdtv[3]; ElfW(Ehdr) *ehdr # ifdef DONT_USE_BOOTSTRAP_MAP = (ElfW(Ehdr) *) &_begin; # else # error This will not work with prelink. = (ElfW(Ehdr) *) bootstrap_map.l_addr; # endif ElfW(Phdr) *phdr = (ElfW(Phdr) *) ((void *) ehdr + ehdr->e_phoff); size_t cnt = ehdr->e_phnum; /* PT_TLS is usually the last phdr. */ while (cnt-- > 0) if (phdr[cnt].p_type == PT_TLS) { void *tlsblock; size_t max_align = MAX (TLS_INIT_TCB_ALIGN, phdr[cnt].p_align); char *p; bootstrap_map.l_tls_blocksize = phdr[cnt].p_memsz; bootstrap_map.l_tls_align = phdr[cnt].p_align; if (phdr[cnt].p_align == 0) bootstrap_map.l_tls_firstbyte_offset = 0; else bootstrap_map.l_tls_firstbyte_offset = (phdr[cnt].p_vaddr & (phdr[cnt].p_align - 1)); assert (bootstrap_map.l_tls_blocksize != 0); bootstrap_map.l_tls_initimage_size = phdr[cnt].p_filesz; bootstrap_map.l_tls_initimage = (void *) (bootstrap_map.l_addr + phdr[cnt].p_vaddr); /* We can now allocate the initial TLS block. This can happen on the stack. We'll get the final memory later when we know all about the various objects loaded at startup time. */ # if TLS_TCB_AT_TP tlsblock = alloca (roundup (bootstrap_map.l_tls_blocksize, TLS_INIT_TCB_ALIGN) + TLS_INIT_TCB_SIZE + max_align); # elif TLS_DTV_AT_TP tlsblock = alloca (roundup (TLS_INIT_TCB_SIZE, bootstrap_map.l_tls_align) + bootstrap_map.l_tls_blocksize + max_align); # else /* In case a model with a different layout for the TCB and DTV is defined add another #elif here and in the following #ifs. */ # error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined" # endif /* Align the TLS block. */ tlsblock = (void *) (((uintptr_t) tlsblock + max_align - 1) & ~(max_align - 1)); /* Initialize the dtv. [0] is the length, [1] the generation counter. */ initdtv[0].counter = 1; initdtv[1].counter = 0; /* Initialize the TLS block. */ # if TLS_TCB_AT_TP initdtv[2].pointer = tlsblock; # elif TLS_DTV_AT_TP bootstrap_map.l_tls_offset = roundup (TLS_INIT_TCB_SIZE, bootstrap_map.l_tls_align); initdtv[2].pointer = (char *) tlsblock + bootstrap_map.l_tls_offset; # else # error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined" # endif p = __mempcpy (initdtv[2].pointer, bootstrap_map.l_tls_initimage, bootstrap_map.l_tls_initimage_size); # ifdef HAVE_BUILTIN_MEMSET __builtin_memset (p, ' ', (bootstrap_map.l_tls_blocksize - bootstrap_map.l_tls_initimage_size)); # else { size_t remaining = (bootstrap_map.l_tls_blocksize - bootstrap_map.l_tls_initimage_size); while (remaining-- > 0) *p++ = ' '; } # endif /* Install the pointer to the dtv. */ /* Initialize the thread pointer. */ # if TLS_TCB_AT_TP bootstrap_map.l_tls_offset = roundup (bootstrap_map.l_tls_blocksize, TLS_INIT_TCB_ALIGN); INSTALL_DTV ((char *) tlsblock + bootstrap_map.l_tls_offset, initdtv); const char *lossage = TLS_INIT_TP ((char *) tlsblock + bootstrap_map.l_tls_offset, 0); # elif TLS_DTV_AT_TP INSTALL_DTV (tlsblock, initdtv); const char *lossage = TLS_INIT_TP (tlsblock, 0); # else # error "Either TLS_TCB_AT_TP or TLS_DTV_AT_TP must be defined" # endif if (__builtin_expect (lossage != NULL, 0)) _dl_fatal_printf ("cannot set up thread-local storage: %s ", lossage); /* So far this is module number one. */ bootstrap_map.l_tls_modid = 1; /* There can only be one PT_TLS entry. */ break; } #endif /* USE___THREAD */ #ifdef ELF_MACHINE_BEFORE_RTLD_RELOC ELF_MACHINE_BEFORE_RTLD_RELOC (bootstrap_map.l_info); #endif if (bootstrap_map.l_addr || ! bootstrap_map.l_info[VALIDX(DT_GNU_PRELINKED)]) { /* Relocate ourselves so we can do normal function calls and data access using the global offset table. */ ELF_DYNAMIC_RELOCATE (&bootstrap_map, 0, 0, 0); } bootstrap_map.l_relocated = 1; /* Please note that we don't allow profiling of this object and therefore need not test whether we have to allocate the array for the relocation results (as done in dl-reloc.c). */ /* Now life is sane; we can call functions and access global data. Set up to use the operating system facilities, and find out from the operating system's program loader where to find the program header table in core. Put the rest of _dl_start into a separate function, that way the compiler cannot put accesses to the GOT before ELF_DYNAMIC_RELOCATE. */ { #ifdef DONT_USE_BOOTSTRAP_MAP ElfW(Addr) entry = _dl_start_final (arg); #else ElfW(Addr) entry = _dl_start_final (arg, &info); #endif #ifndef ELF_MACHINE_START_ADDRESS # define ELF_MACHINE_START_ADDRESS(map, start) (start) #endif return ELF_MACHINE_START_ADDRESS (GL(dl_ns)[LM_ID_BASE]._ns_loaded, entry); } }
_dl_start()首先对动态连接器自己进行重定位，最后调用_dl_start_final()
glibc-2.18elf tld.c
/* This is the second half of _dl_start (below). It can be inlined safely under DONT_USE_BOOTSTRAP_MAP, where it is careful not to make any GOT references. When the tools don't permit us to avoid using a GOT entry for _dl_rtld_global (no attribute_hidden support), we must make sure this function is not inlined (see below). */ #ifdef DONT_USE_BOOTSTRAP_MAP static inline ElfW(Addr) __attribute__ ((always_inline)) _dl_start_final (void *arg) #else static ElfW(Addr) __attribute__ ((noinline)) _dl_start_final (void *arg, struct dl_start_final_info *info) #endif { ElfW(Addr) start_addr; if (HP_TIMING_AVAIL) { /* If it hasn't happen yet record the startup time. */ if (! HP_TIMING_INLINE) HP_TIMING_NOW (start_time); #if !defined DONT_USE_BOOTSTRAP_MAP && !defined HP_TIMING_NONAVAIL else start_time = info->start_time; #endif /* Initialize the timing functions. */ HP_TIMING_DIFF_INIT (); } /* Transfer data about ourselves to the permanent link_map structure. */ #ifndef DONT_USE_BOOTSTRAP_MAP GL(dl_rtld_map).l_addr = info->l.l_addr; GL(dl_rtld_map).l_ld = info->l.l_ld; memcpy (GL(dl_rtld_map).l_info, info->l.l_info, sizeof GL(dl_rtld_map).l_info); GL(dl_rtld_map).l_mach = info->l.l_mach; GL(dl_rtld_map).l_relocated = 1; #endif _dl_setup_hash (&GL(dl_rtld_map)); GL(dl_rtld_map).l_real = &GL(dl_rtld_map); GL(dl_rtld_map).l_map_start = (ElfW(Addr)) _begin; GL(dl_rtld_map).l_map_end = (ElfW(Addr)) _end; GL(dl_rtld_map).l_text_end = (ElfW(Addr)) _etext; /* Copy the TLS related data if necessary. */ #ifndef DONT_USE_BOOTSTRAP_MAP # if USE___THREAD assert (info->l.l_tls_modid != 0); GL(dl_rtld_map).l_tls_blocksize = info->l.l_tls_blocksize; GL(dl_rtld_map).l_tls_align = info->l.l_tls_align; GL(dl_rtld_map).l_tls_firstbyte_offset = info->l.l_tls_firstbyte_offset; GL(dl_rtld_map).l_tls_initimage_size = info->l.l_tls_initimage_size; GL(dl_rtld_map).l_tls_initimage = info->l.l_tls_initimage; GL(dl_rtld_map).l_tls_offset = info->l.l_tls_offset; GL(dl_rtld_map).l_tls_modid = 1; # else # if NO_TLS_OFFSET != 0 GL(dl_rtld_map).l_tls_offset = NO_TLS_OFFSET; # endif # endif #endif #if HP_TIMING_AVAIL HP_TIMING_NOW (GL(dl_cpuclock_offset)); #endif /* Initialize the stack end variable. */ __libc_stack_end = __builtin_frame_address (0); /* Call the OS-dependent function to set up life so we can do things like file access. It will call `dl_main' (below) to do all the real work of the dynamic linker, and then unwind our frame and run the user entry point on the same stack we entered on. */ start_addr = _dl_sysdep_start (arg, &dl_main); #ifndef HP_TIMING_NONAVAIL hp_timing_t rtld_total_time; if (HP_TIMING_AVAIL) { hp_timing_t end_time; /* Get the current time. */ HP_TIMING_NOW (end_time); /* Compute the difference. */ HP_TIMING_DIFF (rtld_total_time, start_time, end_time); } #endif if (__builtin_expect (GLRO(dl_debug_mask) & DL_DEBUG_STATISTICS, 0)) { #ifndef HP_TIMING_NONAVAIL print_statistics (&rtld_total_time); #else print_statistics (NULL); #endif } return start_addr; }
_dl_start_final()收集一些基本的运行时信息后 调用_dl_sysdep_start()
glibc-2.18elfdl-sysdep.c
ElfW(Addr) _dl_sysdep_start ( void **start_argptr, void (*dl_main) (const ElfW(Phdr) *phdr, ElfW(Word) phnum, ElfW(Addr) *user_entry, ElfW(auxv_t) *auxv) ) { const ElfW(Phdr) *phdr = NULL; ElfW(Word) phnum = 0; ElfW(Addr) user_entry; ElfW(auxv_t) *av; #ifdef HAVE_AUX_SECURE # define set_seen(tag) (tag) /* Evaluate for the side effects. */ # define set_seen_secure() ((void) 0) #else uid_t uid = 0; gid_t gid = 0; unsigned int seen = 0; # define set_seen_secure() (seen = -1) # ifdef HAVE_AUX_XID # define set_seen(tag) (tag) /* Evaluate for the side effects. */ # else # define M(type) (1 << (type)) # define set_seen(tag) seen |= M ((tag)->a_type) # endif #endif #ifdef NEED_DL_SYSINFO uintptr_t new_sysinfo = 0; #endif //处理由操作系统建立的环境信息 __libc_stack_end = DL_STACK_END (start_argptr); /* 设置相关参数 1. _dl_argc: 命令行参数的个数 2. _dl_argv: 命令行参数数组 3. _environ: 环境数组 4. _dl_auxv: 传递给动态连接器的附加参数数组 */ DL_FIND_ARG_COMPONENTS (start_argptr, _dl_argc, INTUSE(_dl_argv), _environ, GLRO(dl_auxv)); user_entry = (ElfW(Addr)) ENTRY_POINT; GLRO(dl_platform) = NULL; /* Default to nothing known about the platform. */ //读入_dl_auxv数组存储的信息 for (av = GLRO(dl_auxv); av->a_type != AT_NULL; set_seen (av++)) switch (av->a_type) { case AT_PHDR: phdr = (void *) av->a_un.a_val; break; case AT_PHNUM: phnum = av->a_un.a_val; break; case AT_PAGESZ: GLRO(dl_pagesize) = av->a_un.a_val; break; case AT_ENTRY: user_entry = av->a_un.a_val; break; #ifdef NEED_DL_BASE_ADDR case AT_BASE: _dl_base_addr = av->a_un.a_val; break; #endif #ifndef HAVE_AUX_SECURE case AT_UID: case AT_EUID: uid ^= av->a_un.a_val; break; case AT_GID: case AT_EGID: gid ^= av->a_un.a_val; break; #endif case AT_SECURE: #ifndef HAVE_AUX_SECURE seen = -1; #endif INTUSE(__libc_enable_secure) = av->a_un.a_val; break; case AT_PLATFORM: GLRO(dl_platform) = (void *) av->a_un.a_val; break; case AT_HWCAP: GLRO(dl_hwcap) = (unsigned long int) av->a_un.a_val; break; case AT_HWCAP2: GLRO(dl_hwcap2) = (unsigned long int) av->a_un.a_val; break; case AT_CLKTCK: GLRO(dl_clktck) = av->a_un.a_val; break; case AT_FPUCW: GLRO(dl_fpu_control) = av->a_un.a_val; break; #ifdef NEED_DL_SYSINFO case AT_SYSINFO: new_sysinfo = av->a_un.a_val; break; #endif #ifdef NEED_DL_SYSINFO_DSO case AT_SYSINFO_EHDR: GLRO(dl_sysinfo_dso) = (void *) av->a_un.a_val; break; #endif case AT_RANDOM: _dl_random = (void *) av->a_un.a_val; break; #ifdef DL_PLATFORM_AUXV DL_PLATFORM_AUXV #endif } #ifndef HAVE_AUX_SECURE if (seen != -1) { /* Fill in the values we have not gotten from the kernel through the auxiliary vector. */ # ifndef HAVE_AUX_XID # define SEE(UID, var, uid) if ((seen & M (AT_##UID)) == 0) var ^= __get##uid () SEE (UID, uid, uid); SEE (EUID, uid, euid); SEE (GID, gid, gid); SEE (EGID, gid, egid); # endif /* If one of the two pairs of IDs does not match this is a setuid or setgid run. */ INTUSE(__libc_enable_secure) = uid | gid; } #endif #ifndef HAVE_AUX_PAGESIZE if (GLRO(dl_pagesize) == 0) GLRO(dl_pagesize) = __getpagesize (); #endif #ifdef NEED_DL_SYSINFO if (new_sysinfo != 0) { # ifdef NEED_DL_SYSINFO_DSO /* Only set the sysinfo value if we also have the vsyscall DSO. */ if (GLRO(dl_sysinfo_dso) != 0) # endif GLRO(dl_sysinfo) = new_sysinfo; } #endif #ifdef DL_SYSDEP_INIT DL_SYSDEP_INIT; #endif #ifdef DL_PLATFORM_INIT DL_PLATFORM_INIT; #endif /* Determine the length of the platform name. */ if (GLRO(dl_platform) != NULL) GLRO(dl_platformlen) = strlen (GLRO(dl_platform)); if (__sbrk (0) == _end) /* The dynamic linker was run as a program, and so the initial break starts just after our bss, at &_end. The malloc in dl-minimal.c will consume the rest of this page, so tell the kernel to move the break up that far. When the user program examines its break, it will see this new value and not clobber our data. */ __sbrk (GLRO(dl_pagesize) - ((_end - (char *) 0) & (GLRO(dl_pagesize) - 1))); /* If this is a SUID program we make sure that FDs 0, 1, and 2 are allocated. If necessary we are doing it ourself. If it is not possible we stop the program. */ if (__builtin_expect (INTUSE(__libc_enable_secure), 0)) __libc_check_standard_fds (); //最后调用_dl_main()进行动态连接器的主要任务。 (*dl_main) (phdr, phnum, &user_entry, GLRO(dl_auxv)); return user_entry; }
dl_main()非常长，主要工作是加载可执行文件依赖的所有共享对象，构造符号表，并进行加载时重定位(有些重定位可以延迟到需要时再进行，称为 运行时重定位)
glibc-2.18elf tld.c
static void dl_main ( const ElfW(Phdr) *phdr, ElfW(Word) phnum, ElfW(Addr) *user_entry, ElfW(auxv_t) *auxv) { const ElfW(Phdr) *ph; enum mode mode; struct link_map *main_map; size_t file_size; char *file; bool has_interp = false; unsigned int i; bool prelinked = false; bool rtld_is_main = false; #ifndef HP_TIMING_NONAVAIL hp_timing_t start; hp_timing_t stop; hp_timing_t diff; #endif void *tcbp = NULL; #ifdef _LIBC_REENTRANT /* Explicit initialization since the reloc would just be more work. */ GL(dl_error_catch_tsd) = &_dl_initial_error_catch_tsd; #endif GL(dl_init_static_tls) = &_dl_nothread_init_static_tls; #if defined SHARED && defined _LIBC_REENTRANT && defined __rtld_lock_default_lock_recursive GL(dl_rtld_lock_recursive) = rtld_lock_default_lock_recursive; GL(dl_rtld_unlock_recursive) = rtld_lock_default_unlock_recursive; #endif /* The explicit initialization here is cheaper than processing the reloc in the _rtld_local definition's initializer. */ GL(dl_make_stack_executable_hook) = &_dl_make_stack_executable; /* Process the environment variable which control the behaviour. */ process_envvars (&mode); #ifndef HAVE_INLINED_SYSCALLS /* Set up a flag which tells we are just starting. */ INTUSE(_dl_starting_up) = 1; #endif if (*user_entry == (ElfW(Addr)) ENTRY_POINT) { /* Ho ho. We are not the program interpreter! We are the program itself! This means someone ran ld.so as a command. Well, that might be convenient to do sometimes. We support it by interpreting the args like this: ld.so PROGRAM ARGS... The first argument is the name of a file containing an ELF executable we will load and run with the following arguments. To simplify life here, PROGRAM is searched for using the normal rules for shared objects, rather than $PATH or anything like that. We just load it and use its entry point; we don't pay attention to its PT_INTERP command (we are the interpreter ourselves). This is an easy way to test a new ld.so before installing it. */ rtld_is_main = true; /* Note the place where the dynamic linker actually came from. */ GL(dl_rtld_map).l_name = rtld_progname; while (_dl_argc > 1) if (! strcmp (INTUSE(_dl_argv)[1], "--list")) { mode = list; GLRO(dl_lazy) = -1; /* This means do no dependency analysis. */ ++_dl_skip_args; --_dl_argc; ++INTUSE(_dl_argv); } else if (! strcmp (INTUSE(_dl_argv)[1], "--verify")) { mode = verify; ++_dl_skip_args; --_dl_argc; ++INTUSE(_dl_argv); } else if (! strcmp (INTUSE(_dl_argv)[1], "--inhibit-cache")) { GLRO(dl_inhibit_cache) = 1; ++_dl_skip_args; --_dl_argc; ++INTUSE(_dl_argv); } else if (! strcmp (INTUSE(_dl_argv)[1], "--library-path") && _dl_argc > 2) { library_path = INTUSE(_dl_argv)[2]; _dl_skip_args += 2; _dl_argc -= 2; INTUSE(_dl_argv) += 2; } else if (! strcmp (INTUSE(_dl_argv)[1], "--inhibit-rpath") && _dl_argc > 2) { GLRO(dl_inhibit_rpath) = INTUSE(_dl_argv)[2]; _dl_skip_args += 2; _dl_argc -= 2; INTUSE(_dl_argv) += 2; } else if (! strcmp (INTUSE(_dl_argv)[1], "--audit") && _dl_argc > 2) { process_dl_audit (INTUSE(_dl_argv)[2]); _dl_skip_args += 2; _dl_argc -= 2; INTUSE(_dl_argv) += 2; } else break; /* If we have no further argument the program was called incorrectly. Grant the user some education. */ if (_dl_argc < 2) _dl_fatal_printf (" Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...] You have invoked `ld.so', the helper program for shared library executables. This program usually lives in the file `/lib/ld.so', and special directives in executable files using ELF shared libraries tell the system's program loader to load the helper program from this file. This helper program loads the shared libraries needed by the program executable, prepares the program to run, and runs it. You may invoke this helper program directly from the command line to load and run an ELF executable file; this is like executing that file itself, but always uses this helper program from the file you specified, instead of the helper program file specified in the executable file you run. This is mostly of use for maintainers to test new versions of this helper program; chances are you did not intend to run this program. --list list all dependencies and how they are resolved --verify verify that given object really is a dynamically linked object we can handle --inhibit-cache Do not use " LD_SO_CACHE " --library-path PATH use given PATH instead of content of the environment variable LD_LIBRARY_PATH --inhibit-rpath LIST ignore RUNPATH and RPATH information in object names in LIST --audit LIST use objects named in LIST as auditors "); ++_dl_skip_args; --_dl_argc; ++INTUSE(_dl_argv); /* The initialization of _dl_stack_flags done below assumes the executable's PT_GNU_STACK may have been honored by the kernel, and so a PT_GNU_STACK with PF_X set means the stack started out with execute permission. However, this is not really true if the dynamic linker is the executable the kernel loaded. For this case, we must reinitialize _dl_stack_flags to match the dynamic linker itself. If the dynamic linker was built with a PT_GNU_STACK, then the kernel may have loaded us with a nonexecutable stack that we will have to make executable when we load the program below unless it has a PT_GNU_STACK indicating nonexecutable stack is ok. */ for (ph = phdr; ph < &phdr[phnum]; ++ph) if (ph->p_type == PT_GNU_STACK) { GL(dl_stack_flags) = ph->p_flags; break; } if (__builtin_expect (mode, normal) == verify) { const char *objname; const char *err_str = NULL; struct map_args args; bool malloced; args.str = rtld_progname; args.loader = NULL; args.mode = __RTLD_OPENEXEC; (void) _dl_catch_error (&objname, &err_str, &malloced, map_doit, &args); if (__builtin_expect (err_str != NULL, 0)) /* We don't free the returned string, the programs stops anyway. */ _exit (EXIT_FAILURE); } else { HP_TIMING_NOW (start); _dl_map_object (NULL, rtld_progname, lt_library, 0, __RTLD_OPENEXEC, LM_ID_BASE); HP_TIMING_NOW (stop); HP_TIMING_DIFF (load_time, start, stop); } /* Now the map for the main executable is available. */ main_map = GL(dl_ns)[LM_ID_BASE]._ns_loaded; if (__builtin_expect (mode, normal) == normal && GL(dl_rtld_map).l_info[DT_SONAME] != NULL && main_map->l_info[DT_SONAME] != NULL && strcmp ((const char *) D_PTR (&GL(dl_rtld_map), l_info[DT_STRTAB]) + GL(dl_rtld_map).l_info[DT_SONAME]->d_un.d_val, (const char *) D_PTR (main_map, l_info[DT_STRTAB]) + main_map->l_info[DT_SONAME]->d_un.d_val) == 0) _dl_fatal_printf ("loader cannot load itself "); phdr = main_map->l_phdr; phnum = main_map->l_phnum; /* We overwrite here a pointer to a malloc()ed string. But since the malloc() implementation used at this point is the dummy implementations which has no real free() function it does not makes sense to free the old string first. */ main_map->l_name = (char *) ""; *user_entry = main_map->l_entry; #ifdef HAVE_AUX_VECTOR /* Adjust the on-stack auxiliary vector so that it looks like the binary was executed directly. */ for (ElfW(auxv_t) *av = auxv; av->a_type != AT_NULL; av++) switch (av->a_type) { case AT_PHDR: av->a_un.a_val = (uintptr_t) phdr; break; case AT_PHNUM: av->a_un.a_val = phnum; break; case AT_ENTRY: av->a_un.a_val = *user_entry; break; } #endif } else { /* Create a link_map for the executable itself. This will be what dlopen on "" returns. */ main_map = _dl_new_object ((char *) "", "", lt_executable, NULL, __RTLD_OPENEXEC, LM_ID_BASE); assert (main_map != NULL); main_map->l_phdr = phdr; main_map->l_phnum = phnum; main_map->l_entry = *user_entry; /* Even though the link map is not yet fully initialized we can add it to the map list since there are no possible users running yet. */ _dl_add_to_namespace_list (main_map, LM_ID_BASE); assert (main_map == GL(dl_ns)[LM_ID_BASE]._ns_loaded); /* At this point we are in a bit of trouble. We would have to fill in the values for l_dev and l_ino. But in general we do not know where the file is. We also do not handle AT_EXECFD even if it would be passed up. We leave the values here defined to 0. This is normally no problem as the program code itself is normally no shared object and therefore cannot be loaded dynamically. Nothing prevent the use of dynamic binaries and in these situations we might get problems. We might not be able to find out whether the object is already loaded. But since there is no easy way out and because the dynamic binary must also not have an SONAME we ignore this program for now. If it becomes a problem we can force people using SONAMEs. */ /* We delay initializing the path structure until we got the dynamic information for the program. */ } main_map->l_map_end = 0; main_map->l_text_end = 0; /* Perhaps the executable has no PT_LOAD header entries at all. */ main_map->l_map_start = ~0; /* And it was opened directly. */ ++main_map->l_direct_opencount; /* Scan the program header table for the dynamic section. */ for (ph = phdr; ph < &phdr[phnum]; ++ph) switch (ph->p_type) { case PT_PHDR: /* Find out the load address. */ main_map->l_addr = (ElfW(Addr)) phdr - ph->p_vaddr; break; case PT_DYNAMIC: /* This tells us where to find the dynamic section, which tells us everything we need to do. */ main_map->l_ld = (void *) main_map->l_addr + ph->p_vaddr; break; case PT_INTERP: /* This "interpreter segment" was used by the program loader to find the program interpreter, which is this program itself, the dynamic linker. We note what name finds us, so that a future dlopen call or DT_NEEDED entry, for something that wants to link against the dynamic linker as a shared library, will know that the shared object is already loaded. */ _dl_rtld_libname.name = ((const char *) main_map->l_addr + ph->p_vaddr); /* _dl_rtld_libname.next = NULL; Already zero. */ GL(dl_rtld_map).l_libname = &_dl_rtld_libname; /* Ordinarilly, we would get additional names for the loader from our DT_SONAME. This can't happen if we were actually linked as a static executable (detect this case when we have no DYNAMIC). If so, assume the filename component of the interpreter path to be our SONAME, and add it to our name list. */ if (GL(dl_rtld_map).l_ld == NULL) { const char *p = NULL; const char *cp = _dl_rtld_libname.name; /* Find the filename part of the path. */ while (*cp != ' ') if (*cp++ == '/') p = cp; if (p != NULL) { _dl_rtld_libname2.name = p; /* _dl_rtld_libname2.next = NULL; Already zero. */ _dl_rtld_libname.next = &_dl_rtld_libname2; } } has_interp = true; break; case PT_LOAD: { ElfW(Addr) mapstart; ElfW(Addr) allocend; /* Remember where the main program starts in memory. */ mapstart = (main_map->l_addr + (ph->p_vaddr & ~(GLRO(dl_pagesize) - 1))); if (main_map->l_map_start > mapstart) main_map->l_map_start = mapstart; /* Also where it ends. */ allocend = main_map->l_addr + ph->p_vaddr + ph->p_memsz; if (main_map->l_map_end < allocend) main_map->l_map_end = allocend; if ((ph->p_flags & PF_X) && allocend > main_map->l_text_end) main_map->l_text_end = allocend; } break; case PT_TLS: if (ph->p_memsz > 0) { /* Note that in the case the dynamic linker we duplicate work here since we read the PT_TLS entry already in _dl_start_final. But the result is repeatable so do not check for this special but unimportant case. */ main_map->l_tls_blocksize = ph->p_memsz; main_map->l_tls_align = ph->p_align; if (ph->p_align == 0) main_map->l_tls_firstbyte_offset = 0; else main_map->l_tls_firstbyte_offset = (ph->p_vaddr & (ph->p_align - 1)); main_map->l_tls_initimage_size = ph->p_filesz; main_map->l_tls_initimage = (void *) ph->p_vaddr; /* This image gets the ID one. */ GL(dl_tls_max_dtv_idx) = main_map->l_tls_modid = 1; } break; case PT_GNU_STACK: GL(dl_stack_flags) = ph->p_flags; break; case PT_GNU_RELRO: main_map->l_relro_addr = ph->p_vaddr; main_map->l_relro_size = ph->p_memsz; break; } /* Adjust the address of the TLS initialization image in case the executable is actually an ET_DYN object. */ if (main_map->l_tls_initimage != NULL) main_map->l_tls_initimage = (char *) main_map->l_tls_initimage + main_map->l_addr; if (! main_map->l_map_end) main_map->l_map_end = ~0; if (! main_map->l_text_end) main_map->l_text_end = ~0; if (! GL(dl_rtld_map).l_libname && GL(dl_rtld_map).l_name) { /* We were invoked directly, so the program might not have a PT_INTERP. */ _dl_rtld_libname.name = GL(dl_rtld_map).l_name; /* _dl_rtld_libname.next = NULL; Already zero. */ GL(dl_rtld_map).l_libname = &_dl_rtld_libname; } else assert (GL(dl_rtld_map).l_libname); /* How else did we get here? */ /* If the current libname is different from the SONAME, add the latter as well. */ if (GL(dl_rtld_map).l_info[DT_SONAME] != NULL && strcmp (GL(dl_rtld_map).l_libname->name, (const char *) D_PTR (&GL(dl_rtld_map), l_info[DT_STRTAB]) + GL(dl_rtld_map).l_info[DT_SONAME]->d_un.d_val) != 0) { static struct libname_list newname; newname.name = ((char *) D_PTR (&GL(dl_rtld_map), l_info[DT_STRTAB]) + GL(dl_rtld_map).l_info[DT_SONAME]->d_un.d_ptr); newname.next = NULL; newname.dont_free = 1; assert (GL(dl_rtld_map).l_libname->next == NULL); GL(dl_rtld_map).l_libname->next = &newname; } /* The ld.so must be relocated since otherwise loading audit modules will fail since they reuse the very same ld.so. */ assert (GL(dl_rtld_map).l_relocated); if (! rtld_is_main) { /* Extract the contents of the dynamic section for easy access. */ elf_get_dynamic_info (main_map, NULL); /* Set up our cache of pointers into the hash table. */ _dl_setup_hash (main_map); } if (__builtin_expect (mode, normal) == verify) { /* We were called just to verify that this is a dynamic executable using us as the program interpreter. Exit with an error if we were not able to load the binary or no interpreter is specified (i.e., this is no dynamically linked binary. */ if (main_map->l_ld == NULL) _exit (1); /* We allow here some platform specific code. */ #ifdef DISTINGUISH_LIB_VERSIONS DISTINGUISH_LIB_VERSIONS; #endif _exit (has_interp ? 0 : 2); } struct link_map **first_preload = &GL(dl_rtld_map).l_next; /* Set up the data structures for the system-supplied DSO early, so they can influence _dl_init_paths. */ setup_vdso (main_map, &first_preload); #ifdef DL_SYSDEP_OSCHECK DL_SYSDEP_OSCHECK (_dl_fatal_printf); #endif /* Initialize the data structures for the search paths for shared objects. */ _dl_init_paths (library_path); /* Initialize _r_debug. */ struct r_debug *r = _dl_debug_initialize (GL(dl_rtld_map).l_addr, LM_ID_BASE); r->r_state = RT_CONSISTENT; /* Put the link_map for ourselves on the chain so it can be found by name. Note that at this point the global chain of link maps contains exactly one element, which is pointed to by dl_loaded. */ if (! GL(dl_rtld_map).l_name) /* If not invoked directly, the dynamic linker shared object file was found by the PT_INTERP name. */ GL(dl_rtld_map).l_name = (char *) GL(dl_rtld_map).l_libname->name; GL(dl_rtld_map).l_type = lt_library; main_map->l_next = &GL(dl_rtld_map); GL(dl_rtld_map).l_prev = main_map; ++GL(dl_ns)[LM_ID_BASE]._ns_nloaded; ++GL(dl_load_adds); /* If LD_USE_LOAD_BIAS env variable has not been seen, default to not using bias for non-prelinked PIEs and libraries and using it for executables or prelinked PIEs or libraries. */ if (GLRO(dl_use_load_bias) == (ElfW(Addr)) -2) GLRO(dl_use_load_bias) = main_map->l_addr == 0 ? -1 : 0; /* Set up the program header information for the dynamic linker itself. It is needed in the dl_iterate_phdr() callbacks. */ ElfW(Ehdr) *rtld_ehdr = (ElfW(Ehdr) *) GL(dl_rtld_map).l_map_start; ElfW(Phdr) *rtld_phdr = (ElfW(Phdr) *) (GL(dl_rtld_map).l_map_start + rtld_ehdr->e_phoff); GL(dl_rtld_map).l_phdr = rtld_phdr; GL(dl_rtld_map).l_phnum = rtld_ehdr->e_phnum; /* PT_GNU_RELRO is usually the last phdr. */ size_t cnt = rtld_ehdr->e_phnum; while (cnt-- > 0) if (rtld_phdr[cnt].p_type == PT_GNU_RELRO) { GL(dl_rtld_map).l_relro_addr = rtld_phdr[cnt].p_vaddr; GL(dl_rtld_map).l_relro_size = rtld_phdr[cnt].p_memsz; break; } /* Add the dynamic linker to the TLS list if it also uses TLS. */ if (GL(dl_rtld_map).l_tls_blocksize != 0) /* Assign a module ID. Do this before loading any audit modules. */ GL(dl_rtld_map).l_tls_modid = _dl_next_tls_modid (); /* If we have auditing DSOs to load, do it now. */ if (__builtin_expect (audit_list != NULL, 0)) { /* Iterate over all entries in the list. The order is important. */ struct audit_ifaces *last_audit = NULL; struct audit_list *al = audit_list->next; /* Since we start using the auditing DSOs right away we need to initialize the data structures now. */ tcbp = init_tls (); /* Initialize security features. We need to do it this early since otherwise the constructors of the audit libraries will use different values (especially the pointer guard) and will fail later on. */ security_init (); do { int tls_idx = GL(dl_tls_max_dtv_idx); /* Now it is time to determine the layout of the static TLS block and allocate it for the initial thread. Note that we always allocate the static block, we never defer it even if no DF_STATIC_TLS bit is set. The reason is that we know glibc will use the static model. */ struct dlmopen_args dlmargs; dlmargs.fname = al->name; dlmargs.map = NULL; const char *objname; const char *err_str = NULL; bool malloced; (void) _dl_catch_error (&objname, &err_str, &malloced, dlmopen_doit, &dlmargs); if (__builtin_expect (err_str != NULL, 0)) { not_loaded: _dl_error_printf (" ERROR: ld.so: object '%s' cannot be loaded as audit interface: %s; ignored. ", al->name, err_str); if (malloced) free ((char *) err_str); } else { struct lookup_args largs; largs.name = "la_version"; largs.map = dlmargs.map; /* Check whether the interface version matches. */ (void) _dl_catch_error (&objname, &err_str, &malloced, lookup_doit, &largs); unsigned int (*laversion) (unsigned int); unsigned int lav; if (err_str == NULL && (laversion = largs.result) != NULL && (lav = laversion (LAV_CURRENT)) > 0 && lav <= LAV_CURRENT) { /* Allocate structure for the callback function pointers. This call can never fail. */ union { struct audit_ifaces ifaces; #define naudit_ifaces 8 void (*fptr[naudit_ifaces]) (void); } *newp = malloc (sizeof (*newp)); /* Names of the auditing interfaces. All in one long string. */ static const char audit_iface_names[] = "la_activity " "la_objsearch " "la_objopen " "la_preinit " #if __ELF_NATIVE_CLASS == 32 "la_symbind32 " #elif __ELF_NATIVE_CLASS == 64 "la_symbind64 " #else # error "__ELF_NATIVE_CLASS must be defined" #endif #define STRING(s) __STRING (s) "la_" STRING (ARCH_LA_PLTENTER) " " "la_" STRING (ARCH_LA_PLTEXIT) " " "la_objclose "; unsigned int cnt = 0; const char *cp = audit_iface_names; do { largs.name = cp; (void) _dl_catch_error (&objname, &err_str, &malloced, lookup_doit, &largs); /* Store the pointer. */ if (err_str == NULL && largs.result != NULL) { newp->fptr[cnt] = largs.result; /* The dynamic linker link map is statically allocated, initialize the data now. */ GL(dl_rtld_map).l_audit[cnt].cookie = (intptr_t) &GL(dl_rtld_map); } else newp->fptr[cnt] = NULL; ++cnt; cp = (char *) rawmemchr (cp, ' ') + 1; } while (*cp != ' '); assert (cnt == naudit_ifaces); /* Now append the new auditing interface to the list. */ newp->ifaces.next = NULL; if (last_audit == NULL) last_audit = GLRO(dl_audit) = &newp->ifaces; else last_audit = last_audit->next = &newp->ifaces; ++GLRO(dl_naudit); /* Mark the DSO as being used for auditing. */ dlmargs.map->l_auditing = 1; } else { /* We cannot use the DSO, it does not have the appropriate interfaces or it expects something more recent. */ #ifndef NDEBUG Lmid_t ns = dlmargs.map->l_ns; #endif _dl_close (dlmargs.map); /* Make sure the namespace has been cleared entirely. */ assert (GL(dl_ns)[ns]._ns_loaded == NULL); assert (GL(dl_ns)[ns]._ns_nloaded == 0); GL(dl_tls_max_dtv_idx) = tls_idx; goto not_loaded; } } al = al->next; } while (al != audit_list->next); /* If we have any auditing modules, announce that we already have two objects loaded. */ if (__builtin_expect (GLRO(dl_naudit) > 0, 0)) { struct link_map *ls[2] = { main_map, &GL(dl_rtld_map) }; for (unsigned int outer = 0; outer < 2; ++outer) { struct audit_ifaces *afct = GLRO(dl_audit); for (unsigned int cnt = 0; cnt < GLRO(dl_naudit); ++cnt) { if (afct->objopen != NULL) { ls[outer]->l_audit[cnt].bindflags = afct->objopen (ls[outer], LM_ID_BASE, &ls[outer]->l_audit[cnt].cookie); ls[outer]->l_audit_any_plt |= ls[outer]->l_audit[cnt].bindflags != 0; } afct = afct->next; } } } } /* Set up debugging before the debugger is notified for the first time. */ #ifdef ELF_MACHINE_DEBUG_SETUP /* Some machines (e.g. MIPS) don't use DT_DEBUG in this way. */ ELF_MACHINE_DEBUG_SETUP (main_map, r); ELF_MACHINE_DEBUG_SETUP (&GL(dl_rtld_map), r); #else if (main_map->l_info[DT_DEBUG] != NULL) /* There is a DT_DEBUG entry in the dynamic section. Fill it in with the run-time address of the r_debug structure */ main_map->l_info[DT_DEBUG]->d_un.d_ptr = (ElfW(Addr)) r; /* Fill in the pointer in the dynamic linker's own dynamic section, in case you run gdb on the dynamic linker directly. */ if (GL(dl_rtld_map).l_info[DT_DEBUG] != NULL) GL(dl_rtld_map).l_info[DT_DEBUG]->d_un.d_ptr = (ElfW(Addr)) r; #endif /* We start adding objects. */ r->r_state = RT_ADD; _dl_debug_state (); LIBC_PROBE (init_start, 2, LM_ID_BASE, r); /* Auditing checkpoint: we are ready to signal that the initial map is being constructed. */ if (__builtin_expect (GLRO(dl_naudit) > 0, 0)) { struct audit_ifaces *afct = GLRO(dl_audit); for (unsigned int cnt = 0; cnt < GLRO(dl_naudit); ++cnt) { if (afct->activity != NULL) afct->activity (&main_map->l_audit[cnt].cookie, LA_ACT_ADD); afct = afct->next; } } /* We have two ways to specify objects to preload: via environment variable and via the file /etc/ld.so.preload. The latter can also be used when security is enabled. */ assert (*first_preload == NULL); struct link_map **preloads = NULL; unsigned int npreloads = 0; if (__builtin_expect (preloadlist != NULL, 0)) { /* The LD_PRELOAD environment variable gives list of libraries separated by white space or colons that are loaded before the executable's dependencies and prepended to the global scope list. If the binary is running setuid all elements containing a '/' are ignored since it is insecure. */ char *list = strdupa (preloadlist); char *p; HP_TIMING_NOW (start); /* Prevent optimizing strsep. Speed is not important here. */ while ((p = (strsep) (&list, " :")) != NULL) if (p[0] != ' ' && (__builtin_expect (! INTUSE(__libc_enable_secure), 1) || strchr (p, '/') == NULL)) npreloads += do_preload (p, main_map, "LD_PRELOAD"); HP_TIMING_NOW (stop); HP_TIMING_DIFF (diff, start, stop); HP_TIMING_ACCUM_NT (load_time, diff); } /* There usually is no ld.so.preload file, it should only be used for emergencies and testing. So the open call etc should usually fail. Using access() on a non-existing file is faster than using open(). So we do this first. If it succeeds we do almost twice the work but this does not matter, since it is not for production use. */ static const char preload_file[] = "/etc/ld.so.preload"; if (__builtin_expect (__access (preload_file, R_OK) == 0, 0)) { /* Read the contents of the file. */ file = _dl_sysdep_read_whole_file (preload_file, &file_size, PROT_READ | PROT_WRITE); if (__builtin_expect (file != MAP_FAILED, 0)) { /* Parse the file. It contains names of libraries to be loaded, separated by white spaces or `:'. It may also contain comments introduced by `#'. */ char *problem; char *runp; size_t rest; /* Eliminate comments. */ runp = file; rest = file_size; while (rest > 0) { char *comment = memchr (runp, '#', rest); if (comment == NULL) break; rest -= comment - runp; do *comment = ' '; while (--rest > 0 && *++comment != ' '); } /* We have one problematic case: if we have a name at the end of the file without a trailing terminating characters, we cannot place the . Handle the case separately. */ if (file[file_size - 1] != ' ' && file[file_size - 1] != ' ' && file[file_size - 1] != ' ' && file[file_size - 1] != ':') { problem = &file[file_size]; while (problem > file && problem[-1] != ' ' && problem[-1] != ' ' && problem[-1] != ' ' && problem[-1] != ':') --problem; if (problem > file) problem[-1] = ' '; } else { problem = NULL; file[file_size - 1] = ' '; } HP_TIMING_NOW (start); if (file != problem) { char *p; runp = file; while ((p = strsep (&runp, ": ")) != NULL) if (p[0] != ' ') npreloads += do_preload (p, main_map, preload_file); } if (problem != NULL) { char *p = strndupa (problem, file_size - (problem - file)); npreloads += do_preload (p, main_map, preload_file); } HP_TIMING_NOW (stop); HP_TIMING_DIFF (diff, start, stop); HP_TIMING_ACCUM_NT (load_time, diff); /* We don't need the file anymore. */ __munmap (file, file_size); } } if (__builtin_expect (*first_preload != NULL, 0)) { /* Set up PRELOADS with a vector of the preloaded libraries. */ struct link_map *l = *first_preload; preloads = __alloca (npreloads * sizeof preloads[0]); i = 0; do { preloads[i++] = l; l = l->l_next; } while (l); assert (i == npreloads); } /* Load all the libraries specified by DT_NEEDED entries. If LD_PRELOAD specified some libraries to load, these are inserted before the actual dependencies in the executable's searchlist for symbol resolution. */ HP_TIMING_NOW (start); _dl_map_object_deps (main_map, preloads, npreloads, mode == trace, 0); HP_TIMING_NOW (stop); HP_TIMING_DIFF (diff, start, stop); HP_TIMING_ACCUM_NT (load_time, diff); /* Mark all objects as being in the global scope. */ for (i = main_map->l_searchlist.r_nlist; i > 0; ) main_map->l_searchlist.r_list[--i]->l_global = 1; /* Remove _dl_rtld_map from the chain. */ GL(dl_rtld_map).l_prev->l_next = GL(dl_rtld_map).l_next; if (GL(dl_rtld_map).l_next != NULL) GL(dl_rtld_map).l_next->l_prev = GL(dl_rtld_map).l_prev; for (i = 1; i < main_map->l_searchlist.r_nlist; ++i) if (main_map->l_searchlist.r_list[i] == &GL(dl_rtld_map)) break; bool rtld_multiple_ref = false; if (__builtin_expect (i < main_map->l_searchlist.r_nlist, 1)) { /* Some DT_NEEDED entry referred to the interpreter object itself, so put it back in the list of visible objects. We insert it into the chain in symbol search order because gdb uses the chain's order as its symbol search order. */ rtld_multiple_ref = true; GL(dl_rtld_map).l_prev = main_map->l_searchlist.r_list[i - 1]; if (__builtin_expect (mode, normal) == normal) { GL(dl_rtld_map).l_next = (i + 1 < main_map->l_searchlist.r_nlist ? main_map->l_searchlist.r_list[i + 1] : NULL); #ifdef NEED_DL_SYSINFO_DSO if (GLRO(dl_sysinfo_map) != NULL && GL(dl_rtld_map).l_prev->l_next == GLRO(dl_sysinfo_map) && GL(dl_rtld_map).l_next != GLRO(dl_sysinfo_map)) GL(dl_rtld_map).l_prev = GLRO(dl_sysinfo_map); #endif } else /* In trace mode there might be an invisible object (which we could not find) after the previous one in the search list. In this case it doesn't matter much where we put the interpreter object, so we just initialize the list pointer so that the assertion below holds. */ GL(dl_rtld_map).l_next = GL(dl_rtld_map).l_prev->l_next; assert (GL(dl_rtld_map).l_prev->l_next == GL(dl_rtld_map).l_next); GL(dl_rtld_map).l_prev->l_next = &GL(dl_rtld_map); if (GL(dl_rtld_map).l_next != NULL) { assert (GL(dl_rtld_map).l_next->l_prev == GL(dl_rtld_map).l_prev); GL(dl_rtld_map).l_next->l_prev = &GL(dl_rtld_map); } } /* Now let us see whether all libraries are available in the versions we need. */ { struct version_check_args args; args.doexit = mode == normal; args.dotrace = mode == trace; _dl_receive_error (print_missing_version, version_check_doit, &args); } /* We do not initialize any of the TLS functionality unless any of the initial modules uses TLS. This makes dynamic loading of modules with TLS impossible, but to support it requires either eagerly doing setup now or lazily doing it later. Doing it now makes us incompatible with an old kernel that can't perform TLS_INIT_TP, even if no TLS is ever used. Trying to do it lazily is too hairy to try when there could be multiple threads (from a non-TLS-using libpthread). */ bool was_tls_init_tp_called = tls_init_tp_called; if (tcbp == NULL) tcbp = init_tls (); if (__builtin_expect (audit_list == NULL, 1)) /* Initialize security features. But only if we have not done it earlier. */ security_init (); if (__builtin_expect (mode, normal) != normal) { /* We were run just to list the shared libraries. It is important that we do this before real relocation, because the functions we call below for output may no longer work properly after relocation. */ struct link_map *l; if (GLRO(dl_debug_mask) & DL_DEBUG_PRELINK) { struct r_scope_elem *scope = &main_map->l_searchlist; for (i = 0; i < scope->r_nlist; i++) { l = scope->r_list [i]; if (l->l_faked) { _dl_printf (" %s => not found ", l->l_libname->name); continue; } if (_dl_name_match_p (GLRO(dl_trace_prelink), l)) GLRO(dl_trace_prelink_map) = l; _dl_printf (" %s => %s (0x%0*Zx, 0x%0*Zx)", DSO_FILENAME (l->l_libname->name), DSO_FILENAME (l->l_name), (int) sizeof l->l_map_start * 2, (size_t) l->l_map_start, (int) sizeof l->l_addr * 2, (size_t) l->l_addr); if (l->l_tls_modid) _dl_printf (" TLS(0x%Zx, 0x%0*Zx) ", l->l_tls_modid, (int) sizeof l->l_tls_offset * 2, (size_t) l->l_tls_offset); else _dl_printf (" "); } } else if (GLRO(dl_debug_mask) & DL_DEBUG_UNUSED) { /* Look through the dependencies of the main executable and determine which of them is not actually required. */ struct link_map *l = main_map; /* Relocate the main executable. */ struct relocate_args args = { .l = l, .reloc_mode = ((GLRO(dl_lazy) ? RTLD_LAZY : 0) | __RTLD_NOIFUNC) }; _dl_receive_error (print_unresolved, relocate_doit, &args); /* This loop depends on the dependencies of the executable to correspond in number and order to the DT_NEEDED entries. */ ElfW(Dyn) *dyn = main_map->l_ld; bool first = true; while (dyn->d_tag != DT_NULL) { if (dyn->d_tag == DT_NEEDED) { l = l->l_next; #ifdef NEED_DL_SYSINFO_DSO /* Skip the VDSO since it's not part of the list of objects we brought in via DT_NEEDED entries. */ if (l == GLRO(dl_sysinfo_map)) l = l->l_next; #endif if (!l->l_used) { if (first) { _dl_printf ("Unused direct dependencies: "); first = false; } _dl_printf (" %s ", l->l_name); } } ++dyn; } _exit (first != true); } else if (! main_map->l_info[DT_NEEDED]) _dl_printf (" statically linked "); else { for (l = main_map->l_next; l; l = l->l_next) if (l->l_faked) /* The library was not found. */ _dl_printf (" %s => not found ", l->l_libname->name); else if (strcmp (l->l_libname->name, l->l_name) == 0) _dl_printf (" %s (0x%0*Zx) ", l->l_libname->name, (int) sizeof l->l_map_start * 2, (size_t) l->l_map_start); else _dl_printf (" %s => %s (0x%0*Zx) ", l->l_libname->name, l->l_name, (int) sizeof l->l_map_start * 2, (size_t) l->l_map_start); } if (__builtin_expect (mode, trace) != trace) for (i = 1; i < (unsigned int) _dl_argc; ++i) { const ElfW(Sym) *ref = NULL; ElfW(Addr) loadbase; lookup_t result; result = _dl_lookup_symbol_x (INTUSE(_dl_argv)[i], main_map, &ref, main_map->l_scope, NULL, ELF_RTYPE_CLASS_PLT, DL_LOOKUP_ADD_DEPENDENCY, NULL); loadbase = LOOKUP_VALUE_ADDRESS (result); _dl_printf ("%s found at 0x%0*Zd in object at 0x%0*Zd ", INTUSE(_dl_argv)[i], (int) sizeof ref->st_value * 2, (size_t) ref->st_value, (int) sizeof loadbase * 2, (size_t) loadbase); } else { /* If LD_WARN is set, warn about undefined symbols. */ if (GLRO(dl_lazy) >= 0 && GLRO(dl_verbose)) { /* We have to do symbol dependency testing. */ struct relocate_args args; unsigned int i; args.reloc_mode = ((GLRO(dl_lazy) ? RTLD_LAZY : 0) | __RTLD_NOIFUNC); i = main_map->l_searchlist.r_nlist; while (i-- > 0) { struct link_map *l = main_map->l_initfini[i]; if (l != &GL(dl_rtld_map) && ! l->l_faked) { args.l = l; _dl_receive_error (print_unresolved, relocate_doit, &args); } } if ((GLRO(dl_debug_mask) & DL_DEBUG_PRELINK) && rtld_multiple_ref) { /* Mark the link map as not yet relocated again. */ GL(dl_rtld_map).l_relocated = 0; _dl_relocate_object (&GL(dl_rtld_map), main_map->l_scope, __RTLD_NOIFUNC, 0); } } #define VERNEEDTAG (DT_NUM + DT_THISPROCNUM + DT_VERSIONTAGIDX (DT_VERNEED)) if (version_info) { /* Print more information. This means here, print information about the versions needed. */ int first = 1; struct link_map *map; for (map = main_map; map != NULL; map = map->l_next) { const char *strtab; ElfW(Dyn) *dyn = map->l_info[VERNEEDTAG]; ElfW(Verneed) *ent; if (dyn == NULL) continue; strtab = (const void *) D_PTR (map, l_info[DT_STRTAB]); ent = (ElfW(Verneed) *) (map->l_addr + dyn->d_un.d_ptr); if (first) { _dl_printf (" Version information: "); first = 0; } _dl_printf (" %s: ", DSO_FILENAME (map->l_name)); while (1) { ElfW(Vernaux) *aux; struct link_map *needed; needed = find_needed (strtab + ent->vn_file); aux = (ElfW(Vernaux) *) ((char *) ent + ent->vn_aux); while (1) { const char *fname = NULL; if (needed != NULL && match_version (strtab + aux->vna_name, needed)) fname = needed->l_name; _dl_printf (" %s (%s) %s=> %s ", strtab + ent->vn_file, strtab + aux->vna_name, aux->vna_flags & VER_FLG_WEAK ? "[WEAK] " : "", fname ?: "not found"); if (aux->vna_next == 0) /* No more symbols. */ break; /* Next symbol. */ aux = (ElfW(Vernaux) *) ((char *) aux + aux->vna_next); } if (ent->vn_next == 0) /* No more dependencies. */ break; /* Next dependency. */ ent = (ElfW(Verneed) *) ((char *) ent + ent->vn_next); } } } } _exit (0); } if (main_map->l_info[ADDRIDX (DT_GNU_LIBLIST)] && ! __builtin_expect (GLRO(dl_profile) != NULL, 0) && ! __builtin_expect (GLRO(dl_dynamic_weak), 0)) { ElfW(Lib) *liblist, *liblistend; struct link_map **r_list, **r_listend, *l; const char *strtab = (const void *) D_PTR (main_map, l_info[DT_STRTAB]); assert (main_map->l_info[VALIDX (DT_GNU_LIBLISTSZ)] != NULL); liblist = (ElfW(Lib) *) main_map->l_info[ADDRIDX (DT_GNU_LIBLIST)]->d_un.d_ptr; liblistend = (ElfW(Lib) *) ((char *) liblist + main_map->l_info[VALIDX (DT_GNU_LIBLISTSZ)]->d_un.d_val); r_list = main_map->l_searchlist.r_list; r_listend = r_list + main_map->l_searchlist.r_nlist; for (; r_list < r_listend && liblist < liblistend; r_list++) { l = *r_list; if (l == main_map) continue; /* If the library is not mapped where it should, fail. */ if (l->l_addr) break; /* Next, check if checksum matches. */ if (l->l_info [VALIDX(DT_CHECKSUM)] == NULL || l->l_info [VALIDX(DT_CHECKSUM)]->d_un.d_val != liblist->l_checksum) break; if (l->l_info [VALIDX(DT_GNU_PRELINKED)] == NULL || l->l_info [VALIDX(DT_GNU_PRELINKED)]->d_un.d_val != liblist->l_time_stamp) break; if (! _dl_name_match_p (strtab + liblist->l_name, l)) break; ++liblist; } if (r_list == r_listend && liblist == liblistend) prelinked = true; if (__builtin_expect (GLRO(dl_debug_mask) & DL_DEBUG_LIBS, 0)) _dl_debug_printf (" prelink checking: %s ", prelinked ? "ok" : "failed"); } /* Now set up the variable which helps the assembler startup code. */ GL(dl_ns)[LM_ID_BASE]._ns_main_searchlist = &main_map->l_searchlist; /* Save the information about the original global scope list since we need it in the memory handling later. */ GLRO(dl_initial_searchlist) = *GL(dl_ns)[LM_ID_BASE]._ns_main_searchlist; /* Remember the last search directory added at startup, now that malloc will no longer be the one from dl-minimal.c. */ GLRO(dl_init_all_dirs) = GL(dl_all_dirs); /* Print scope information. */ if (__builtin_expect (GLRO(dl_debug_mask) & DL_DEBUG_SCOPES, 0)) { _dl_debug_printf (" Initial object scopes "); for (struct link_map *l = main_map; l != NULL; l = l->l_next) _dl_show_scope (l, 0); } if (prelinked) { if (main_map->l_info [ADDRIDX (DT_GNU_CONFLICT)] != NULL) { ElfW(Rela) *conflict, *conflictend; #ifndef HP_TIMING_NONAVAIL hp_timing_t start; hp_timing_t stop; #endif HP_TIMING_NOW (start); assert (main_map->l_info [VALIDX (DT_GNU_CONFLICTSZ)] != NULL); conflict = (ElfW(Rela) *) main_map->l_info [ADDRIDX (DT_GNU_CONFLICT)]->d_un.d_ptr; conflictend = (ElfW(Rela) *) ((char *) conflict + main_map->l_info [VALIDX (DT_GNU_CONFLICTSZ)]->d_un.d_val); _dl_resolve_conflicts (main_map, conflict, conflictend); HP_TIMING_NOW (stop); HP_TIMING_DIFF (relocate_time, start, stop); } /* Mark all the objects so we know they have been already relocated. */ for (struct link_map *l = main_map; l != NULL; l = l->l_next) { l->l_relocated = 1; if (l->l_relro_size) _dl_protect_relro (l); /* Add object to slot information data if necessasy. */ if (l->l_tls_blocksize != 0 && tls_init_tp_called) _dl_add_to_slotinfo (l); } } else { /* Now we have all the objects loaded. Relocate them all except for the dynamic linker itself. We do this in reverse order so that copy relocs of earlier objects overwrite the data written by later objects. We do not re-relocate the dynamic linker itself in this loop because that could result in the GOT entries for functions we call being changed, and that would break us. It is safe to relocate the dynamic linker out of order because it has no copy relocs (we know that because it is self-contained).
先重定位一个对象文件所依赖的所有对象文件再重定位这个对象文件 */ int consider_profiling = GLRO(dl_profile) != NULL; #ifndef HP_TIMING_NONAVAIL hp_timing_t start; hp_timing_t stop; #endif /* If we are profiling we also must do lazy reloaction. */ GLRO(dl_lazy) |= consider_profiling; HP_TIMING_NOW (start); unsigned i = main_map->l_searchlist.r_nlist; while (i-- > 0) { struct link_map *l = main_map->l_initfini[i]; /* While we are at it, help the memory handling a bit. We have to mark some data structures as allocated with the fake malloc() implementation in ld.so. */ struct libname_list *lnp = l->l_libname->next; while (__builtin_expect (lnp != NULL, 0)) { lnp->dont_free = 1; lnp = lnp->next; } /* Also allocated with the fake malloc(). */ l->l_free_initfini = 0; if (l != &GL(dl_rtld_map)) _dl_relocate_object (l, l->l_scope, GLRO(dl_lazy) ? RTLD_LAZY : 0, consider_profiling); /* Add object to slot information data if necessasy. */ if (l->l_tls_blocksize != 0 && tls_init_tp_called) _dl_add_to_slotinfo (l); } HP_TIMING_NOW (stop); HP_TIMING_DIFF (relocate_time, start, stop); /* Now enable profiling if needed. Like the previous call, this has to go here because the calls it makes should use the rtld versions of the functions (particularly calloc()), but it needs to have _dl_profile_map set up by the relocator. */ if (__builtin_expect (GL(dl_profile_map) != NULL, 0)) /* We must prepare the profiling. */ _dl_start_profile (); } #ifndef NONTLS_INIT_TP # define NONTLS_INIT_TP do { } while (0) #endif if (!was_tls_init_tp_called && GL(dl_tls_max_dtv_idx) > 0) ++GL(dl_tls_generation); /* Now that we have completed relocation, the initializer data for the TLS blocks has its final values and we can copy them into the main thread's TLS area, which we allocated above. */ _dl_allocate_tls_init (tcbp); /* And finally install it for the main thread. If ld.so itself uses TLS we know the thread pointer was initialized earlier. */ if (! tls_init_tp_called) { const char *lossage #ifdef USE___THREAD = TLS_INIT_TP (tcbp, USE___THREAD); #else = TLS_INIT_TP (tcbp, 0); #endif if (__builtin_expect (lossage != NULL, 0)) _dl_fatal_printf ("cannot set up thread-local storage: %s ", lossage); } /* Make sure no new search directories have been added. */ assert (GLRO(dl_init_all_dirs) == GL(dl_all_dirs)); if (! prelinked && rtld_multiple_ref) { /* There was an explicit ref to the dynamic linker as a shared lib. Re-relocate ourselves with user-controlled symbol definitions. We must do this after TLS initialization in case after this re-relocation, we might call a user-supplied function (e.g. calloc from _dl_relocate_object) that uses TLS data. */ #ifndef HP_TIMING_NONAVAIL hp_timing_t start; hp_timing_t stop; hp_timing_t add; #endif HP_TIMING_NOW (start); /* Mark the link map as not yet relocated again. */ GL(dl_rtld_map).l_relocated = 0; _dl_relocate_object (&GL(dl_rtld_map), main_map->l_scope, 0, 0); HP_TIMING_NOW (stop); HP_TIMING_DIFF (add, start, stop); HP_TIMING_ACCUM_NT (relocate_time, add); } /* Do any necessary cleanups for the startup OS interface code. We do these now so that no calls are made after rtld re-relocation which might be resolved to different functions than we expect. We cannot do this before relocating the other objects because _dl_relocate_object might need to call `mprotect' for DT_TEXTREL. */ _dl_sysdep_start_cleanup (); #ifdef SHARED /* Auditing checkpoint: we have added all objects. */ if (__builtin_expect (GLRO(dl_naudit) > 0, 0)) { struct link_map *head = GL(dl_ns)[LM_ID_BASE]._ns_loaded; /* Do not call the functions for any auditing object. */ if (head->l_auditing == 0) { struct audit_ifaces *afct = GLRO(dl_audit); for (unsigned int cnt = 0; cnt < GLRO(dl_naudit); ++cnt) { if (afct->activity != NULL) afct->activity (&head->l_audit[cnt].cookie, LA_ACT_CONSISTENT); afct = afct->next; } } } #endif /* Notify the debugger all new objects are now ready to go. We must re-get the address since by now the variable might be in another object. */ r = _dl_debug_initialize (0, LM_ID_BASE); r->r_state = RT_CONSISTENT; _dl_debug_state (); LIBC_PROBE (init_complete, 2, LM_ID_BASE, r); #if defined USE_LDCONFIG && !defined MAP_COPY /* We must munmap() the cache file. */ _dl_unload_cache (); #endif /* Once we return, _dl_sysdep_start will invoke the DT_INIT functions and then *USER_ENTRY. */ }
重定位完成后返回到_dl_sysdep_start()，然后返回 到_dl_start_final()，然后再返回到_dl_start()，继续返回到_start
glibc-2.18sysdepsi386dl-machine.h
#define RTLD_START asm (" ... _dl_start_user: movl %eax, %edi call 0b addl $_GLOBAL_OFFSET_TABLE_, %ebx movl _dl_skip_args@GOTOFF(%ebx), %eax popl %edx leal (%esp,%eax,4), %esp subl %eax, %edx push %edx " RTLD_START_SPECIAL_INIT " movl _rtld_local@GOTOFF(%ebx), %eax leal 8(%esp,%edx,4), %esi leal 4(%esp), %ecx movl %esp, %ebp andl $-16, %esp pushl %eax pushl %eax pushl %ebp pushl %esi xorl %ebp, %ebp /* _start调用动态连接器的初始化函数(以调用每个共享对象的初始化代码 _init()..) */ call _dl_init_internal@PLT /* 把动态连接器的终止函数(以调用每个共享对象的终止代码 _fini)地址存入EDX寄存器以传给可执行文件 */ leal _dl_fini@GOTOFF(%ebx), %edx movl (%esp), %esp /* 跳转到可执行文件的入口处开始执行 */ jmp *%edi .previous ");
动态连接器任务完成后将控制权转移给用户程序，此时用户程序才正是开始执行，整个流程如下
0x1: 共享库的初始化 && 终止函数
Win32下可以通过DllMain来进行初始化和终止工作，而Linux下则没有与之完全对应的函数，但可以通过一些方法模拟它的部分功能，即实现_init/_fini两个函数
但是，我们直接在程序中去重写/实现这2个函数，例如: test.c
#include <stdio.h> void _init(void) { printf("%s", __func__); } void _fini(void) { printf("%s", __func__); } //gcc test.c -o test
由此可见，这两个符号已经被编译器的脚手架代码占用了，我们不能再使用。这两个函数是用来初始化/销毁初始化全局变量/对象的，抢占这两个函数可能导致初始化/销毁初始化全局变量/对象出错
在C++中，我们可以使用类的构造函数和析构函数来完成初始化和终止工作，但是而C语言中，根本没有构造和析构函数，我们可以使用gcc的扩展
#include <stdio.h> __attribute ((constructor)) void _init(void) { printf("%s/n", __func__); } __attribute ((destructor)) void _fini(void) { printf("%s/n", __func__); } int main(int n) { return n; } //gcc t.c -o t
Relevant Link:
http://blog.csdn.net/tigerscorpio/article/details/6227730 http://www.ibm.com/developerworks/cn/linux/l-dynamic-libraries/ http://blog.csdn.net/absurd/article/details/928972
0x2: 共享对象初始化/终止函数的声明方式对初始化/终止调用顺序的影响
1. 使用GCC修饰符声明初始化函数(非标准函数名)
static __attribute__((constructor)) void do_init(void) {...}
使用这种方法声明初始化函数要注意的是，初始化的函数"do_init"的调用不会在so的其他函数被调用之前被执行
/* 1. 程序没有在执行程序的main之前，没有按照代码逻辑先执行初始化函数do_init，导致fopen64函数地址没有正确获取，即这个时候的fopen64函数地址是无效的 https://sourceware.org/ml/libc-help/2014-09/msg00018.html http://stackoverflow.com/questions/6083337/overriding-malloc-using-the-ld-preload-mechanism 2. 如果在hook的fopen64函数中发现fopen64的函数地址无效，则会调用dlsym进行动态获取，dlsym在内部调用了malloc进行堆内存分配 3. 而在这个项目的程序中使用了自己定制化的内存管理库取代了libjemalloc.so，即使用定制化的malloc() api。定制化的malloc()的代码逻辑会采用加锁的方式调用fopen 这种情况导致了malloc和dlsym的deadlock 4. 对do_init初始化函数未成功调用的解释 http://stackoverflow.com/questions/9472519/shared-library-constructor-not-working#comment12230604_9472519 /* When you compile as a .o with setup(), gcc just treats it as a normal function (since you're not compiling as a .so, so it doesn't care). Then, ld doesn't see any _init() or anything like a DT_INIT in the ELF's dynamic section, and assumes there's no constructors. When you compile as a .o with _init(), gcc also treats it as a normal function. In fact, it looks to me like the object files are identical except for the names of the functions themselves! So once again, ld looks at the .o file, but this time sees a _init() function, which it knows it's looking for, and decides it's a constructor, and correspondingly creates a DT_INIT entry in the new .so. */ 5. 正确的初始化函数的声明方式 static __attribute__((constructor)) void _init(void) {...}
2. 使用GCC修饰符声明初始化函数(标准函数名)
static void _init(void) __attribute__((constructor)); static void _init(void) {..} static void fini(void) __attribute__((constructor)); static void fini(void) {..}
3. 直接声明初始化函数(标准函数名)
void _init(void) {..} void _fini(void) {..} /* 使用这种初始化函数声明方式，需要在GCC编译的时候，指定"-nostartfiles"参数，否则会引发"multiple-definition"错误http://www.faqs.org/docs/Linux-HOWTO/Program-Library-HOWTO.html#INIT-AND-CLEANUP */
Relevant Link:
http://stackoverflow.com/questions/9759880/automatically-executed-functions-when-loading-shared-libraries http://www.linux-mag.com/id/1028/ http://hackerboss.com/overriding-system-functions-for-fun-and-profit/ https://github.com/AbsoluteZero2A03/Socksify http://hydra.geht.net/tino/howto/linux/socksify/ http://pkg-ruby-extras.alioth.debian.org/cgi-bin/gemwatch/em-socksify
7. 静态库/共享库->编译/使用、动态加载
0x1: 静态库
静态库的本质就是将多个目标文件打包成一个文件。在使用时链接静态库就是将库中被调用的代码复制到调用模块中。优点是使用静态库的代码在运行时无需依赖库，且执行效率高，缺点是静态库占用空间大，库中代码一旦修改必须重新链接
1. 编译生成静态库
math.h(接口文件)
#ifndef MATH_H #define MATH_H double add(double x, double y); void show(double result); #endif
calc.c(计算模块)
#include <stdio.h> #include "math.h" double add(double x, double y){ return x + y; }
show.c(显示模块)
#include <stdio.h> #include "math.h" void show(double result){ printf("%lf ", result); }
将C源文件编译成目标文件
gcc -c calc.c
gcc -c show.c
将目标文件打包成静态库文件
//静态库文件名规范: lib + Yourname + .a ar -r libmath.a calc.o show.o
静态库生成成功后，只要向第三方哭使用者提供libmath.a和math.h即可，因为libmath.a中包含函数的实现逻辑代码，而使用前需要进行声明的声明语句在math.h中
2. 使用静态库
main.c
#include <stdio.h> #include "math.h" int main(){ printf("5.3 + 2.8 = "); show(add(5.3, 2.8)); return 0; }
与静态库文件一同(将静态库中相应代码复制到主程序中)生成可执行文件
gcc main.c libmath.a or export LIBRARY_PATH=静态库文件所在路径 gcc main.c –lmath //选项-l用于指定静态库名 or //gcc main.c –lmath –L静态库文件所在路径 gcc main.c -lmath -L.
0x2: 共享库
共享库和静态库最大的不同就是，链接共享库并不需要将库中被调用的代码复制到调用模块中，相反被嵌入到调用模块中的仅仅是被调用代码在共享库中的相对地址。如果共享库中的代码同时为多个进程所用，共享库的实例在整个内存空间中仅需一份，这正是共享的意义所在，共享库的优点是占用空间小，即使修改了库中的代码，只要接口保持不变，无需重新链接，缺点是使用共享库的代码在运行时需要依赖库，执行效率略低
1. 共享库的构建
math.h(接口文件)
#ifndef MATH_H
#define MATH_H
double add(double x, double y);
void show(double result);
#endif
calc.c(计算模块)
#include <stdio.h>
#include "math.h"
double add(double x, double y){
return x + y;
}
show.c(显示模块)
#include <stdio.h>
#include "math.h"
void show(double result){
printf("%lf
", result);
}
将C源文件编译成目标文件
gcc -c -fpic calc.c gcc -c -fpic show.c //选项fpic是指生成位置无关代码，即调用代码通过相对地址标识被调用代码的位置，模块中的指令与该模块被加载到内存中的位置无关
将目标文件打包成共享库文件
gcc -shared calc.o show.o -o libmath.so //编译和链接也可以合并为一步完成 gcc -shared -fpic calc.c show.c -o libmath.so
PIC(Position Independent Code, 位置无关代码)
1. 调用代码通过相对地址标识被调用代码的位置，模块中的指令与该模块被加载到内存中的位置无关 2. -fPIC: 大模式: 生成代码比较大，运行速度比较慢，所有平台都支持 3. -fpic: 小模式: 生成代码比较小，运行速度比较快，仅部分平台支持
2. 共享库的使用
main.c
#include <stdio.h> #include "math.h" int main(){ printf("5.3 + 2.8 = "); show(add(5.3, 2.8)); return 0; }
与共享库文件一同生成可执行文件
gcc main.c libmath.so or export LIBRARY_PATH=共享库文件所在路径 gcc main.c –lmath //选项-l用于指定共享库名 or gcc main.c –lmath –L共享库文件所在路径
这里需要明白的，使用共享库编译主程序和使用静态库编译主程序最大的不同的是，使用静态库编译主程序时已经把相关逻辑代码复制到了主程序中，主程序可以直接执行
而指定共享库编译主程序仅仅是将对应函数在共享库中的地址嵌入到调用模块中，我么还需要设置相关环境变量，保证主程序运行时能够定位找到共享库
export LD_LIBRARY_PATH=共享库文件所在路径 //export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
0x3: 动态加载
从本质上讲，通过编译器指令链接共享库也是动态加载的一种，只是这个工作是ld(共享库加载器)在背后完成的。而除此之外，我们还可以在程序代码中通过指定API实现动态加载指定共享库
1. 头文件和库
在程序中动态加载共享库需要调用一组特殊的函数，它们被声明于一个专门的头文件，并在一个独立的库中予以实现，使用这组函数需要包含此头文件，并链接该库
1. #include <dlfcn.h> 2. -ldl
2. 加载共享库
将共享库载入内存并获得其访问句柄
void *dlopen(const char *filename, int flag); 1. *filename: 共享库路径，若只提供文件名，则根据LD_LIBRARY_PATH环境变量搜索 2. flag: 加载方式 1) RTLD_LAZY: 延迟加载，只有当使用到了共享库中的符号(如调用库中的函数)时才加载 2) RTLD_NOW: 立即加载
函数返回的共享库访问句柄唯一地标识了系统内核所维护的共享库对象，将作为后续函数调用的参数
3. 获取函数地址
从指定共享库中获取与给定函数名对应的函数入口地址
void *dlsym(void *handle, const char *symbol); 1. *handle: 共享库访问句柄 2. 函数名 //返回的函数指针是void*类型，需要强制转换为实际函数指针类型才能调用
4. 卸载共享库
从内存中卸载共享库
int dlclose(void *handle); 1. *handle: 共享库访问句柄
关于共享库的卸载，有几点需要明白
1. 所卸载的共享库未必真的从内存中立即消失，因为其他程序可能还需要使用该库 2. 只有所有使用该库的程序都显式或隐式地卸载了该库(例如退出)，该库所占用的内存空间才会真正得到释放 3. 无论所卸载的共享库是否真正被释放，传递给dlclose函数的句柄参数都会在该函数成功返回后立即失效
5. 获取错误信息
char *dlerror(void); //如果发生错误则返回指向上一次错误的错误信息字符串的指针，否则返回NULL
6. 代码示例
#include <stdio.h> #include <dlfcn.h> #include <stdlib.h> int main(){ void* handle = dlopen("libmath.so", RTLD_LAZY); if(!handle){ fprintf(stderr, "dlopen: %s ", dlerror()); exit(EXIT_FAILURE); } double (*add)(double, double) = (double(*)(double, double))dlsym(handle, "add"); if(!add){ fprintf(stderr, "dlsym: %s ", dlerror()); exit(EXIT_FAILURE); } void (*show)(double) = (void(*)(double))dlsym(handle, "show"); if(!show){ fprintf(stderr, "dlsym: %s ", dlerror()); exit(EXIT_FAILURE); } printf("5.3 + 2.8 = "); show(add(5.3, 2.8)); if(dlclose(handle)){ fprintf(stderr, "dlclose: %s ", dlerror()); exit(EXIT_FAILURE); } return 0; } //gcc dl.c -ldl
8. 和静态库/动态库相关的辅助工具
0x1: 查看符号表
列出目标文件、可执行文件、静态库或共享库中的符号
nm libmath.so
0x2: 反汇编
显示二进制模块的反汇编信息
objdump -S a.out
0x3: 去除冗余信息
去除目标文件、可执行文件、静态库和共享库中的符号表、调试信息等
strip a.out
0x4: 查看共享库依赖
查看可执行文件或共享库所依赖的共享库文件
ldd a.out
Copyright (c) 2016 LittleHann All rights reserved