Dynamic linking defers some address resolution until load or call time. The program records dependencies and symbol requirements; the loader finds the objects, establishes mappings, and performs relocations. This allows libraries to be updated independently while maintaining the ABI. Clean code pages of the same library file can be shared among multiple processes, whereas private data and copy-on-write pages cannot be considered shared in the same way.
When troubleshooting failures, distinguish in order: whether the library file was found, whether the target symbol is in the correct scope, whether the symbol version matches, and finally whether the parameters and object layout at call time are compatible. Identical filenames do not imply identical ABIs.
Finding Libraries, Finding Symbols, and Writing Addresses Are Three Steps
DT_NEEDED records dependency names. After finding the library, the loader uses the dynamic table to locate symbols, strings, versions, and relocation information. Relocations tell the loader "where and how to compute and write," while symbol lookup determines which definition a given name corresponds to within the current scope.
Thus, loading libfoo.so does not mean all foo references are necessarily bound to it; pluggable global symbols may be satisfied by definitions in other objects. Symbols are not just functions; data, TLS, and IFUNCs have their own handling requirements. ELF Dynamic Linking
Why PLT/GOT Supports Lazy Binding
Traditional x86-64 PLT stubs jump via the GOT slot first. When unresolved, it leads to the loader's resolution path; the resolver determines the function address and usually updates the slot; subsequent calls reuse this result. The specific number of reserved slots, instruction forms, and relocation index encoding are architecture implementation details and should not be treated as a universal layout for all ELF binaries.
Lazy binding primarily applies to function call relocations where applicable; data references cannot arbitrarily wait until the first read to patch the address. -z now or LD_BIND_NOW request early binding; compiler options like -fno-plt can also change the call form. Therefore, "the cost of dynamic linking is that each function is slightly slower the first time" is incomplete.
The GOT is not just an "array of global variable addresses." Different targets and relocation types place function addresses, data addresses, and runtime support information in respective locations. errno is a thread-local object and should not be used as a teaching example for ordinary global variables via extern int errno.
Which slot changes on the first call?
Call foo twice, then bar. The second foo reuses its resolved slot while the first bar may still enter the resolver. Current slot contents and the last call path are separate observations.
Preparing the visual
Which slot changes on the first call?
Lazy binding resolves applicable slots independently. Calling one symbol need not resolve another; eager binding does this work at load time.
The experiment should print 42. During linking, -lfoo finds libfoo.so; the library's SONAME causes the executable to usually record libfoo.so.1; $ORIGIN in RUNPATH allows the loader to find it in the directory where the main program resides. Single quotes prevent the shell from expanding $ORIGIN prematurely. Bindings for libc and others will also appear in LD_DEBUG; locate your own call by looking for foo.
Search Paths: RUNPATH Does Not Automatically Propagate to Grandchildren Dependencies
The following applies to the glibc dynamic linker. For dependency names without slashes, the generalized search order is:
Applicable DT_RPATH, provided there is no corresponding DT_RUNPATH.
LD_LIBRARY_PATH, which is ignored in secure execution mode.
The DT_RUNPATH of the initiating dependent object, used for its direct DT_NEEDED dependencies.
/etc/ld.so.cache, containing the system cache and architecture-specific directory selections.
Default system library directories, whose paths and whether they are skipped depend on the architecture and link options.
Dependencies containing slashes are treated as paths and do not undergo the same name-based search. One key difference between RPATH and RUNPATH is the scope of propagation: the main program's RUNPATH finding libA does not imply it can also find libB, which libA requires; libA usually needs its own appropriate search configuration. $ORIGIN expands based on the location of the object containing the marker, not the current working directory. ld.so(8)
Build-time -L does not automatically become a runtime path. If you copy the library during deployment but encounter "file not found," first check NEEDED, RUNPATH, and actual filenames before considering symbol issues.
Reproducing "Main program can find A, but cannot find B" with Two-Level Dependencies
Create bin and lib directories in a new experiment directory. Here, deliberately place both libraries in the same directory, but in the first version, A does not set RUNPATH. The three source files are as follows.
In a clean environment without additional LD_LIBRARY_PATH, preloading, or system library name collisions, the last step should report that libB.so.1 cannot be found. app's RUNPATH has already allowed it to find A, but B is a direct dependency of A. The -rpath-link,lib at link time only helps the linker find indirect dependencies; it does not write this directory into A's runtime path. GNU ld Path Options
Rebuild only A, adding its own RUNPATH:
cc -fPIC -shared chain-a.c -Llib -l:libB.so.1 \
-Wl,-soname,libA.so.1,--enable-new-dtags,-rpath,'$ORIGIN'\
-o lib/libA.so.1./bin/app
This should output 42. app does not need to be relinked, as it still records libA.so.1. $ORIGIN inside A expands to the directory where lib resides; even if app is executed from a different working directory, this relative object location relationship holds. Relative path dlopen("./plugin.so", ...) is interpreted by the current working directory and should not be confused with the concept of $ORIGIN.
Check Location
First Version
Fixed Version
app's NEEDED
libA.so.1
Unchanged
app's RUNPATH
$ORIGIN/../lib
Unchanged
A's NEEDED
libB.so.1
Unchanged
A's RUNPATH
None
$ORIGIN
Execution Result
Startup fails when B is not found
Prints 42 in this example
Putting directories uniformly into LD_LIBRARY_PATH might also make this example succeed, but that changes the process's search environment and cannot be used to demonstrate that A possesses the path information needed for independent deployment. When troubleshooting, record environment values and dynamic tags within objects separately.
Who needs the library, who supplies its path?
Search configuration belongs to the object requesting a dependency. app and A initiate different edges; A needs its own path for the second edge. Changing the working directory does not change these $ORIGIN expansions.
Preparing the visual
Who needs the library, who supplies its path?
RUNPATH serves the owning object’s direct dependencies. Finding A does not ensure A finds B; $ORIGIN belongs to the object carrying the tag.
cc -fPIC -shared override.c -o override.soLD_PRELOAD="$PWD/override.so"./dyn-demo
In this example with default pluggable symbols, it should print 141. This does not mean it can override arbitrary code: static binding, hidden visibility, direct calls within libraries, compiler inlining, or certain link optimizations may cause calls to bypass pluggable dynamic lookup. Secure execution mode also restricts preloading.
Wrapping open with LD_PRELOAD can be used for observation or adaptation, but this does not enable complete isolation: the program may call other interfaces or execute system calls directly. Understanding these boundaries helps determine whether "preload is not working" is due to a search issue or a call path issue.
dlopen: Incorporating Object Lifecycle into the Interface
Below continues loading the previously mentioned libfoo.so.1. The result of POSIX's dlsym can be used for corresponding function pointers; this is a POSIX environment convention and cannot be generalized as an object/function pointer conversion rule for arbitrary ISO C platforms.
#include<dlfcn.h>#include<stdio.h>intmain(void){void*handle =dlopen("./libfoo.so.1", RTLD_NOW | RTLD_LOCAL);if(handle ==NULL){fprintf(stderr,"%s\n",dlerror());return1;}dlerror();int(*foo)(int)=(int(*)(int))dlsym(handle,"foo");constchar*error =dlerror();if(error !=NULL){fprintf(stderr,"%s\n", error);dlclose(handle);return1;}/* In this example, foo is known to be a callable function; a NULL value for general symbols does not necessarily indicate a lookup error. */printf("%d\n",foo(41));if(dlclose(handle)!=0){fprintf(stderr,"%s\n",dlerror());return1;}return0;}
Compile: cc -Wall -Wextra plugin-host.c -ldl -o plugin-host. Running it should print 42.
RTLD_NOW resolves needed undefined symbols in advance; RTLD_LAZY only delays function bindings where applicable; the caller must choose the appropriate mode and cannot broadly state that LAZY is the default for all calls. RTLD_LOCAL restricts symbols from entering the global scope; it does not mean only this handle can see the object.
Clear old dlerror first, then check for new errors after dlsym. Checking only if the returned pointer is NULL is insufficient to cover all symbol cases. dlclose decrements the reference count but does not guarantee immediate unloading; regardless of whether it is actually unloaded, cached function pointers must not be used after the lifecycle is no longer guaranteed. dlopen(3), dlsym(3)
Before Unloading, There Is More Than Just Function Pointers to Reclaim
Assume the plugin creates an object and returns it to the host; the object's destructor, vtable, or callbacks still point to plugin code. Even if the host no longer calls the original foo, destructing this object may still jump back to the plugin. Therefore, interfaces should ideally provide create/destroy pairs, making allocation and release responsibilities clear, and requiring objects to be destroyed before the last valid library reference is closed.
The closing process can be understood as a state change with preconditions:
State
Allowed Operations
Condition to Enter Next State
Accepting Work
Create objects, submit calls, register callbacks
Stop accepting new work at entry points
Draining
Wait for existing calls and tasks to finish
Plugin is no longer running, async callbacks are revoked and not executing
Reclaiming
Call plugin's own destruction interfaces
All data and callbacks requiring plugin code are released
Closing Handle
dlclose, remove host cache entry
No subsequent paths enter the plugin
These steps require the host's own synchronization protocol; RTLD_LOCAL only affects lookup scope and does not wait for worker threads on behalf of the host, nor does it constitute memory isolation. If the callback revocation API only guarantees "no new callbacks will start," the host must still wait for started callbacks to return. Retaining valid library references for objects is a common design, but ring dependencies between objects and task convergence during plugin exit must still be handled.
Symbol Versions and IFUNC
GNU symbol versions allow different versions to be associated with the same name; foo@@VERS_2.0 usually represents the default version, while foo@VERS_1.0 represents a non-default older version. The version requirements recorded by the caller must be met by the provider; changing a SONAME or creating a symlink will not fix parameter, struct layout, or version incompatibilities.
IFUNC allows the resolver to choose the actual implementation, such as selecting a function based on available CPU features. The resolution timing may be very early or related to specific binding paths; the resolver cannot assume that normal application initialization is complete. When reading startup anomalies, IFUNC is one of the early execution paths to check.
ABI Compatibility Requires Checking One Layer Beyond "Finding the Symbol"
Suppose the plugin entry is still called process, but the old version accepts two integers, while the new version accepts a struct pointer. The loader can successfully find this name, but the call may not conform to the convention; the dynamic symbol table usually does not verify the complete parameter types of C functions for you.
Layer
Example
What Passing This Layer Still Doesn't Prove
File Identity
SONAME satisfies libfoo.so.1
Content implements the original ABI
Name and Version
foo, required version nodes exist
Parameters, return values, and object layout match host expectations
Call Contract
Fixed-width fields, sizes, calling conventions
Ownership and concurrent usage are correct
Lifecycle Contract
create/destroy pairing, draining before close
Business semantics of old and new implementations are exactly the same
Custom plugin ABIs can include version numbers and table sizes in the interface table returned at the entry point, allowing the host to check for existence before reading optional tail fields. This design requires both parties to agree on a stable initial layout and compatibility strategy first; arbitrarily inserting fields into the middle of an old struct and then bumping the version number does not allow the old host to run safely.
Quick Troubleshooting
Symptom
Check First
cannot open shared object file
NEEDED, SONAME, path, target architecture
undefined symbol
Dynamic symbol exports, dependencies, lookup scope
version not found
readelf -V output and deployed library versions
preload ineffective
Visibility, direct calls, inlining, secure execution mode
occasional crash after dlclose
Object, function pointer, and worker thread lifecycles
readelf -dW, readelf --dyn-syms -W, and readelf -rW answer dependency, dynamic symbol, and relocation issues, respectively. Ordinary nm relies on symbol tables that may have been removed; to view dynamic symbols, use nm -D. The experiments in this article are based on glibc; musl's loader implementation and diagnostic options should be verified against its documentation separately.