Introduction and Implementation of TLS
- kyy
Overview
TLS, Thread Local Storage, also known as Thread-Local Storage variables, refers to individual storage for each thread that is not shared among processes. For TLS variables, different threads point to different storage spaces. Its implementation involves support from high-level programming languages, compilers, and linkers.
Allocation of TLS Variables
In high-level programming languages, when defining and allocating TLS variables in a multi-threaded environment, modifications to TLS variables by different threads do not affect each other. TLS variables can be accessed for read and write operations in assignment statements just like regular variables.
Dynamic Management of Threads
Except for the main thread, other threads are created and destroyed during program execution. When loading shared object files, it is not possible to determine when and how many threads will be created. Therefore, dynamic management is required for the TLS variable libc. Since different threads access different address spaces of TLS variables, libc needs to manage the address space of TLS variables and dynamically obtain their addresses.
The compiler and linker place all TLS variables in the same TLS program segment in a dynamic library. The offset of TLS variables relative to the TLS program segment is fixed.
Implementation of TLS Variables
To optimize the performance of accessing TLS variables, the implementation of TLS variables adopts four approaches, with increasing performance and decreasing implementation scenarios:
Generic Dynamic: The generic approach requires a function call to obtain the address of each TLS variable. Can be accessed across dynamic library references.
Local Dynamic: The local approach is used when multiple TLS variables are referenced within a function. The address of the TLS program segment is obtained through a function call, and the address of TLS variables is obtained through the offset from the TLS program segment. Can only be accessed within the dynamic library.
Initial Exec: Indirect addressing between segment registers and TLS variable offsets. Used for referencing and accessing TLS variables in dynamically loaded libraries at the start of the main program.
Local Exec: Direct addressing between segment registers and TLS variable offsets. Can only be accessed within the main program.
Additionally, there is an optimized access mode called TLSDESC, which optimizes access to TLS variables in the static TLS block. It reduces register contamination caused by function calls and minimizes atomic operation access, among other optimizations.
Compiler Access Modes for TLS Variables
The compiler generally supports five access modes through the tls-dialect and tls-model options:
-mtls-dialect=trad -ftls-model=global-dynamic : GD
-mtls-dialect=trad -ftls-model=local-dynamic ：LD
-mtls-dialect=trad -ftls-model=initial-exec ：IE
-mtls-dialect=trad -ftls-model=local-exec ：LE, supports the main program and does not support dynamic libraries.
-mtls-dialect=desc ：TLSDESCThese options determine how the compiler generates code to access TLS variables.
Note: The above compilation options are for aarch64 architecture. For x86_64 architecture, the tls-dialect compilation options have different values: gnu and gnu2, which correspond to trad and desc, respectively.
The TLS variable modifiers in C/C++
Specifier | Notes |
|---|---|
__thread |
|
| |
_Thread_local |
|
| |
thread_local |
|
|
Introduction to TLS Data Structure
Drepper provides two implementations of memory layout based on different scenarios: static loading and dynamic loading of shared objects. The basic principles are similar.
Define a dynamic thread-specific array called dtv (dynamic thread vector), where each element points to the position of the TLS program segment content of the dynamic library in the current thread. This array can dynamically grow.
Allocate a static block of physical memory called the static TLS block. Once allocated, it cannot be increased or decreased. Most of the memory is used to store the contents of the TLS segment, and a small portion is used to store the Thread Control Block (TCB), which points to the dtv array.
For dynamically loaded dynamic libraries, allocate a memory block called the dynamic TLS block for each TLS program segment of the dynamic library. The dtv elements point to this memory block.
Use a dedicated segment register in the CPU to store the thread pointer address, such as the %fs segment register on x86_64 or the tpidr_el0 segment register on aarch64.
TLS Data Layout
Here, we present the TLS data layout for arrch64 in both the GNU and Bionic implementations. The GNU implementation is similar to the structure shown in Figure 1, while the TCB definition differs in the Bionic implementation. Other aspects remain similar.
In the physical layout of the Bionic thread stack shown in Figure 3, the address increases from top to bottom. The allocation of the static TLS block memory occurs on the thread stack. The size of the TCB is determined by the size of the bionic_tcb data structure, which may not be 16 bytes. The static TLS block is located immediately after the bionic_tcb data structure.
In the physical layout of the GNU thread stack shown in Figure 4, the address increases from bottom to top. The allocation of the static TLS block memory occurs on the thread stack. The size of the TCB is determined by the tcbhead_t data structure, which is typically 16 bytes. The static TLS block is located immediately after the tcbhead_t data structure.
Static TLS Space:
Static TLS space refers to a contiguous physical memory region allocated on the thread stack. It encompasses both the static TLS space and TLS management space. The dynamic library's tls_offset is calculated relative to this space.
Thread Pointer:
The thread pointer needs to be stored in a segment register. In Variant 1, the TCB's space allocation needs to be consistent with the compiler to meet the requirements of LE (little-endian) access. For example, in Bionic, the TCB occupies 64 bytes, requiring the compiler to generate instructions for accessing main program TLS variables with an additional offset of 64. In Variant 2, the offset is conventionally set to 0, eliminating this requirement.
TCB Management Space:
The TCB management space records addresses for the thread pointer, dynamic thread vector (dtv), and management of thread-local data. In Bionic's aarch64 implementation, the space is defined as nine arrays of eight bytes each, totaling 72 bytes. It is larger than the offset between the thread pointer and the first TLS variable by 8 bytes, causing the thread pointer to point to the address of the second element. Additionally, the TCB management space may have alignment requirements, which can result in a different starting address compared to the static TLS space. This may vary depending on the specific implementation of Bionic.
Dynamic Thread Vector (dtv):
The dynamic thread vector records the starting positions of the TLS program segments for each dynamic library. Its address is stored in the space specified by the thread pointer.
Initialization Process of TLS Data Structure in Bionic
The initialization of the TLS data structure in Bionic by the dynamic linker is divided into two parts: static loading of libraries during the main program's loading process and dynamic loading of libraries using dlopen during the main program's execution.
TLS Initialization of TLS in Static Loading Libraries
During the loading process of the main program, the dynamic linker initializes the TLS data structure using two global variables: StaticTlsLayout and TlsModules.
StaticTlsLayout is of type StaticTlsLayout and is used to calculate the offset of each dependency library's TLS program segment in the static TLS block.
TlsModules is of type TlsModules and is used to calculate all the dynamic libraries that have TLS program segments.
During the main program loading process, all TLS program segments are stored in the static TLS block memory. The specific steps are as follows：
Initialization of bionic_tcb and tpidr_el0: The function libc_init_main_thread_early is called to initialize bionic_tcb and tpidr_el0. This involves setting up the initial values for bionic_tcb and setting the tpidr_el0 register.
Initialization of TLS_SLOT_DTV in bionic_tcb: The function init_tcb_dtv is called to initialize the TLS_SLOT_DTV field in the bionic_tcb structure. The update flag for this field is set to 0.
Calling linker_setup_exe_static_tls: This function is called assuming that the main program contains TLS program segments. The following actions are taken:
a. Reserve space and determine the position of the bionic_tcb object, TLS space, and their locations in a variable of type StaticTlsLayout.
b. Call register_tls_module to obtain the module ID and add it to a variable of type TlsModules.
c. Reserve space and determine the position of the bionic_tls object in the variable.Loading of TLS program segments in the main program dependencies: If there are TLS program segments in the main program's dependencies, the function soinfo::register_soinfo_tls is called. This reserves space for TLS segments and their positions in the StaticTlsLayout variable, and adds them to the TlsModules variable.
Calling linker_finalize_static_tls: This function calculates the total size of the reserved space in the StaticTlsLayout variable, which represents the size of the static TLS block.
Calling __allocate_thread_mapping: This function allocates memory space on the main program's stack for the static TLS block. The static TLS block includes bionic_tcb, bionic_tls, and TLS program segments.
Calling __init_static_tls: This function copies the contents of the TLS segments from the dynamic libraries specified in the TlsModules variable into the reserved space of the static TLS block.
Calling bionic_tcb::copy_from_bootstrap: This function synchronizes the contents of bionic_tcb with the initial values set during the first step of initialization.
Calling __init_tcb: This function updates bionic_tcb with additional information.
Calling __init_bionic_tls_ptrs: This function updates the addresses of bionic_tls.
Relocation Initialization of TLS Variable's GOT Table Entries
GD (Global Dynamic): The relocation types for GD include R_AARCH64_TLS_DTPMOD64 and R_AARCH64_TLS_DTPREL64. These types initialize the adjacent GOT table entries with the module ID and the offset of the variable within its TLS segment, respectively.
LD (Local Dynamic): The implementation of LD is the same as GD for AArch64. However, for x86_64, the relocation type is R_X86_64_DTPMOD64, and the adjacent GOT table entries are initialized with the module ID and an offset of 0.
IE (Initial Executable): The relocation type for IE is R_AARCH64_TLS_TPREL64. It initializes the value of the corresponding GOT table entry with the offset on the static TLS block.
LE (Local Executable): There are no relocation entries or GOT table entries to initialize, so no initialization is needed.
TLSDESC: The relocation type for TLSDESC is R_AARCH64_TLSDESC. It initializes the adjacent GOT table entries with the address of the
tlsdesc_resolver_staticfunction and the offset on the static TLS block.
From the initialization process described above, it can be seen that the dtv (Dynamic Thread Vector) array is initially empty. The dtv array is only created when accessing TLS variables. This delayed allocation has the following benefits:
For IE and LE access to TLS variables, the address of the TLS variable can be obtained directly using the segment register and the offset, without the need to look up the
dtvarray.For GD and LD access to TLS variables, the
__tls_get_addrfunction is called to obtain the TLS variable's address. When checking that thedtvarray is empty, thedtvarray is reallocated based on the number of TLS dynamic libraries.For TLSDESC access to TLS variables, the
tlsdesc_resolver_staticfunction directly returns the offset of the TLS variable on the static TLS block, without the need to allocate thedtvarray.
From the examples mentioned above, it is evident that the TLSDESC access mode provides significant optimization for accessing TLS variables on the static TLS block compared to GD. The TLSDESC mode directly returns the offset of the static TLS block in the GOT table entry, while GD requires accessing the dtv array to calculate the address.
Dynamic Loading Library Initialization of TLS
In bionic, when dynamically loading a library through dlopen, the initialization steps for TLS are as follows:
In the do_dlopen -> find_library -> soinfo::register_soinfo_tls -> register_tls_module flow, the module ID is obtained and added to the TlsModules variable.
In the soinfo::relocate -> plain_relocate -> plain_relocate_impl -> process_relocation -> process_relocation_impl flow, the relocation initializes the TLS variables' GOT table entries.
For GD (Global Dynamic): The relocation types include R_AARCH64_TLS_DTPMOD64 and R_AARCH64_TLS_DTPREL64. The adjacent GOT table entries are initialized with the module ID and the variable's offset within its TLS segment.
For LD (Local Dynamic): The implementation for AArch64 is the same as GD.
For IE (Initial Executable) / LE (Local Executable):They are not supported.
For TLSDESC: The relocation type is R_AARCH64_TLSDESC. The adjacent GOT table entries are initialized with the address of the tlsdesc_resolver_dynamic function and the address of the TlsDynamicResolverArg variable.
a. Initialize the TlsIndex in TlsDynamicResolverArg with the module ID and the offset of the TLS variable within its TLS program segment. Additionally, initialize the update flag with the library's update flag, which indicates whether the dynamic library has been updated.
b. To store the TlsDynamicResolverArg variable, it is saved in the soinfo::tlsdescargs array. To handle reallocation of the array, the Relocator::deferred_tlsdesc_relocs buffer defers relocation information. The TLS variable's GOT table entries are updated once all relocation operations for the library are completed.
Initialization of TLS during Thread Creation
When creating a thread using pthread_create, the following steps are involved in initializing TLS:
The TLS data structures on the main program are copied. This is done within the pthread_create function, specifically in the __allocate_thread function.
The __allocate_thread_mapping function is called to allocate the thread's stack space, which includes the static TLS block. The allocation order includes the stack guard, stack, static TLS block, and guard page.
The __init_static_tls function is called to copy the contents of the TLS segment from the TlsModules variable of dynamic libraries to the static TLS block.
The __init_tcb function is called to update the bionic_tcb (Thread Control Block).
The __init_tcb_dtv function is called to initialize the TLS_SLOT_DTV in the bionic_tcb, with the update flag set to 0.
The __init_bionic_tls_ptrs function is called to update the bionic_tls addresses.
The clone system call is invoked, passing the address of the static TLS block to clone. The kernel then sets the value of the tpidr_el0 register, which represents the thread pointer, according to the provided static TLS block address.
Implementation of the __tls_get_addr Function
The __tls_get_addr function is used by GD (Global Dynamic) and LD (Local Dynamic) access methods to retrieve the absolute address of a TLS variable. This function involves updating the dtv (Dynamic Thread Vector) data, and the update conditions are controlled by three generation flags.
Global Generation: The global generation is stored in __libc_tls_generation_copy, which is a copy of TlsModules::generation. Each time a dynamic library with a TLS program segment is added, this value is incremented to indicate the addition of a new dynamic library. There is no need to handle dynamic library removal.
Generation in dtv Array: The generation value in the dtv array is stored in the first element of the array, initialized to 0. When updating the dtv array, the generation is updated to match the current global generation value. If it is not equal to the global generation, it indicates that a new dynamic library has been loaded, and the contents of the dtv array need to be updated.
Generation in Dynamic Library: The generation value in the dynamic library is stored in TlsModule::first_generation. This value is initialized with the global generation value when the library is loaded. It is used to determine if the dynamic library pointed to by dtv has changed, i.e., whether it is an old dynamic library.
struct TlsIndex {
size_t module_id;
size_t offset;
};
// The value of "ti" (Thread Index) is stored in the GOT (Global Offset Table) entries of the dynamic library. It is initialized during relocation and occupies two entries in the table.
extern "C" void* __tls_get_addr(const TlsIndex* ti){
// get the dtv
TlsDtv* dtv = __get_tcb_dtv(__get_bionic_tcb());
// retrieve the global dynamic library update flag
size_t generation = atomic_load(&__libc_tls_generation_copy);
if (__predict_true(generation == dtv->generation)) {
void* mod_ptr = dtv->modules[__tls_module_id_to_idx(ti->module_id)];
if (__predict_true(mod_ptr != nullptr)) {
return static_cast<char*>(mod_ptr) + ti->offset + TLS_DTV_OFFSET;
}
}
return tls_get_addr_slow_path(ti);
}The tls_get_addr_slow_path function includes the allocation and initialization of dtv (Dynamic Thread Vector) and the dynamic TLS block.
__attribute__((noinline)) static void* tls_get_addr_slow_path(const TlsIndex* ti) {
TlsModules& modules = __libc_shared_globals()->tls_modules;
bionic_tcb* tcb = __get_bionic_tcb();
ScopedSignalBlocker ssb;
// To prevent multiple threads from simultaneously modifying the __libc_shared_globals()->tls_modules global variable, you can use a mutex to enforce mutual exclusion
ScopedWriteLock locker(&modules.rwlock);
// update the dtv array or reallocate its memory
update_tls_dtv(tcb);
TlsDtv* dtv = __get_tcb_dtv(tcb);
const size_t module_idx = __tls_module_id_to_idx(ti->module_id);
void* mod_ptr = dtv->modules[module_idx];
if (mod_ptr == nullptr) {
// If the dtv array does not exist, you would need to allocate memory, copy the contents of the dynamic library's TLS program segment to the new memory, and initialize the module pointer.
const TlsSegment& segment = modules.module_table[module_idx].segment;
mod_ptr = __libc_shared_globals()->tls_allocator.memalign(segment.alignment, segment.size);
if (segment.init_size > 0) {
memcpy(mod_ptr, segment.init_ptr, segment.init_size);
}
dtv->modules[module_idx] = mod_ptr;
// Reports the allocation to the listener, if any.
if (modules.on_creation_cb != nullptr) {
modules.on_creation_cb(mod_ptr, static_cast<void*>(static_cast<char*>(mod_ptr) + segment.size));
}
}
return static_cast<char*>(mod_ptr) + ti->offset + TLS_DTV_OFFSET;
}The steps to implement the update_tls_dtv function for dynamically allocating dtv array space are as follows:
Condition for updating the dtv array: If the update flag of the dtv array is not equal to the global dynamic library update flag, it indicates that the dynamic library has been updated and the current dtv array points to the old dynamic library.
Reallocate the dtv array if the total number of dynamic libraries with TLS program segments is greater than the size of the dtv array.
Allocate new space for the dtv array based on the number of dynamic libraries.
Backup the contents of the dtv array to the new space.
Call __set_tcb_dtv to update to the new dtv array.
To achieve lock-free operation, don't free the old dtv array space. Instead, insert it into a garbage collection queue for later cleanup at the end of the program.
Update the dtv array elements corresponding to the static TLS block and dynamic TLS blocks of the dynamic libraries.
Free the memory of the old dynamic TLS block
Clear the dtv array element.
Update the update flag of the dtv array to the global dynamic library update flag.
The access method of TLSDECS
The TLSDESC access method provides two ways to obtain the offset address of a TLS variable relative to the static TLS block: tlsdesc_resolver_static for TLS variables on the static TLS block and tlsdesc_resolver_dynamic for TLS variables on the dynamic TLS block. Both of these methods are implemented using assembly language and do not adhere to the register parameter passing conventions of C/C++ function calls. Instead, they use the return value register for passing parameters, such as the x0 register for AArch64 or the rax register for x86_64.
The implementation of tlsdesc_resolver_static is straightforward as it simply returns the value of TlsDescriptor::arg.
For tlsdesc_resolver_dynamic, there are several optimizations based on the update flags:
Global generation: Stored in __libc_tls_generation_copy, which is a copy of TlsModules::generation. It is incremented each time a dynamic library with TLS program segments is added, indicating the addition of a new dynamic library. There is no need to handle dynamic library removal.
Generation in the dtv array: Stored in the first element of the dtv array, dtv[0]. It is initialized to 0 and updated to the value of the global generation whenever the dtv array is updated. If it is different from the global generation, it indicates that new dynamic libraries have been loaded, and the dtv array needs to be updated.
Generation in the dynamic library: Stored in TlsModule::first_generation. It is initialized with the value of the global generation when the library is loaded. This value is used to determine if the dynamic library pointed to by dtv has changed, indicating an old dynamic library.
Generation in the TLS variable's GOT entry: Stored in TlsDynamicResolverArg::generation. It is initialized with TlsModule::first_generation. As long as it is not greater than the generation in the dtv array, there is no need to reallocate the dtv array. If it is greater, it means that the TLS program segment of that dynamic library is not initialized in the dtv array.
The implementation steps for tlsdesc_resolver_dynamic are as follows:
Fast path (conditions: TlsDynamicResolverArg::generation <= dtv[0] && dtv[mod_id] != NULL): Return dtv[mod_id] + TlsDynamicResolverArg::TlsIndex::offset as the relative offset to the static TLS block.
Slow path:
Call __tls_get_addr to obtain the absolute address of the TLS variable. and Calculate the relative offset to the static TLS block.
The initialization process of the GNU TLS data structure
It is similar to the Bionic TLS data structure, and the principles are comparable. The differences between them include:
When initializing TLS for statically loaded libraries, GNU reserves 144 bytes of space for the static TLS block.
When initializing TLS for dynamically loaded libraries, GNU first tries to obtain space from the reserved static TLS block. If it is insufficient, it falls back to the dynamic TLS block. This approach allows dynamic libraries using the IE access mode for TLS variables to be accessed properly when loaded with dlopen.
The function names for TLSDESC implementation differ: _dl_tlsdesc_return, _dl_tlsdesc_dynamic, and _dl_tlsdesc_undefweak in GNU correspond to tlsdesc_resolver_static, tlsdesc_resolver_dynamic, and tlsdesc_resolver_unresolved_weak respectively.
The purpose of reserving the static TLS space is twofold. Firstly, it supports dynamic libraries loaded with IE access mode for TLS variables. Secondly, it optimizes the performance of the TLSDESC access mode.
In the Linux environment, a typical use case for reserving static TLS space is in graphics acceleration libraries (such as OpenGL/EGL). Linux applications generally use graphics API dispatch libraries like glvnd, which dynamically load OpenGL/EGL libraries using dlopen. These OpenGL/EGL libraries often use TLS variables in IE access mode, typically a pointer variable pointing to a data structure. This reduces the space occupied by the reserved static TLS block.
Some considerations regarding reserving static TLS space are as follows:
Allocation Timing: glibc attempts to allocate the static TLS space during relocation and supports two relocation types: R_AARCH64_TLSDESC and R_AARCH64_TLS_TPREL.
Initialization Data: The static TLS space for all threads is initialized. The TLS data structures are located on the thread stack, with some threads using the user stack and others using the system stack. Initialization is performed in the _dl_init_static_tls function.
Concurrent Access: The allocation of reserved static TLS space is implemented in dl_open_worker_begin, and a large lock (dl_load_tls_lock) is acquired before calling the function. When initializing static TLS data, a lock (dl_stack_cache_lock) is obtained within the _dl_init_static_tls function to ensure thread safety.