Introduction and Implementation of TLS

Overview​

TLS, Thread Local Storage, also known as Thread-Local Storage variables, refers to individual storage for each thread that is not shared among processes. For TLS variables, different threads point to different storage spaces. Its implementation involves support from high-level programming languages, compilers, and linkers.

Allocation of TLS Variables​

In high-level programming languages, when defining and allocating TLS variables in a multi-threaded environment, modifications to TLS variables by different threads do not affect each other. TLS variables can be accessed for read and write operations in assignment statements just like regular variables.

Dynamic Management of Threads

Except for the main thread, other threads are created and destroyed during program execution. When loading shared object files, it is not possible to determine when and how many threads will be created. Therefore, dynamic management is required for the TLS variable libc. Since different threads access different address spaces of TLS variables, libc needs to manage the address space of TLS variables and dynamically obtain their addresses.

The compiler and linker place all TLS variables in the same TLS program segment in a dynamic library. The offset of TLS variables relative to the TLS program segment is fixed.

Implementation of TLS Variables

To optimize the performance of accessing TLS variables, the implementation of TLS variables adopts four approaches, with increasing performance and decreasing implementation scenarios:

1. Generic Dynamic: The generic approach requires a function call to obtain the address of each TLS variable. Can be accessed across dynamic library references.

2. Local Dynamic: The local approach is used when multiple TLS variables are referenced within a function. The address of the TLS program segment is obtained through a function call, and the address of TLS variables is obtained through the offset from the TLS program segment. Can only be accessed within the dynamic library.

3. Initial Exec: Indirect addressing between segment registers and TLS variable offsets. Used for referencing and accessing TLS variables in dynamically loaded libraries at the start of the main program.

4. Local Exec: Direct addressing between segment registers and TLS variable offsets. Can only be accessed within the main program.

Additionally, there is an optimized access mode called TLSDESC, which optimizes access to TLS variables in the static TLS block. It reduces register contamination caused by function calls and minimizes atomic operation access, among other optimizations.

Compiler Access Modes for TLS Variables

The compiler generally supports five access modes through the tls-dialect and tls-model options:

-mtls-dialect=trad -ftls-model=global-dynamic : GD
-mtls-dialect=trad -ftls-model=local-dynamic ：LD
-mtls-dialect=trad -ftls-model=initial-exec  ：IE
-mtls-dialect=trad -ftls-model=local-exec  ：LE, supports the main program and does not support dynamic libraries.
-mtls-dialect=desc ：TLSDESC

These options determine how the compiler generates code to access TLS variables.

Note: The above compilation options are for aarch64 architecture. For x86_64 architecture, the tls-dialect compilation options have different values: gnu and gnu2, which correspond to trad and desc, respectively.

The TLS variable modifiers in C/C++​

Introduction to TLS Data Structure​

Drepper provides two implementations of memory layout based on different scenarios: static loading and dynamic loading of shared objects. The basic principles are similar.

1. Define a dynamic thread-specific array called dtv (dynamic thread vector), where each element points to the position of the TLS program segment content of the dynamic library in the current thread. This array can dynamically grow.

2. Allocate a static block of physical memory called the static TLS block. Once allocated, it cannot be increased or decreased. Most of the memory is used to store the contents of the TLS segment, and a small portion is used to store the Thread Control Block (TCB), which points to the dtv array.

3. For dynamically loaded dynamic libraries, allocate a memory block called the dynamic TLS block for each TLS program segment of the dynamic library. The dtv elements point to this memory block.

4. Use a dedicated segment register in the CPU to store the thread pointer address, such as the %fs segment register on x86_64 or the tpidr_el0 segment register on aarch64.

TLS Data Layout​

Here, we present the TLS data layout for arrch64 in both the GNU and Bionic implementations. The GNU implementation is similar to the structure shown in Figure 1, while the TCB definition differs in the Bionic implementation. Other aspects remain similar.

In the physical layout of the Bionic thread stack shown in Figure 3, the address increases from top to bottom. The allocation of the static TLS block memory occurs on the thread stack. The size of the TCB is determined by the size of the bionic_tcb data structure, which may not be 16 bytes. The static TLS block is located immediately after the bionic_tcb data structure.

In the physical layout of the GNU thread stack shown in Figure 4, the address increases from bottom to top. The allocation of the static TLS block memory occurs on the thread stack. The size of the TCB is determined by the tcbhead_t data structure, which is typically 16 bytes. The static TLS block is located immediately after the tcbhead_t data structure.

Static TLS Space:​

Static TLS space refers to a contiguous physical memory region allocated on the thread stack. It encompasses both the static TLS space and TLS management space. The dynamic library's tls_offset is calculated relative to this space.

• Thread Pointer:

The thread pointer needs to be stored in a segment register. In Variant 1, the TCB's space allocation needs to be consistent with the compiler to meet the requirements of LE (little-endian) access. For example, in Bionic, the TCB occupies 64 bytes, requiring the compiler to generate instructions for accessing main program TLS variables with an additional offset of 64. In Variant 2, the offset is conventionally set to 0, eliminating this requirement.

• TCB Management Space:

The TCB management space records addresses for the thread pointer, dynamic thread vector (dtv), and management of thread-local data. In Bionic's aarch64 implementation, the space is defined as nine arrays of eight bytes each, totaling 72 bytes. It is larger than the offset between the thread pointer and the first TLS variable by 8 bytes, causing the thread pointer to point to the address of the second element. Additionally, the TCB management space may have alignment requirements, which can result in a different starting address compared to the static TLS space. This may vary depending on the specific implementation of Bionic.

• Dynamic Thread Vector (dtv):

The dynamic thread vector records the starting positions of the TLS program segments for each dynamic library. Its address is stored in the space specified by the thread pointer.

Initialization Process of TLS Data Structure in Bionic​

The initialization of the TLS data structure in Bionic by the dynamic linker is divided into two parts: static loading of libraries during the main program's loading process and dynamic loading of libraries using dlopen during the main program's execution.

TLS Initialization of TLS in Static Loading Libraries​

During the loading process of the main program, the dynamic linker initializes the TLS data structure using two global variables: StaticTlsLayout and TlsModules.

• StaticTlsLayout is of type StaticTlsLayout and is used to calculate the offset of each dependency library's TLS program segment in the static TLS block.

• TlsModules is of type TlsModules and is used to calculate all the dynamic libraries that have TLS program segments.

During the main program loading process, all TLS program segments are stored in the static TLS block memory. The specific steps are as follows：

1. Initialization of bionic_tcb and tpidr_el0: The function libc_init_main_thread_early is called to initialize bionic_tcb and tpidr_el0. This involves setting up the initial values for bionic_tcb and setting the tpidr_el0 register.

2. Initialization of TLS_SLOT_DTV in bionic_tcb: The function init_tcb_dtv is called to initialize the TLS_SLOT_DTV field in the bionic_tcb structure. The update flag for this field is set to 0.

3. Calling linker_setup_exe_static_tls: This function is called assuming that the main program contains TLS program segments. The following actions are taken:
   a. Reserve space and determine the position of the bionic_tcb object, TLS space, and their locations in a variable of type StaticTlsLayout.
   b. Call register_tls_module to obtain the module ID and add it to a variable of type TlsModules.
   c. Reserve space and determine the position of the bionic_tls object in the variable.

4. Loading of TLS program segments in the main program dependencies: If there are TLS program segments in the main program's dependencies, the function soinfo::register_soinfo_tls is called. This reserves space for TLS segments and their positions in the StaticTlsLayout variable, and adds them to the TlsModules variable.

5. Calling linker_finalize_static_tls: This function calculates the total size of the reserved space in the StaticTlsLayout variable, which represents the size of the static TLS block.

6. Calling __allocate_thread_mapping: This function allocates memory space on the main program's stack for the static TLS block. The static TLS block includes bionic_tcb, bionic_tls, and TLS program segments.

7. Calling __init_static_tls: This function copies the contents of the TLS segments from the dynamic libraries specified in the TlsModules variable into the reserved space of the static TLS block.

8. Calling bionic_tcb::copy_from_bootstrap: This function synchronizes the contents of bionic_tcb with the initial values set during the first step of initialization.

9. Calling __init_tcb: This function updates bionic_tcb with additional information.

10. Calling __init_bionic_tls_ptrs: This function updates the addresses of bionic_tls.

11. Relocation Initialization of TLS Variable's GOT Table Entries

• GD (Global Dynamic): The relocation types for GD include R_AARCH64_TLS_DTPMOD64 and R_AARCH64_TLS_DTPREL64. These types initialize the adjacent GOT table entries with the module ID and the offset of the variable within its TLS segment, respectively.

• LD (Local Dynamic): The implementation of LD is the same as GD for AArch64. However, for x86_64, the relocation type is R_X86_64_DTPMOD64, and the adjacent GOT table entries are initialized with the module ID and an offset of 0.

• IE (Initial Executable): The relocation type for IE is R_AARCH64_TLS_TPREL64. It initializes the value of the corresponding GOT table entry with the offset on the static TLS block.

• LE (Local Executable): There are no relocation entries or GOT table entries to initialize, so no initialization is needed.

• TLSDESC: The relocation type for TLSDESC is R_AARCH64_TLSDESC. It initializes the adjacent GOT table entries with the address of the tlsdesc_resolver_static function and the offset on the static TLS block.

From the initialization process described above, it can be seen that the dtv (Dynamic Thread Vector) array is initially empty. The dtv array is only created when accessing TLS variables. This delayed allocation has the following benefits:

• For IE and LE access to TLS variables, the address of the TLS variable can be obtained directly using the segment register and the offset, without the need to look up the dtv array.

• For GD and LD access to TLS variables, the __tls_get_addr function is called to obtain the TLS variable's address. When checking that the dtv array is empty, the dtv array is reallocated based on the number of TLS dynamic libraries.

• For TLSDESC access to TLS variables, the tlsdesc_resolver_static function directly returns the offset of the TLS variable on the static TLS block, without the need to allocate the dtv array.

From the examples mentioned above, it is evident that the TLSDESC access mode provides significant optimization for accessing TLS variables on the static TLS block compared to GD. The TLSDESC mode directly returns the offset of the static TLS block in the GOT table entry, while GD requires accessing the dtv array to calculate the address.

动态加载库初始化TLS​

bionic通过dlopen动态加载库，初始TLS的步骤如下：

1. 在do_dlopen->find_library->soinfo::register_soinfo_tls→register_tls_module流程中，获得module id，并加入至TlsModules类型变量中

2. 在soinfo::relocate->plain_relocate->plain_relocate_impl->process_relocation→process_relocation_impl流程中重定位初始化TLS变量的GOT表项

   • GD：重定位类型包括R_AARCH64_TLS_DTPMOD64和R_AARCH64_TLS_DTPREL64，分别将其相邻的GOT表项初始化为module id和变量在其TLS段中的偏移

   • LD：aarch64对LD的实现与GD相同

   • IE/LE：不支持

   • TLSDESC：重定位类型为R_AARCH64_TLSDESC，将其相邻的GOT表项初始化为tlsdesc_resolver_dynamic函数地址和TlsDynamicResolverArg类型变量地址

     • a. 初始化TlsDynamicResolverArg中TlsIndex的module id以及offset，offset的值为TLS变量在其TLS程序段的偏移量。另外初始化更新标志为库的更新标志，该标志表示动态库是否有更新;

     • b. 为了存储TlsDynamicResolverArg类型变量，将变量保存在soinfo::tlsdescargs数组中，为处理数组重新分配内存，Relocator::deferred_tlsdesc_relocs缓冲重定位信息，当该库的所有重定位操作完成后，再更新TLS变量的GOT表项.

线程创建过程中初始化TLS​

调用pthread_create创建线程，需要对主程序上的所有TLS数据结构进行拷贝。（pthread_create->__allocate_thread）

1. 调用__allocate_thread_mapping分配线程栈空间，包含了静态TLS块空间。（Allocate in order: stack guard, stack, static TLS, guard page）

2. 调用__init_static_tls，将TlsModules类型变量中动态库的TLS段内容拷贝至静态TLS块空间中。

3. 调用__init_tcb更新bionic_tcb

4. 调用__init_tcb_dtv初始bionic_tcb中的TLS_SLOT_DTV，其更新标志值为0。

5. 调用__init_bionic_tls_ptrs更新bionic_tls地址

6. 调用clone，将静态TLS块地址传递给clone，由内核设置段寄存器tpidr_el0值

__tls_get_addr函数实现​

GD/LD访问方式使用tls_get_addr函数获取TLS变量绝对地址。tls_get_addr函数涉及对dtv数据更新，其更新的条件由3个更新标志（generation）控制

• 全局generation，保存在__libc_tls_generation_copy，为TlsModules::generation一个副本，每次新增拥有TLS程序段的动态库时，递增该值，表示有动态库新增。不需要处理动态库删除问题

• dtv数组中的generation，保存在数组中的第一个元素，初始化为0，每次更新dtv数组时，更新generation只为当时的全局generation值。与全局generation不相等，说明有新的动态库加载，需要更新dtv数组内容

• 动态库的generation，保存在TlsModule::first_generation，该值初始化为加载该库时全局generation的值。该值用于判断dtv指向的动态库是否有变化，即是否为旧的动态库

struct TlsIndex {
  size_t module_id;
  size_t offset;
};

// ti的值保存在动态库的GOT表项中，在重定位时初始化，占两个表项内容

extern "C" void* __tls_get_addr(const TlsIndex* ti){

//  获取dtv数组

TlsDtv* dtv = __get_tcb_dtv(__get_bionic_tcb());

// 获取全局动态库更新标志

size_t generation = atomic_load(&__libc_tls_generation_copy);
  if (__predict_true(generation == dtv->generation)) {

void* mod_ptr = dtv->modules[__tls_module_id_to_idx(ti->module_id)];
    if (__predict_true(mod_ptr != nullptr)) {

      // 无动态库更新，且内存已分配，则进入快速路径，返回TLS变量偏移地址
      return static_cast<char*>(mod_ptr) + ti->offset + TLS_DTV_OFFSET;
    }

    // 延时分配动态库的动态TLS块内存，只有访问该动态库的TLS变量时才分配内存，进入慢速路径
  }

  // 有动态库更新或者第一次访问，进入dtv和动态TLS块的分配和初始化

  return tls_get_addr_slow_path(ti);
}

tls_get_addr_slow_path函数包含dtv和动态TLS块的分配和初始化.

__attribute__((noinline)) static void* tls_get_addr_slow_path(const TlsIndex* ti) {
  TlsModules& modules = __libc_shared_globals()->tls_modules;
  bionic_tcb* tcb = __get_bionic_tcb();
  ScopedSignalBlocker ssb;

  // 互斥写，防止多线程同时修改__libc_shared_globals()->tls_modules全局变量
  ScopedWriteLock locker(&modules.rwlock);

  // 更新dtv数组或者重新分配数组内存

  update_tls_dtv(tcb);

  TlsDtv* dtv = __get_tcb_dtv(tcb);
  const size_t module_idx = __tls_module_id_to_idx(ti->module_id);
  void* mod_ptr = dtv->modules[module_idx];
  if (mod_ptr == nullptr) {

    // 不存在，则分配内存，将动态库TLS程序段内容拷贝至新内存，并初始化该模块指针
    const TlsSegment& segment = modules.module_table[module_idx].segment;
    mod_ptr = __libc_shared_globals()->tls_allocator.memalign(segment.alignment, segment.size);
    if (segment.init_size > 0) {
      memcpy(mod_ptr, segment.init_ptr, segment.init_size);
    }
    dtv->modules[module_idx] = mod_ptr;

    // Reports the allocation to the listener, if any.
    if (modules.on_creation_cb != nullptr) {
      modules.on_creation_cb(mod_ptr, static_cast<void*>(static_cast<char*>(mod_ptr) + segment.size));
    }
  }

  return static_cast<char*>(mod_ptr) + ti->offset + TLS_DTV_OFFSET;
}

update_tls_dtv动态分配dtv数组空间，代码太多不再列出，其实现步骤如下：

1. 更新dtv数组的条件：dtv数组的更新标志与全局动态库更新标志不相等，说明动态库有更新，当前指向的为旧的动态库

2. 重新分配dtv数组，条件条件为拥有TLS程序段的动态库总数量大于dtv数组大小

   • a. 根据动态库数量重新分配dtv数组空间

   • b. 将dtv数组中的内容备份至新的dtv数组空间

   • c. 调用__set_tcb_dtv更新为新的dtv数组

   • d. 为实现无锁操作，不释放旧的dtv数组空间，而是将其插入一个垃圾回收队列中，待程序结束时回收

3. 重新更新静态TLS块对应动态库的dtv数组元素

4. 重新更新动态TLS块对应动态库的dtv数组元素，条件：动态库的更新标志大于dtv数组更新标志（表明dtv数组指向的为旧的动态库）

   • a. 释放旧的动态库动态TLS块内存

   • b. 将dtv数组元素清零

5. 更新dtv数组的更新标志为全局动态库更新标志

TLSDESC访问方式实现​

TLSDESC访问方式有两种方式获取TLS变量相对静态TLS块的偏移地址：一种对于静态TLS块上的TLS变量由tlsdesc_resolver_static获取，另一种对于动态TLS块的TLS变量由tlsdesc_resolver_dynamic获取。这两种方式都采用汇编实现，不遵循C/C++函数调用的寄存器传参规范。其使用规范中的返回值寄存器传参，如aarch64的x0寄存器，x86_64的rax寄存器。

/* Type used to represent a TLS descriptor in the GOT.  */

struct TlsDescriptor {
  TlsDescResolverFunc* func;
  size_t arg;
};

// tlsdesc_resolver_static函数，其TlsDescriptor::arg值为TLS变量的相对偏移

// tlsdesc_resolver_dynamic函数，其TlsDescriptor::arg值为下列类型变量地址

struct TlsDynamicResolverArg {
  size_t generation;
  TlsIndex index;
};

struct TlsIndex {
  size_t module_id;
  size_t offset;
};

tlsdesc_resolver_static的实现相当简单，返回TlsDescriptor::arg值即可。

tlsdesc_resolver_dynamic对更新标志的判断有所优化，共有4个generation更新标志：

1. 全局generation，保存在__libc_tls_generation_copy，为TlsModules::generation一个副本，每次新增拥有TLS程序段的动态库时，递增该值，表示有动态库新增。不需要处理动态库删除问题

2. dtv数组中的generation，保存在数组中的第一个元素dtv[0]，初始化为0，每次更新dtv数组时，更新generation只为当时的全局generation值。与全局generation不相等，说明有新的动态库加载，需要更新dtv数组内容

3. 动态库的generation，保存在TlsModule::first_generation，该值初始化为加载该库时全局generation的值。该值用于判断dtv指向的动态库是否有变化，即是否为旧的动态库

4. TLS变量GOT表项中指向的TlsDynamicResolverArg::generation，该值初始化为TlsModule::first_generation值，该值只要不大于dtv数组中的generation不需要重新分配dtv数组，否则表示该动态库的TLS程序段在dtv数组中未初始化。

tlsdesc_resolver_dynamic实现步骤：

1. 快速路径，条件：TlsDynamicResolverArg::generation <= dtv[0] && dtv[mod_id] != NULL a. 返回 dtv[mod_id] + TlsDynamicResolverArg::TlsIndex::offset相对于静态TLS块的偏移

2. 慢速路径，调用__tls_get_addr获取TLS变量的绝对地址，返回与静态TLS块的相对偏移

gnu TLS数据结构初始化流程​

原理与bionic TLS数据结构初始化流程类似，不再详述。两者的差异包括：

• 静态加载库初始化TLS时，gnu会为静态TLS块预留144字节空间

• 动态加载库初始化TLS时，先从预留的静态TLS块获取空间，不足时采用动态TLS块，这种方式可满足调用dlopen动态加载的TLS变量IE访问方式的动态库

• TLSDESC实现函数名称不同：_dl_tlsdesc_return/_dl_tlsdesc_dynamic/_dl_tlsdesc_undefweak -> tlsdesc_resolver_static/tlsdesc_resolver_dynamic/tlsdesc_resolver_unresolved_weak

预留静态TLS空间的作用有两个，一个是支持动态加载IE访问模式的库，另一个是优化TLSDESC访问模式性能。

在linux环境下，图形加速库（OpenGL/EGL）的使用预留静态TLS空间的典型应用。一般linux应用程序会使用图形API转发库，如glvnd，图形API转发库通过dlopen动态加载OpenGL/EGL库，而OpenGL/EGL库一般都使用了IE访问模式的TLS变量，通常是一个指针变量，指向一个数据结构，从而减少静态TLS块预留空间的占用。

预留静态TLS空间注意事项：

• 分配时机：glibc在重定位时尝试分配静态TLS空间，支持的两个重定位类型，分别为R_AARCH64_TLSDESC和R_AARCH64_TLS_TPREL

• 初始化数据：对所有线程的静态TLS空间进行初始化，TLS数据结构在线程栈上，有的线程使用用户栈，有的使用系统栈，在_dl_init_static_tls函数中实现

• 并发互斥访问：预留静态TLS空间分配在dl_open_worker_begin中实现，调用函数前加了一把大锁dl_load_tls_lock；初始化静态TLS数据时，在_dl_init_static_tls函数内加了一把锁dl_stack_cache_lock