In short, Stack overflow when VLD TLS slot >= 64 and allocate memory in new thread.
Windows by default has 64 TLS slots. When you need more than 64, Windows will allocate expansion slots for each thread in the invocation to TLS related functions. This TLS expansion calls RtlAllocateHeap to allocate memory. RtlAllocateHeap has been hooked by VLD to trace memory allocation, which calls getTls() in VLD along the call chain.
However, Windows only expand TLS for current thread when needed. When a newly created thread tries to allocate memory, VLD will kicks in to trace allocation. When VLD gets a slot >= 64, results in a TLS expansion. Expansion tries to allocate memory, but calls back to VLD, which will use TLS to trace memory allocation. Thus, infinite loops occurs.
How VLD gets a TLS slot >= 64? It is loaded when there are more than 64 slots already.
*To workaround this problem*, you can load VLD ASAP when Maxwell starts so it will get a TLS slot <64.
For a detailed explanation of Windows TLS internals, please refer to [Thread Local Storage, part 2: Explicit TLS](http://www.nynaeve.net/?p=181).
```
#include <windows.h>
#include <process.h>
void run(void *) {
void *ptr = malloc(100);
free(ptr);
}
int main(int argc, wchar_t *argv[]) {
for (int i = 0; i < 0x40; ++i) {
DWORD slot = TlsAlloc();
}
::LoadLibraryA("load_vld.dll");
::_beginthread(run, 0, NULL);
::Sleep(1000);
return 0;
}
```
Windows by default has 64 TLS slots. When you need more than 64, Windows will allocate expansion slots for each thread in the invocation to TLS related functions. This TLS expansion calls RtlAllocateHeap to allocate memory. RtlAllocateHeap has been hooked by VLD to trace memory allocation, which calls getTls() in VLD along the call chain.
However, Windows only expand TLS for current thread when needed. When a newly created thread tries to allocate memory, VLD will kicks in to trace allocation. When VLD gets a slot >= 64, results in a TLS expansion. Expansion tries to allocate memory, but calls back to VLD, which will use TLS to trace memory allocation. Thus, infinite loops occurs.
How VLD gets a TLS slot >= 64? It is loaded when there are more than 64 slots already.
*To workaround this problem*, you can load VLD ASAP when Maxwell starts so it will get a TLS slot <64.
For a detailed explanation of Windows TLS internals, please refer to [Thread Local Storage, part 2: Explicit TLS](http://www.nynaeve.net/?p=181).
```
#include <windows.h>
#include <process.h>
void run(void *) {
void *ptr = malloc(100);
free(ptr);
}
int main(int argc, wchar_t *argv[]) {
for (int i = 0; i < 0x40; ++i) {
DWORD slot = TlsAlloc();
}
::LoadLibraryA("load_vld.dll");
::_beginthread(run, 0, NULL);
::Sleep(1000);
return 0;
}
```