Sanitizers
Spice gives you memory safety in most of your code, but unsafe blocks, raw pointers, C interop and threads still leave
room for bugs that do not show up until much later, if at all: a write one element past the end of a buffer, a read of
freed memory or two threads updating the same variable. Sanitizers catch these bugs while the program runs. The compiler
instruments the generated code with extra checks, and a runtime library reports the first violation, together with the
stack traces that lead to it.
Spice offers four sanitizers, all based on the ones of LLVM:
| Sanitizer | Option | Finds |
|---|---|---|
| AddressSanitizer | --sanitizer=address |
Out-of-bounds accesses, use after free, use after scope, memory leaks |
| ThreadSanitizer | --sanitizer=thread |
Data races between threads |
| MemorySanitizer | --sanitizer=memory |
Reads of uninitialized memory (Linux only) |
| TypeSanitizer | --sanitizer=type |
Accesses to an object through a pointer of an incompatible type (aliasing) |
Enabling a sanitizer¶
The --sanitizer option is available on the build, run and
test subcommands. Pass it together with -g, so that the reports show the file, line and column of
every frame instead of only the function name:
spice run builds and runs the program in one step, and spice test runs the #[test] functions of a file with the
instrumentation enabled:
A few things to keep in mind:
- One sanitizer at a time. Each of them comes with its own runtime, and they cannot be combined. Build once per sanitizer, if you want to use more than one.
- Some options are implied. AddressSanitizer and MemorySanitizer turn on
--use-lifetime-markers, so that they know when a local variable goes out of scope. TypeSanitizer turns on--use-tbaa-metadata, since this metadata tells it the type of every memory access. - Instrument with few optimizations. Debug builds, the default build mode, use
-O0, which is the best fit for sanitizer runs: the reports point to the right source locations, and the optimizer cannot remove the bug before the sanitizer gets to see it.-O1is fine as well, if the instrumented program is too slow. Release builds default to-O2, so pass-O0or-O1explicitly, if you instrument one. See the MemorySanitizer section for an example of a bug that-O2hides. - The runtime comes from your C toolchain. The compiler links the program with
-fsanitize=<sanitizer>, so the sanitizer runtime is the one of the Clang or GCC installation, that Spice uses as linker invoker. Spice prefers Clang and falls back to GCC. GCC does not implement MemorySanitizer and TypeSanitizer, so those two need Clang. - Instrumented programs are slower and use more memory. Use the sanitizers to test and debug your program, not in the builds you ship.
AddressSanitizer¶
AddressSanitizer (ASan) finds the most common memory bugs: accesses outside of heap, stack or global objects, accesses to memory after it was freed, accesses to local variables after their scope ended and, on Linux, memory leaks. It slows a program down by a factor of about two, which is why it is the sanitizer to reach for first.
Out-of-bounds access¶
The loop below writes five elements into a buffer that only has room for four:
| Spice | |
|---|---|
Without instrumentation, the program exits normally: the write lands in memory that happens to belong to nobody yet.
Built with -g --sanitizer=address, it stops at the first invalid write:
Read a report from the top:
- The first line names the kind of bug, here
heap-buffer-overflow. - The first stack trace shows where the invalid access happened: line 5, column 30 of
overflow.spice, thenumbers[i]subscript. - The paragraph below describes the memory that was accessed: the address lies right behind a 16-byte region, i.e. the
four
ints, and the second stack trace shows where that region was allocated. - The
SUMMARYline repeats the most important facts in one line, which is handy to search for in long logs.
The shadow bytes at the end show ASan's view of the memory around the address. You rarely need them.
The program exits with exit code 1 after the first report.
Use after free¶
A pointer that outlives the memory it points to is a dangling pointer. Reading through it is undefined behavior, which often goes unnoticed, since the freed memory still holds the old value for a while:
| Spice | |
|---|---|
ASan reports a heap-use-after-free and adds the stack trace of the deallocation to the one of the allocation:
Use after scope¶
The same goes for a pointer to a local variable, whose scope already ended:
| Spice | |
|---|---|
| Text Only | |
|---|---|
This check relies on the lifetime markers, which --sanitizer=address turns on for you.
Memory leaks¶
On Linux, ASan includes LeakSanitizer. When the program exits, it looks for heap memory, that is no longer reachable from
anywhere, and reports where it was allocated. Variables with the heap qualifier
are freed automatically when they go out of scope, so a leak typically comes from memory that was allocated through
C interop and never handed back:
| Spice | |
|---|---|
A program with a leak exits with exit code 1, even though it ran to the end. Set ASAN_OPTIONS=detect_leaks=0 to turn
the leak check off, e.g. while you hunt a different bug.
ThreadSanitizer¶
ThreadSanitizer (TSan) finds data races: two threads access the same memory at the same time, at least one of them
writes, and nothing orders the two accesses, like a mutex does. Data races are notoriously
hard to debug, since they depend on the timing of the threads and may disappear as soon as you add a printf or attach
a debugger. TSan detects them, even if the race did not lead to a wrong result in this particular run.
Both threads below increment the same global counter without synchronization:
| Spice | |
|---|---|
Built with -g --sanitizer=thread, the report shows both conflicting accesses, the variable they access and where the
involved threads were started:
Unlike ASan, TSan does not stop the program at the first report. It reports every race once, lets the program finish
and then exits with exit code 66. The counter also shows what the race costs: 2,987 increments got lost.
To fix the race, guard the counter with a Mutex from std/os/mutex. A LockGuard acquires the mutex in its
constructor and releases it in its destructor, i.e. at the end of every loop round:
TSan stays silent now, and the program prints Counter: 200000 every time.
MemorySanitizer¶
MemorySanitizer (MSan) finds reads of uninitialized memory. It does not report the read itself, since copying uninitialized memory around is harmless. Instead, it tracks which bits of memory are initialized and reports the first time an uninitialized value decides something: a branch, a pointer that gets dereferenced or a call argument. Spice enables the eager checks of MSan, so passing an uninitialized value to a function or returning one is reported right there, instead of where the value is used later.
Adding up the values is fine, but the if on line 14 depends on them. MSan reports it and exits with exit code 1:
| Text Only | |
|---|---|
MSan comes with a few restrictions:
- Linux only. The compiler rejects
--sanitizer=memoryfor other targets. - Clang only. GCC does not ship an MSan runtime, so Clang has to be installed.
- Everything has to be instrumented. MSan knows nothing about memory written by code, that was not compiled with
MSan, so it reports false positives for values coming from such code. The C standard library is covered by the
runtime, but other C or C++ libraries you link against are not, unless you build them with
-fsanitize=memoryyourself. - Mind the optimization level. Built with
-O2, the example above passes without a report. The optimizer is allowed to assume any value for the uninitialized elements, so it removes the check, before MSan instruments the code. Clang behaves the same way for the equivalent C program. Stay at-O0, the default of debug builds, or at-O1for MSan runs.
TypeSanitizer¶
TypeSanitizer (TySan) finds violations of the strict aliasing rules: accessing an object through a pointer of an
incompatible type. The optimizer relies on these rules, as it assumes that a double* and a long* never point to the
same memory. If they do, the optimized program may read stale values or reorder writes in surprising ways. This kind of
bug typically hides in unsafe code, that reinterprets the bytes of a value with a pointer cast:
| Spice | |
|---|---|
By default, TySan only reports the violation and lets the program continue, which then exits with its regular exit code.
Set TYSAN_OPTIONS=halt_on_error=1 to stop at the first report with exit code 1 instead, e.g. in CI.
To reinterpret the bytes of a value without breaking the aliasing rules, copy them into a variable of the target type.
memcpy is allowed to access any memory, and the optimizer turns the copy into a plain register move:
| Spice | |
|---|---|
Like MSan, TySan needs Clang as linker invoker. TypeSanitizer is a recent addition to LLVM and still considered experimental there, so expect it to miss some violations.
Runtime options¶
Every sanitizer runtime reads its options from an environment variable, when the program starts. Separate multiple
options with ::
| Bash | |
|---|---|
| Sanitizer | Environment variable | Useful options |
|---|---|---|
| AddressSanitizer | ASAN_OPTIONS |
detect_leaks=0, detect_stack_use_after_return=1, log_path=<file> |
| LeakSanitizer | LSAN_OPTIONS |
suppressions=<file> to ignore known leaks, e.g. in a third-party library |
| ThreadSanitizer | TSAN_OPTIONS |
halt_on_error=1, exitcode=<code>, suppressions=<file> |
| MemorySanitizer | MSAN_OPTIONS |
exitcode=<code>, log_path=<file> |
| TypeSanitizer | TYSAN_OPTIONS |
halt_on_error=1 |
Pass help=1 in any of them to make the program print all options of that runtime. The Clang documentation of
AddressSanitizer,
LeakSanitizer,
ThreadSanitizer,
MemorySanitizer and
TypeSanitizer covers them in detail.
Limitations¶
- Freestanding builds cannot use sanitizers, since their runtimes need a hosted libc. See Freestanding builds.
- The TPDE backend does not support sanitizers, so the compiler rejects
--sanitizerin combination with--backend=tpde. Use the default LLVM backend for instrumented builds. - Uninstrumented code is invisible to the sanitizers. Bugs inside a C library you link against, that was not built with the same sanitizer, go unnoticed, and with MemorySanitizer they even cause false positives.
- Sanitizers only see the code that runs. A clean sanitizer run tells you that this particular run had no bugs of that kind, not that the program is free of them. Run the sanitizers over your tests, so that they cover as many code paths as possible.