strace Those Crashes On Linux

strace is a great tool on Linux that every developer should be aware of.  It helps trace system APIs – the APIs used, parameters passed and the error value returned.   It does not require root permissions to run.  You can generate a system call trace on a command that is executed or you can trace an already running process.

Sample Usage

strace ./my_crashing_binary

strace ./my_crashing_binary 2>out.txt

strace -p 12345 2>out.txt

The strace Output

A sample output of strace is given below –

<snip>
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fe1000
mprotect(0x645000, 4096, PROT_READ) = 0
mprotect(0x51a000, 4096, PROT_READ) = 0
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7fe16c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
munmap(0xb7fe3000, 115297) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 5), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fff000
write(1, "Hello World\n", 12) = 12
brk(0) = 0x940e000
brk(0x942f000) = 0x942f000
open("/myfile", O_RDONLY) = -1 ENOENT (No such file or directory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV (core dumped) +++

In the above example each line of the strace output has the system call, the parameters passed to it and the return value.

Take note that just before the above program crashed, it tried to access /myfile. With experience it is not hard to guess that the file is being accessed even though it was not found.  Either this file is absolutely necessary for the program to run or the required error checks are missing that could have averted the crash.

When Is strace Useful?

strace is very useful while debugging crashes.  Some scenarios where strace can be helpful are –

1. Debugging why an installation crashes on a machine.
2. Detecting file contention issues such as a file name clash while creating temporary files.
3. Debugging random crashes that are most probably due to the program running out of memory or due to it requesting an arbitrarily large chunk of memory.
4. Finding out how the program interacts with the file system.
5. Debugging crashes reproducibly only on one machine.
6. Debugging crashes in unfamilar code or in cases when sources are unavailable.

More Information

This article is meant as a pointer to the strace utility but it is not a substitute for the man page.  Go to the man page to explore all options strace provides.

man strace

strace is not a substitute for a debugger but a lightweight tool which can be used to generate logs, diagnose common problems and study the usage of system calls in a program.  Like any other program in a programmer’s toolset, it can save a lot of debugging time if used at the right time.