Using Strace to Trace Problems

More on open source software

Having trouble figuring out why Apache isn’t starting, or another program is crashing and burning, and the logfiles are giving no clue? Time to reach for strace.

If you’re having trouble figuring out why a program keeps crashing and burning, consider Strace, an easy-to-use diagnostic tool. More experienced users will find Strace handy for performance testing and more.

What’s strace? The strace utility is used to run a command and display its system calls, so you can see exactly what the program is doing until it exits. Experienced users can work with strace to do performance testing and so on, but even beginners can use strace as a diagnostic tool to see why a program is crashing.

Here’s what you do. First, simply start your application or service using Strace, like so:

strace commandname

You’ll then see a bunch of output to the terminal. For example, running man without an argument would looks like this:

execve("/usr/bin/man", ["man"], [/* 37 vars */]) = 0
brk(0)                                  = 0x8d94000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb774d000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3

... 
open("/usr/share/locale-langpack/en/LC_MESSAGES/man-db.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "What manual page do you want?n", 30What manual page do you want?
) = 30
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_START or TCSETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
exit_group(1)                           = ?

I trimmed out a bunch of material, but you can see the first few lines and the final lines that indicate man has encountered an error. Specifically, no file or directory is specified. Now it’s not a major puzzle why man exited with an error here because it prints out the helpful “What manual page do you want?” But if it didn’t, you’d have a clue with the error -1 ENOENT (No such file or directory).

Looking for errors? Usually, they’ll have an -1 to indicate an error. Not all errors are fatal, but they can provide useful clues.

You can use the -u username option to run as a specific user. This allows you to be logged in as root and run applications as the appropriate user for the service. Note that you usually will be running strace as root, so this option comes in handy.

What if a process is already running? If you must examine some application flakiness while a process is still running, you can use the -p PID option to examine a running process, where PID is the process ID number.

As always, it’s a good idea to familiarize yourself with a tool before you need it. Spend some time playing with strace now, and the next time you have a buggy app to contend with, you’ll know your way around it when it counts.

Joe ‘Zonker’ Brockmeier is a freelance writer and editor with more than 10 years covering IT. Formerly the openSUSE Community Manager for Novell, Brockmeier has written for Linux Magazine, Sys Admin, Linux Pro Magazine, IBM developerWorks, Linux.com, CIO.com, Linux Weekly News, ZDNet, and many other publications. You can reach Zonker at jzb@zonker.net and follow him on Twitter.

Follow ServerWatch on Twitter