Numerical Development on OSX (in the Command Line)

I’ve been working on C implementations of my research projects, which can of course be a perilous project. I’ve found some tools that make it hugely, hugely better.

Homebrew

You can’t do a list like this without mentioning homebrew. You want homebrew instead of MacPorts or Fink or bailing twine and chewing gum or whatever else you were thinking about using. Just do it: You can find the homepage at brew.sh or just install with:

$ ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)"

libgmalloc

It’s really easy to accidentally use freed memory if you aren’t careful with your pointers. These bugs are pretty hard to find, because they induce bugs far away from the actual mistake. One thing that makes it much, much easier is libgmalloc, a library included in OSX. On Linux, I believe that Electric Fence does a similar thing. The point of these libraries is to fail as soon as you dereference a freed pointer or read/write past the end of the buffer, instead of much later, when your program is exhibiting some undefined behavior (because you read/wrote memory important for something other than you thought.)

It is very straightforward to use. You just define an environment variable to load that library instead, and then run your code:

$ DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib ./my_buggy_code

The upside is: If you have a memory bug, the program will crash. The downsides: It will run slower. It also might miss slight overruns.

You can try to catch more of the ‘slight overruns’ with the MALLOC_STRICT_SIZE option.

You can also use it inside of gdb/lldb:

(gdb) set env DYLD_INSERT_LIBRARIES /usr/lib/libgmalloc.dylib
(lldb) settings set target.env-vars DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib

which brings me to my next item:

lldb

I am more of a ‘add print statements and inspect run logs post-mortem’ kind of programmer (and not very proud of that…), but sometimes there is a genuine call to run a debugger and step through execution. For those times, I’ve always just used gdb. But it turns out that gdb doesn’t work as well on OSX–I found I was missing symbol names for a bunch of stuff, which is really annoying. So, lately, I’ve been using LLVM’s lldb. There is extremely good documentation, tutorials, examples, gotchas, and troubleshooting available at LLDB Homepage.

It’s an extremely good idea to run lldb with libgmalloc, because then you can actually home in on what freed variables are being accessed. For example, let’s debug the code in the libgmalloc man page. The following commands compile the code, start the debugger, define the environment variable, and run:

$ cc -g3 -o gmalloctest gmalloctest.c
$ lldb ./gmalloctest
(lldb) settings set target.env-vars DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib
(lldb) run
Process 28744 stopped
* thread #1: tid = 0x1c03, 0x0000000100000ec3 gmalloctest`main + 51 at gmalloc\
   test.c:10, stop reason = EXC_BAD_ACCESS (code=1, address=0x1003b3000)
    frame #0: 0x0000000100000ec3 gmalloctest`main + 51 at gmalloctest.c:10
   7   	    unsigned i;
   8
   9	  for (i = 0; i < 200; i++) {
-> 10  	      buffer[i] = i;
   11  	        }
   12
   13	  for (i = 0; i < 200; i++) {

(I left out some GaurdMalloc and lldb startup messages.)

gperftools

Google Performance Tools is a complete package for doing performance diagnostics. In particular, they have a CPU profiler and a drop-in replacement malloc library. I haven’t used the malloc library (and in fact it’s broken on my install,) so I’ll just discuss the CPU profiler.

Installation is easy, if you followed the first step. Just run

$ brew install google-perftools

Then you need to specify that the library should run, in

Compile your code with the -lprofiler flag (preferred option!) and run $ CPUPROFILE=/tmp/myprofile ./mybuggycode

Run a command like: $ CPUPROFILE=/tmp/myprofile DYLD_INSERT_LIBRARY=/usr/local/lib/libprofiler.dylib ./mybuggycode

This inserts the library and specifies a CPU profile file.

So, now the big downside: You must recompile your code to work with Google Profiler, at least for modern versions of OSX. The relevant flag is -Wl,-no_pie. This, in my opinion, is a pretty strong reason to stick with Instruments.

valgrind

Valgrind is another useful memory error finding tool. It can find a few more mistakes There are a few gotchas to running valgrind on OS X, though. For some reason, valgrind doesn’t seem to always pick up the symbols. This is simple to fix, though:

valgrind --dsymutil=yes ./my_buggy_code

There are a ton of other useful options. Instead of rewriting the manual, I’ll just list a few of my other favorites:

• leak-check=full
• show-reachable=yes
• num-callers=20
• track-fds=yes Lately, though, I’ve been trying to use Instruments. OSX includes a really great utility for doing both profiling and memory checks. It’s definitely worth looking into as well. But I’ll have to save it for another day and another post.