This version of this document is no longer maintained. For the latest documentation, see http://www.qnx.com/developers/docs.

Finding Memory Errors

You can select a topic from this diagram:

Use the QNX Memory Analysis perspective to solve memory problems.

In this chapter...

• Introduction
  • Memory management in QNX Neutrino
  • What the QNX Memory Analysis perspective can reveal
• Analyzing your program
• Tracing memory events

Introduction

Have you ever had a customer say, "The program was working fine for days, then it just crashed"? If so, chances are good that your program had a memory error -- somewhere.

Debugging memory errors can be frustrating; by the time a problem appears, often by crashing your program, the corruption may already be widespread, making the source of the problem difficult to trace.

The QNX Memory Analysis perspective shows you how your program uses memory and can help ensure that your program won't cause problems. The perspective helps you quickly pinpoint memory errors in your development and testing environments before your customers get your product.

The QNX Memory Analysis perspective may produce incorrect results when more than one IDE is communicating with the same target system. To use this perspective, make sure only one IDE is connected to the target system.

Memory management in QNX Neutrino

By design, the architecture of the OS helps ensure that faults, including memory errors, are confined to the program that caused them. Programs are less likely to cause a cascade of faults because processes are isolated from each other and from the microkernel. Even device drivers behave like regular debuggable processes:

This robust architecture ensures that crashing one program has little or no effect on other programs throughout the system. When a program faults, you can be sure that the error is restricted to that process's operation.

Neutrino's full memory protection means that almost all the memory addresses your program encounters are virtual addresses. The process manager maps your program's virtual memory addresses to the actual physical memory; memory that is contiguous in your program may be transparently split up in your system's physical memory:

The process manager allocates memory in small pages (typically 4K each). To determine the size for your system, use the sysconf(_SC_PAGESIZE) function.

As you'll see when you use the memory-analysis tools, the IDE categorizes your program's virtual address space as follows:

• program
• stack
• shared library
• objects
• heap

Process memory layout on an x86.

The Memory Information and Malloc Information views of the QNX System Information perspective provide detailed, live views of a process' memory. See the Getting System Information chapter for more information.

Program memory

Program memory holds the executable contents of your program. The code section contains the read-only execution instructions (i.e. your actual compiled code); the data section contains all the values of the global and static variables used during your program's lifetime:

Stack memory

Stack memory holds the local variables and parameters your program's functions use. Each process in Neutrino contains at least the main thread; each of the process's threads has an associated stack. When the program creates a new thread, the program can either allocate the stack and pass it into the thread-creation call, or let the system allocate a default stack size and address:

When your program runs, the process manager reserves the full stack in virtual memory, but not in physical memory. Instead, the process manager requests additional blocks of physical memory only when your program actually needs more stack memory. As one function calls another, the state of the calling function is pushed onto the stack. When the function returns, the local variables and parameters are popped off the stack.

The used portion of the stack holds your thread's state information and takes up physical memory. The unused portion of the stack is initially allocated in virtual address space, but not physical memory:

At the end of each virtual stack is a guard page that the microkernel uses to detect stack overflows. If your program writes to an address within the guard page, the microkernel detects the error and sends the process a SIGSEGV signal.

As with other types of memory, the stack memory appears to be contiguous in virtual process memory, but not necessarily so in physical memory.

Shared-library memory

Shared-library memory stores the libraries you require for your process. Like program memory, library memory consists of both code and data sections. In the case of shared libraries, all the processes map to the same physical location for the code section and to unique locations for the data section:

Object memory

Object memory represents the areas that map into a program's virtual memory space, but this memory may be associated with a physical device. For example, the graphics driver may map the video card's memory to an area of the program's address space:

Heap memory

Heap memory represents the dynamic memory used by programs at runtime. Typically, processes allocate this memory using the malloc(), realloc(), and free() functions. These calls ultimately rely on the mmap() function to reserve memory that the malloc library distributes.

The process manager usually allocates memory in 4K blocks, but allocations are typically much smaller. Since it would be wasteful to use 4K of physical memory when your program wants only 17 bytes, the malloc library manages the heap. The library dispenses the paged memory in smaller chunks and keeps track of the allocated and unused portions of the page:

Each allocation uses a small amount of fixed overhead to store internal data structures. Since there's a fixed overhead with respect to block size, the ratio of allocator overhead to data payload will be larger for smaller allocation requests.

When your program uses the malloc() function to request a block of memory, the malloc library returns the address of an appropriately sized block. To maintain constant-time allocations, the malloc library may break some memory into fixed blocks. For example, the library may return a 20-byte block to fulfill a request for 17 bytes, a 1088-byte block for a 1088-byte request, and so on.

When the malloc library receives an allocation request that it can't meet with its existing heap, the library requests additional physical memory from the process manager. As your program frees memory, the library merges adjacent free blocks to form larger free blocks wherever possible. If an entire memory page becomes free as a result, the library returns that page to the system. The heap thus grows and shrinks in 4K increments:

What the Memory Analysis perspective can reveal

The main system allocator has been instrumented to keep track of statistics associated with allocating and freeing memory. This lets the memory statistics module nonintrusively inspect any process's memory usage.

When you launch your program with the Memory Analysis tool, your program uses the debug version of the malloc library (libmalloc.so). Besides the normal statistics, this library also tracks the history of every allocation and deallocation, and provides cover functions for the string and memory functions (e.g. strcmp(),memcpy(),memmove()). Each cover function validates the corresponding function's arguments before using them. For example, if you allocate 16 bytes, then forget the terminating null character and attempt to copy a 16-byte string into the block using the strcpy() function, the library detects the error.

The debug version of the malloc library uses more memory than the nondebug version. When tracing all calls to malloc() and free(), the library requires additional CPU overhead to process and store the memory-trace events.

The QNX Memory Analysis perspective can help you pinpoint and solve various kinds of problems, including:

• memory leaks
• memory errors

Memory leaks

Memory leaks can occur if your program allocates memory and then forgets to free it later. Over time, your program consumes more memory than it actually needs.

In its mildest form, a memory leak means that your program uses more memory than it should. QNX Neutrino keeps track of the exact memory your program uses, so once your program terminates, the system recovers all the memory, including the lost memory.

If your program has a severe leak, or leaks slowly but never terminates, it could consume all memory, perhaps even causing certain system services to fail.

These tools in the QNX Memory Analysis perspective can help you find and fix memory leaks:

• Memory Trace view -- shows you all the instances where you program allocates, reallocates, and frees memory. The view lets you hide allocations that have a matching call to free(); the remaining allocations are either still in use or forgotten.
• Memory Problems view -- shows you all memory errors, including leaks (unreachable blocks).

Memory errors

Memory errors can occur if your program tries to free the same memory twice or uses a stale or invalid pointer. These "silent" errors can cause surprising, random application crashes. The source of the error can be extremely difficult to find, because the incorrect operation could have happened in a different section of code long before an innocent operation triggered a crash.

In the event of a such an error, the IDE can stop your program's execution and let you see all the allocations that led up to the error. The Memory Problems view displays memory errors and the exact line of source code that generated each error. The Memory Trace view lets you find the prior call that accessed the same memory address, even if your program made the call days earlier.

To learn more about the common causes of memory problems, see Heap Analysis: Making Memory Errors a Thing of the Past in the QNX Neutrino Programmer's Guide.

Analyzing your program

To extract the most information from your program, you should launch it with the Memory Analysis tool enabled:

1. Create a Run or Debug type of QNX Application launch configuration as you normally would, but don't click Run or Debug.
2. In the Create, manage, and run configurations dialog, click the Tools tab.
3. Click Add/Delete Tool.
4. In the Tools Selection dialog, check the Memory Analysis tool.
5. Click OK.
6. Click the Memory Analysis tab.
7. Configure the Memory Analysis settings for your program:

   

   Memory Errors

   This group of configuration options controls the Memory Analysis module's behavior when memory errors are detected.

   Enable error detection

   Check this to detect memory allocation, deallocation, and access errors.

   Verify parameters in string and memory functions

   When enabled, check the parameters in calls to str*() and mem*() functions for sanity.

   Perform full heap integrity check on every allocation/deallocation

   When enabled, check the heap's memory chains for consistency before every allocation or deallocation. Note that this checking comes with a performance penalty.

   Enable bounds checking (where possible)

   When enabled, check for buffer overruns and underruns. Note that this is only possible for dynamically allocated buffers.

   When an error is detected

   Memory Analysis will take the selected action when a memory error is detected. By default, it will report the error and attempt to continue, but you can also choose to launch the debugger or terminate the process.

   Limit trace-back depth to

   Specify the number of stack frames to record when logging a memory error.

   Memory Tracing

   This group of configuration options controls the Memory Analysis module's memory tracing features.

   Enable memory allocation/deallocation tracing

   When checked, trace all memory allocations and deallocations.

   Enable leak detection

   When checked, detect memory leaks.

   Perform leak check automatically every n seconds

   When checked, look for memory leaks every n seconds (default: 60).

   Perform leak check when process exits

   When checked, look for memory leaks when the process exits, before the operating system cleans up the process' resources.

   Limit back-trace depth to

   Specify the number of stack frames to record when tracing memory events.

   Advanced

   Click the Advanced button to modify the advanced configuration properties such as the path to the debugging malloc() library, the destination for logged data, etc.

   

8. If you want the IDE to automatically change to the QNX Memory Analysis perspective when you run or debug, check Switch to this tool's perspective on launch.
9. Click Apply to save your changes.
10. Click Run or Debug. The IDE starts your program and lets you analyze your program's memory.

Tracing memory events

When you launch a Memory Analysis session, a new entry is made in the Memory Analysis Sessions view:

When you select an entry in the Memory Analysis Sessions view, the Memory Trace view shows the associated memory events:

Each event includes the address of the memory block, the block size, the workbench resource and location that caused the memory event, as well as the project folder name and memory analysis session name.

Select a row in the Memory Trace view to see a backtrace of one or more function calls in the Event Backtrace view.

Double-click a row in the Memory Trace view to open that event's source file and go to the line that caused the memory event.

Select a column in the Memory Trace view to sort the data by the entries in that column.

The Memory Trace uses the following icons to indicate different types of events:

An allocated block that has become a leak (no references to this block exist, it cannot be deallocated by the application until it exits).

An allocation that has a matching deallocation.

An allocation event.

A deallocation that has a matching allocation.

A deallocation event.

Filtering memory trace events

You can filter the events displayed in the Memory Trace view by choosing Filters... from the view's menu ():

To filter memory trace events:

1. In the Filters dialog, check the Enabled box.
2. To limit the number of displayed items, check Limit visible items to: and enter the maximum number of items to display.
3. Select one or more memory events in the Show events of type: list by checking their boxes.

   You can quickly select or deselect all memory events using the Select All and Deselect All buttons below the list.

4. Use the radio buttons to limit the displayed events to this Memory Analysis session, the currently selected source file, or the current working set, if any.

   You can select a working set by clicking the Select... button.

5. To hide matching allocations and deallocations, check Hide matching allocation/deallocation pairs.
6. Click OK to close the Filters dialog and apply your filter settings.

Memory Problems view

The Memory Problems view displays memory leaks, errors (such as buffer overruns), and warnings.

Each entry in the Memory Problems view displays a description, the memory address, size, and the source code location (resource and line number) of the memory problem.

Select a column in the Memory Problems view to sort the data by the entries in that column.

Double-click an entry to open an editor with the source code line highlighted.

Filtering memory problems

You can filter the problems displayed in the Memory Problems view by choosing Filters... from the view's menu ():

To filter memory problems:

1. In the Filters dialog, check the Enabled box.
2. To limit the number of displayed items, check Limit visible items to: and enter the maximum number of items to display.
3. Select one or more memory events in the Show events of type: list by checking their boxes.

   You can quickly select or deselect all memory events using the Select All and Deselect All buttons below the list.

4. Use the radio buttons to limit the displayed events to this Memory Analysis session, the currently selected source file, or the current working set, if any.

   You can select a working set by clicking the Select... button.

5. To search the descriptions for a specific string, enter it in the Where description field and select contains from the drop-down menu.

   You can search for descriptions except ones that contain the specified string by choosing does not contain from the drop-down menu.

6. Click OK to close the Filters dialog and apply your filter settings.

Event Backtrace view

The Event Backtrace view displays a call stack trace leading up to your selected memory event or error:

To display error information in the Event Backtrace view:

Click on a memory event in the Memory Trace view.