Announcement

Collapse
No announcement yet.

Debugging OTM Native Heap Crashes on AIX

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Debugging OTM Native Heap Crashes on AIX

    Hello all,

    I'm currently debugging an issue for a client, who is running on AIX. I hate to say it, but this has unfortunately become one of the more troublesome platforms for OTM, both in terms of performance (Java performance is poor -- even though the processors are fantastic) and stability (Java crashes A LOT under load).

    The issue we're encountering now is that the WebLogic server is randomly crashing as the JVM runs out of native memory (note -- NOT the Java Heap).

    For more information on how Java allocates memory on AIX, see the following article:I'll most more details as I get them, but right now, we're running svmon at regular intervals in order to understand the memory usage and determine what we can do about it.
    Code:
    svmon -P <weblogic_java_pid> -m -i <interval>
    To determine the <weblogic_java_pid>, just run:
    Code:
    ps -ef | grep java | grep weblogic\.Server
    The worst-case scenario is that this is a bug in how OTM and the IBM JDK work together, which causes a memory leak in the native memory area -- since it would require a code fix, which would be especially hard to track down and resolve.

    Thanks,
    Chris

  • #2
    Re: Debugging OTM Native Heap Crashes on AIX

    As a quick update - here's a website that describes the things you'll want to accomplish and log before calling IBM -- otherwise they'll just make you do it anyways.

    IBM - Mustgather: Gathering data for OutOfMemoryErrors on AIX

    The biggest help is the script to automate the collection of svmon and other data (attached below).

    AIX_memory_leak.sh.zip

    Thanks,
    Chris

    Comment

    Working...
    X