Somewhat more detailed usage instructions

[pjdevries]

Description

Hi @arnaud-lb. This is not a not a regular feature request and also not a bug report. But since there is no specific support section and I didn't now where better to ask this question, I think posting it as a feature request is be the best next thing :)

I'm usually somewhat slow of understanding and this is one of those cases. Now I finally got memprof working and got some results, I need to analyze them. Not my daily routine, so it takes some effort. In this special case, I need to figure out where exactly memory is kept in use, resulting in the fatal memory exhaustion error. With a memory_limit of 4096M, something appears to be seriously wrong. One of the Features mentioned in the README is: "Reports non-freed memory at arbitrary points in the program". However, using KCachegrind I have no clue where or how that data is visualized.

Can you help me get started? If this is the wrong place for these kind of usage questions, please point me in the right direction for next time.

[arnaud-lb]

Hi @pjdevries

I need to write something since forever but I did not took the time to do it yet.

Here are some pointers:

• Memprof reports how much memory was leaked by every function in your program
• If some memory is allocated, and freed later (including long after the function has returned), it's not counted in the profile. Only the non-freed (a.k.a. leaked) memory is counted in the profile.
• To find the cause of a memory leak, you need to find the function that leaked the most memory

KCacheGrind allows to do that by navigating in functions in different ways.

The left pane is a list of all functions seen by the profile:

[screenshot]

Here is the meaning of the columns:

• Incl.: This is the amount of memory leaked by this function, including all the functions that was called during its execution. So if you have a function f() that called g(), and g() leaked 3MB, those 3MB are included in the value of this column.
• Self: This is the amount of memory leaked by this function, excluding the function that was called during its execution. So if you have a function f() that called g(), and g() leaked 3MB, those 3MB are excluded from the value of this column.
• Function: This is the function or method name

You can sort this table by clicking on the headers.

I recommend clicking on the % button here, to display memory amounts in percentage of the total memory usage:

[screenshot]

Sorting by Self will usually show that a couple functions are responsible for a large portion of all the memory usage. If that's not the case, it means that a larger component or group of function is responsible for the leak. In this case, sorting by Incl. is more helpful.

In any case, clicking on one function allows to explore the call graph in various ways in the right pane:

[screenshot]

For example the "Call Graph" visualization at the bottom allows to find the call path leading to the function. If a function has more than one parent, you can see which path leaks the most memory. You can click around to focus on a different function.

"Caller" refers to parent functions (functions that called the selected function), while "Callee" refers to child functions (functions that was called by the selected function).

Note about allocating vs holding memory:

Usually the memory allocated by one function is held somewhere else. So, one function may be responsible for allocating a lot of memory, but an other piece of code is responsible for holding this memory, preventing it from being freed.

Memprof reports where the memory was allocated but not where it's being held.

So, after finding the biggest allocator/leaker, you need to find where this memory goes:

• Is it stored in a class property ?
• Is it returned to the called. In this case, what is the caller doing with this memory ?

Examples:

function f() {
    return str_repeat("a", 1024*1024);
}

$a = f();

// If we dump the profile now, memprof will report that f() has leaked
// at least 1MB because the string it generated is not freed at this point.
// f() has allocated at least 1MB, and this memory is being held in the `$a` variable.

$a = null;

// If we dump the profile now, memprof will report that f() has leaked
// nothing because the string it generated has been freed
// (after $a=null, the string is not referenced anywhere, so it's freed)

Let me know if this helps or if something needs clarifications. I will use this as a base when writing something later.

[pjdevries]

Hi @arnaud-lb.

Thanx again for the swift response. Yes, your elaborate explanation helps an awful lot. It is exactly the kind of information I have been trying to find, DuckDuckGoing for quite a bit. This will definitely help me find the sore spot.

Thank you very much!

[yu0307]

@arnaud-lb,
if by any chance, this message still reaches you after 2 years.....
so inc, is actually leak? not memory consumption?
does that mean when using memprof, the generated dump is actually measuring leak only, while using xdebug the dump is having inc. reflecting memory usage?
this can't be possibly correct? I put a memprof_enable(); and memprof_dump_callgrind in my index file on a bare bone symfony setup, and it's already showing close to 1mb leak under inc.? does that mean virtually all php frameworks out there will all have memory leaks and eventually will all exhaust memories and crash the server?

PS: if they are indeed all leaks, does that mean none of them will get picked up by the garbage collector? or there's a way to tell what's left even after garbage collector is ran?

Please kindly advise

[maxgalbu]

does that mean when using memprof, the generated dump is actually measuring leak only, while using xdebug the dump is having inc. reflecting memory usage?

this is really the point here... clearing this would help a lot

[mad-briller]

does that mean when using memprof, the generated dump is actually measuring leak only, while using xdebug the dump is having inc. reflecting memory usage?

this is really the point here... clearing this would help a lot

@maxgalbu this is covered in the initial comment:

If some memory is allocated, and freed later (including long after the function has returned), it's not counted in the profile. Only the non-freed (a.k.a. leaked) memory is counted in the profile.

If memory allocated by a function still exists when the dump is generated, it is reported; incl. is the amount of memory allocated by the function and all it's children that remains, and self is the memory allocated without it's children that remains

if you're taking a dump as that last thing in your application, only memory that has "leaked" will remain in the profile. It does not show overall usage. Leaked in this case could be legitimate uses which are just not cleaned up though, for example class properties on classes that have yet to be garbage collected.

In response to @yu0307's question of

does that mean virtually all php frameworks out there will all have memory leaks and eventually will all exhaust memories and crash the server?

This kind of "leak" at the end of your request lifecycle isn't necessarily negative because the php engine will deallocate all resources for the request when it finishes.