Memory leak introduced in v10.15.3

[anthony-tuininga]

• Version: v10.15.3
• Platform: Linux
• Subsystem: napi

Our module that has been migrated to N-API is experiencing a memory leak in version 10.15.3 but not in version 10.15.2. I verified that reverting the changes made in this pull request eliminates the memory leak. Clearly that isn't the right solution but hopefully it will point in the right direction!

[addaleax]

Is there any chance you could share a reproduction?

[anthony-tuininga]

I have a reproduction that works with our node-oracledb driver -- but if you don't have an Oracle Database handy that won't help much. If you do I can share the code that causes the issue; otherwise, it will take some time to develop a test case that demonstrates the issue!

[addaleax]

Depending on the complexity, looking at the code might help, because we know we’re looking for code that uses napi_create_reference() et al?

/cc @mhdawson as the PR author

[anthony-tuininga]

The code for the driver can be found here. The code that is probably the most likely to be the source of the issue can be found here. Specifically, it acquires the value of a reference (for the constructor), creates an instance and then calls napi_wrap() to associate a structure with the newly created object.

[anthony-tuininga]

I attach a zip file containing a simple module that simply creates an instance and wraps it:
issue_26667.zip. The pattern suggests that there is a race condition involved and that only occasionally it leaks -- but if you do it often enough it becomes obvious. Hopefully this is sufficient to track down the issue. After building the module, you can run the script as follows:

NODE_PATH=build/Release node --expose-gc demo.js

This will create a file called stats.txt which contains a dump of the number of iterations processed as well as the RSS and heap size.

[addaleax]

The pattern suggests that there is a race condition involved and that only occasionally it leaks -- but if you do it often enough it becomes obvious. Hopefully this is sufficient to track down the issue.

@anthony-tuininga Are you saying that only occasionally there is a leak visible when running the program, or that you need many iterations within a single process in order to see the leak?

Because I’m having a hard time seeing an actual leak here – at the very least, the finalizer callback always seems to be called for exactly 25000 objects each iteration (so, the expected amount), at least locally…

[anthony-tuininga]

What I meant was that the amount of memory lost is not sufficient to indicate that memory is being lost each iteration. Instead memory must be lost on some iterations only. Unfortunately, I ran both 10.15.2 and 10.15.3 with my simple module and discovered that both leak memory, albeit very slowly. I ran each of the loops 4 billion times and heap memory increased about 4 MB in both cases and RSS about 10 MB. So my example doesn't demonstrate the problem. :-(

But there is a marked difference between the two versions when using the node-oracledb driver test case and the difference goes away if I revert the pull request I mentioned earlier. I'll see if I can find a better test case next week.

[anthony-tuininga]

Ok. I created a cut-down version of the node-oracledb module which has all of the Oracle bits ripped out. It demonstrates the problem. With Node.js 10.15.2 it stabilises around 350,000 iterations but with Node.js 10.15.3 it increases at around 100 bytes/iteration.

issue_26667.zip

Once unzipped and built, run with the following command:

NODE_PATH=. node --expose-gc demo.js

[anthony-tuininga]

@addaleax, with the new test case are you able to replicate the issue?

[mhdawson]

This is what I see running your example on master:
``
Num Iters,RSS,Heap Total,Heap Used,Ext
25000,84611072,43147264,2840488,9146
50000,82440192,39477248,2716024,9146
75000,84344832,38952960,2722368,9146
100000,84377600,39477248,2724104,9146
125000,86728704,39477248,2724104,9146
150000,89051136,39477248,2724104,9146
175000,91439104,39477248,2724240,9146
200000,94646272,38428672,2725080,9146
225000,96698368,38952960,2725080,9146
250000,99041280,39477248,2713416,9146
275000,101924864,38428672,2725080,9146
300000,104325120,38428672,2725088,9146
325000,106901504,39477248,2725088,9146
350000,109899776,38428672,2725088,9146
375000,111890432,38428672,2725088,9146
400000,114360320,38952960,2725088,9146
425000,116314112,39477248,2725088,9146
450000,119160832,39477248,2725088,9146
475000,121499648,38952960,2713424,9146
500000,124059648,39477248,2725088,9146
525000,126201856,38952960,2725088,9146
550000,128806912,38952960,2713424,9146
575000,131485696,38952960,2725088,9146
600000,134086656,38952960,2726368,9146
625000,136589312,39477248,2714704,9146
650000,139042816,38428672,2726368,9146
675000,141357056,38952960,2726368,9146

It that similar to what you were seeing on 10.15.3?

[anthony-tuininga]

Yes. If you run with Node.js 10.15.2 you will see that the RSS remains relatively stable.

Num Iters,RSS,Heap Total,Heap Used,Ext
25000,136855552,42713088,4252656,8272
50000,140029952,43237376,4256360,8272
75000,141082624,43761664,4264688,8272
100000,142774272,44285952,4251600,8272
125000,147193856,44810240,4263944,8272
150000,147931136,45334528,4252536,8272
175000,148590592,45334528,4264080,8272
200000,148500480,45334528,4264120,8272
225000,148443136,45334528,4252744,8272
250000,148815872,45858816,4264136,8272
275000,149032960,45858816,4264184,8272
300000,149032960,45858816,4252744,8272
325000,149032960,45858816,4264152,8272
350000,149037056,45858816,4264120,8272
375000,149037056,45858816,4252744,8272
400000,149037056,45858816,4252728,8272
425000,149045248,45858816,4264168,8272
450000,149045248,45858816,4252760,8272
475000,149045248,45858816,4264184,8272
500000,149045248,45858816,4252760,8272
525000,148520960,45334528,4264152,8272
550000,148656128,45334528,4264136,8272
575000,148709376,45334528,4264136,8272
600000,148873216,45334528,4264152,8272
625000,148852736,45858816,4252776,8272
650000,149315584,45858816,4264104,8272
675000,149315584,45858816,4264152,8272

[mhdawson]

I'll run with valgrind to see if anything obvious shows up.