EMFILE error appears in iojs v1.2.0 on Windows

[miklen]

In our Windows build environment we are opening a lot of files through the use of browserify and debowerify. Our build stopped working when we upgraded to the latest NodeJS, with an EMFILE error.

I reverted back to NodeJS 0.12.7 and the build works again. I started testing the different versions and found that the error started happning on iojs v.1.2.0 (iojs 1.1.0 runs our build with no errors).

The error I'm getting is this:

{ [Error: EMFILE: too many open files, open 'c:\Projekter\GitProjects\POS\bower_components\jqueryui\.bower.json']
  errno: -4066,
  code: 'EMFILE',
  syscall: 'open',
  path: 'c:\\Projekter\\GitProjects\\POS\\bower_components\\jquery-ui\\.bower.js
on',
  details: 'EMFILE: too many open files, open \'c:\\Projekter\\GitProjects\\POS\\bower_components\\jquery-ui\\.bower.json\'' }

In the versions of iojs and nodejs which are throwing the errors, I found that limiting the number of require calls in our app made it work again. Leaving out parts of the app is not really an option, so I hope someone can shed some light on this for me here.

[JungMinu]

would you describe in detail?

[miklen]

Sorry, I misclicked enter. I'm editing the OP right now.

[silverwind]

v1.1.0...v1.2.0, maybe e653080.

@miklen can you try to git bisect this?

[miklen]

@silverwind I've never tried it before, but I'll see if I can figure it out. I'll get back when I know which commit it is... or to declare failure :)

[cjihrig]

@miklen are you seeing this error on a more recent version of Node (4.2.x and up)?

[miklen]

@cjihrig yes. I've tried it with both 4.2.2 LTS and 5.1.0 stable, and both versions produce the error. The most recent version I've found to not produce the error is iojs 1.1.0. I tried all minor versions down from the last iojs version and down and all above 1.1.0 produces the error.

[miklen]

It looks like I'm not able to build node from the source code. When I try I get this:

c:\code\node>vcbuild nosign
Looking for Visual Studio 2015
Found Visual Studio 2015
ctrpp not found in WinSDK path--using pre-gen files from tools/msvs/genfiles.
creating  icu_config.gypi
{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'asan': 0,
                 'host_arch': 'x64',
                 'icu_small': 'false',
                 'node_byteorder': 'little',
                 'node_install_npm': 'true',
                 'node_prefix': '/usr/local',
                 'node_release_urlbase': '',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'true',
                 'node_use_lttng': 'false',
                 'node_use_openssl': 'true',
                 'node_use_perfctr': 'true',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'python': 'c:\\Program Files\\Python27\\python.exe',
                 'target_arch': 'ia32',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 0,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_random_seed': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 1}}
creating  config.gypi
creating  config.mk
Project files generated.
  ares__read_line.c
  ares__timeval.c
  ares_free_string.c
  ares_getenv.c
  ares_init.c
  ares_library_init.c
  ares_llist.c
  ares_mkquery.c
  ares_nowarn.c
  ares_options.c
  ares_parse_a_reply.c
  ares_parse_aaaa_reply.c
  ares_parse_mx_reply.c
  ares_parse_naptr_reply.c
  node_js2c
  'c:\Program' is not recognized as an internal or external command,
  operable program or batch file.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targ
ets(171,5): error MSB6006: "cmd.exe" exited with code 1. [c:\code\node\node_js2
c.vcxproj]
  ares_parse_ns_reply.c
  Assembling asm\x86-win32-masm\aes\aes-586.asm to Release\obj\openssl\aes-586.
  obj.
  ares_parse_ptr_reply.c
  ares_parse_soa_reply.c
  fs-poll.c
  inet.c
  threadpool.c
  gtest-death-test.cc
  ares_parse_srv_reply.c
  gtest-filepath.cc
  uv-common.c
  gtest-port.cc
  version.c
  ares_parse_txt_reply.c
  gtest-printers.cc
  async.c
  http_parser.c
[... continues with a lot of building...]

When it was done it did not work however. When I try to verify the build I get this:

c:\code\node> node -e "console.log('Hello from node.js ' + process.version)"
'node' is not recognized as an internal or external command,
operable program or batch file.

Any ideas how to get it to build so I can test the commits?

[silverwind]

According to http://stackoverflow.com/questions/17300638/building-node-js-v0-10-12-on-windows, reinstalling Python in a path without spaces might help, e.g. C:\python27.

[miklen]

Thanks - I got it working. And to my surprise, it looks like the issue has been resolved in the current HEAD commit on the master branch. I installed 5.1.0 again to recheck that I get the error in that version, and I do.

I will try and investigate which commit since 5.1.0 and the HEAD on master fixed the issue.

[silverwind]

It might be interesting to inspect open file handles when you hit EMFILE. Not sure what the best tooling for that is on Windows, but you could give Handle a try, or inspect the process in Process Explorer.

[miklen]

I tried bisecting the commits between HEAD and 5.1.0 and found that they all works. I then tried checking out the v5.1.0 commit (which fails for me when I install it from nodejs.org), and that commit works as well. So apparently I am unable to reproduce the error when I compile the code myself.

The error is not isolated to my machine though. Everyone in my team has the issue when using the installer versions from the website.

I'll see if I can inspect the file handles when I experience the error.