I'm getting this exception in Java:
java.io.FileNotFoundException: (Too many open files)
I'm looking for the ways to eliminate this problem.
This error obviously indicates that JVM has allocated too many handles and underlying OS won't let it have more. Either I've got leak somewhere with improperly closed connections/streams.
This process runs for days non-stop and eventually throws the exception. It repeatedly happens after 12-14 days of up-time.
How do you fight this? Is there a way to get a list of allocated handles in JVM or track when it hits certain amount? I'd love to have them printed and see how it grows and when. I can't use a profiler because it's a production system and have difficulties to reproduce it in development. Any suggestion?
I am monitoring free heap size and raising an "alarm" when it approaches 1% of the total specified in -Xmx. I also know that if my thread count hits above 500, then something definitely goes out of hand. Now, is there a way to know that my JVM allocates too many handles from OS and doesn't give them back, e.g. sockets, opened files, etc. If I'd knew that, I'd know where to look and when.
解决方案
You didn't say which OS you are running on, but if you are running on Linux you can use the lsof command
lsof -p
That will list all the files opened by the JVM. Or if you are running on Windows you can Process Explorer which will show all the open files for all the processes.
Doing this will hopefully allow you to narrow down which bit of the code is keeping the files open.