Comments on: Hadoop and Linux kernel 2.6.27 – epoll limits http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/ Mon, 13 Sep 2010 17:20:49 +0000 hourly 1 http://wordpress.org/?v=3.3 By: Twitter Trackbacks for pero on anything » Blog Archive » Hadoop and Linux kernel 2.6.27 - epoll limits [aprilmayjune.org] on Topsy.com http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12703 Twitter Trackbacks for pero on anything » Blog Archive » Hadoop and Linux kernel 2.6.27 - epoll limits [aprilmayjune.org] on Topsy.com Sat, 03 Apr 2010 01:22:54 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12703 [...] pero on anything » Blog Archive » Hadoop and Linux kernel 2.6.27 - epoll limits pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits – view page – cached Yesterday we faced a strange problem. A newly set up Hadoop cluster got unstable after a few minutes. Logs reported a lot of exceptions like: java.io.IOException: Too many open files at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) at sun.nio.ch.EPollArrayWrapper.(EPollArrayWrapper.java:68) at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:52) Filter tweets [...] [...] pero on anything » Blog Archive » Hadoop and Linux kernel 2.6.27 – epoll limits pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits – view page – cached Yesterday we faced a strange problem. A newly set up Hadoop cluster got unstable after a few minutes. Logs reported a lot of exceptions like: java.io.IOException: Too many open files at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method) at sun.nio.ch.EPollArrayWrapper.(EPollArrayWrapper.java:68) at sun.nio.ch.EPollSelectorImpl.(EPollSelectorImpl.java:52) Filter tweets [...]

]]> By: fs http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12642 fs Wed, 08 Apr 2009 02:35:37 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12642 /proc/sys/fs/epoll/max_user_instances is not in 2.6.27.21-170.2.56.fc10.x86_64 kernel. Any suggestions on how to handle that. ? Thanks!! uname -a Linux node1 2.6.27.21-170.2.56.fc10.x86_64 #1 SMP Mon Mar 23 23:08:10 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux ls -al /proc/sys/fs/epoll total 0 -rw-r--r-- 1 root root 0 2009-04-07 02:01 max_user_watches /proc/sys/fs/epoll/max_user_instances is not in 2.6.27.21-170.2.56.fc10.x86_64 kernel.

Any suggestions on how to handle that. ?

Thanks!!

uname -a
Linux node1 2.6.27.21-170.2.56.fc10.x86_64 #1 SMP Mon Mar 23 23:08:10 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
ls -al /proc/sys/fs/epoll
total 0
-rw-r–r– 1 root root 0 2009-04-07 02:01 max_user_watches

]]> By: Cloudera Hadoop & Big Data Blog » Blog Archive » Configuration Parameters: What can you just ignore? http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12641 Cloudera Hadoop & Big Data Blog » Blog Archive » Configuration Parameters: What can you just ignore? Mon, 30 Mar 2009 19:08:15 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12641 [...] http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/ for more [...] [...] http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/ for more [...]

]]> By: Derek Lewis http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12639 Derek Lewis Mon, 16 Mar 2009 20:54:22 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12639 Thank you so much for posting this. I spent a good while searching around to try to fix my "too many open files" error, despite having a plenty high ulimit. My problem was unrelated to hadoop, but this page helped immensely. Thank you so much for posting this. I spent a good while searching around to try to fix my “too many open files” error, despite having a plenty high ulimit. My problem was unrelated to hadoop, but this page helped immensely.

]]> By: tiagolp http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12635 tiagolp Fri, 06 Feb 2009 01:48:40 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12635 Thanks for the information. I just ran across the exact same problem and your post saved me hours of useless debugging. Thanks for the information. I just ran across the exact same problem and your post saved me hours of useless debugging.

]]> By: Solution Hacker - Streaming data to your grid http://aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12632 Solution Hacker - Streaming data to your grid Thu, 22 Jan 2009 23:28:50 +0000 http://pero.blogs.aprilmayjune.org/2009/01/22/hadoop-and-linux-kernel-2627-epoll-limits/#comment-12632 [...] JDK 1.6 uses epoll to implement NIO [...] [...] JDK 1.6 uses epoll to implement NIO [...]

]]>