How to Fix Hadoop for non-Unix platform (Windows OS) ?
Overview
Users of org.expasy.jpl.io.util.JPLHMapSerializer on windows may experiment the following exception as Hadoop makes system call to unix tool chmod :
Exception in thread "main" java.io.IOException: Cannot run program "chmod": CreateProcess error=2, The system cannot find the file specified : at java.lang.ProcessBuilder.start(Unknown Source) at org.apache.hadoop.util.Shell.runCommand(Shell.java:201) at org.apache.hadoop.util.Shell.run(Shell.java:183) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:376) at org.apache.hadoop.util.Shell.execCommand(Shell.java:462) at org.apache.hadoop.util.Shell.execCommand(Shell.java:445) at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:543) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:535) at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:336) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:400) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:610) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:591) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:498) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:490) at org.apache.hadoop.hbase.io.hfile.HFile$Writer.<init>(HFile.java:306) at org.expasy.jpl.io.util.JPLHMapSerializer.init(JPLHMapSerializer.java:125)
Fixing Dependence Bug
The solution is to install cygwin in your windows system or install a subset of it as only chmod and its dlls are needed. We will provide, in the following, a solution for the second alternative:
First Step: Get "chmod" Resources
Here are the archives for different Windows architectures:
- Windows 32-bit - contains chmod.exe, cygwin1.dll, cygiconv-2.dll, cygintl-8.dll and cyggcc_s-1.dll
- Windows 64-bit - not yet available
Second Step: Set your Path in Windows
Do not forget to set your PATH variable for chmod in Windows else chmod will not be found!!
- First right-click on your My Computer icon on your Desktop and click on Properties. Alternatively you could just press the Windows Key+Pause Break Key
- Then in the new window that opened, click on the Advanced tab
- Click on Environment Variables
- In System Variables, edit or create the PATH variable and enter the path name to cygwin-chmod directory