Tuesday, February 08, 2011

PIG and Skewed JOIN Issue with Empty Files

Infra
     HADOOP 0.20.2
     PIG 0.6

ISSUE
     SKEWED JOIN on EMPTY FILES

RESULT
     JOB FAILS

CONCLUSION
     DON'T DO  SKEWED JOIN IF INPUT DATA IS EMPTY if You Still want JOB Not to come in Failed JOB List





Recursively Delete A Directory

Sweet and Powerful. Isn't It

rm -rf `find . -type d -name <dir>`