Feb 5, 2016

Ways of invoking Pig or Pig script from command line




The following examples are based on Apache Pig 0.15.0 latest at the time. Please make sure to review my later post in relation to this.

Link:
Apache-Pig-commandline-Options



a) Pig in local interactive mode: 
The following will run in the single virtual machine and the files that you are working on will be in local file systems only. This command will take you on Pig's grunt interactive shell.

On unix terminal run: 
$ cd /usr/share/cloudera/pig/ 
$ bin/pig –x local 

b) Pig in Map reduce mode: 
By default Pig runs in a map reduce mode in a Hadoop cluster. This command will take you on Pig's grunt interactive shell in a Hadoop cluster.

On unix terminal run:
$ cd /usr/share/cloudera/pig/ 
$ bin/pig –x
or  
$ bin/pig 

c) Pig in Tez mode:
It is similar to local mode, except internally Pig will invoke tez runtime engine. Tez is new framework from Hortonworks which is now a part of Apache. It is still under finalization. This command will take you on Pig's grunt interactive shell.

On unix terminal run: 
$ cd /usr/share/cloudera/pig/ 
$ bin/pig –x tez


d) Pig separate scripts in local interactive mode or Map Reduce mode:
We can list out pig scripts in a text file and invoke Pig scripts in local or Map Reduce mode.


On unix terminal run:
$ cd /usr/share/cloudera/pig/ 
$ bin/pig –x local myscript.pig
or 
$ bin/pig myscript.pig       (This will run in Map Reduce mode)



Note: 
In this post it is assumed that the pig binaries are installed in this /usr/share/cloudera/pig/ directory on a local or hadoop cluster. Thus this location will use to execute/initiate the Pig framework.  

Thanks!


References:
http://pig.apache.org

0 comments:

Post a Comment