The current verion of Floe2 has been tested for Ubuntu 14.04 Server hosts and the following instructions are based on that. Support for other platforms is coming soon.
Follow the below steps for each machine hosting any of the Floe2 components (e.g. coordinator, container). For client machine, you may skip zmq and jzmq installation.
Download and install zmq 4.0.4+ from ZMQ Download Page. Instructions are repeated here for convenience, however it is suggested to follow the latest instructions on the ZMQ site.
#pre-reqs for installing zmq.
sudo apt-get install g++ gcc libtool autoconf automake make
#download and extract zmq.
wget http://download.zeromq.org/zeromq-4.0.4.tar.gz
tar xvzf zeromq-4.0.4.tar.gz
cd zeromq-4.0.4
#configure and install using default parameters.
./configure
make
sudo make install
Download and install jzmq from JZMQ Download Page. Instructions are repeated here for convenience, however it is suggested to follow the latest instructions on the ZMQ site.
#pre-reqs for installing jzmq.
sudo apt-get install autoconf automake libtool gcc g++ make uuid-dev git openjdk-7-jdk pkg-config
#download the v3.1.0 tar from [jzmq github archive](https://github.com/zeromq/jzmq/archive/v3.1.0.tar.gz).
wget https://github.com/zeromq/jzmq/archive/v3.1.0.tar.gz
tar xvzf v3.1.0.tar.gz
cd jzmq-3.1.0
#configure, build and install jzmq.
./autogen.sh
./configure
make
sudo make install
#configure environment.
sudo bash
#For non-debian systems.
echo /usr/local/lib > /etc/ld.so.conf.d/local.conf
ldconfig
#configure LD_LIBRARY_PATH
echo export LD_LIBRARY_PATH=/usr/local/lib > /etc/profile.d/ldlibrarypath.sh
exit
#LOG OUT AND LOG BACK IN (or restart the system)
#verify zmq and jzmq
echo $LD_LIBRARY_PATH
#you shoud see "/usr/local/lib" as output.
cd <jzmq dir>/src/main/perf
java local_lat tcp://127.0.0.1:5000 1 100
#This should run without errors. Press cntrl-c to exit.
#You won't see any output on console if it runs successfully.
#If you see errors,
#please visit http://zeromq.org/bindings:java for troubleshooting.
Floe2 can be run in several modes:
Local Mode: - Single Process (used for development and debugging)
All components of Floe2 run in a single process (including zookeeper, coordinator, container). User just executes the user application (see examples/HelloWorld) and the framework launches all the components in a single process.
Psuedo Distributed Mode: - Single Machine, multi procs. (used for testing the application functionality (not performance) in an environment similar to the distributed mode before actual deployment on a cluster)
All components of Floe2 run on a single machine but in seperate processes. User is responsible for running each component manually.
Distributed Mode: - Cluster of Machines (distributed deployment of Floe2) Components span multiple machines with (preferably) separate machines dedicated for different components. One dedicated machine for Zookeeper, one for Coordinator and Resource Manager, and multiple machines for containers (one container per machine).
Download and Build Floe2
(A release version is not available yet, use the github master branch)
#install maven.
sudo apt-get install maven python
#clone github master branch.
git clone https://github.com/usc-cloud/floe2.git
#compile and install using maven.
cd floe2
mvn install
Configure Floe2 and run in local mode
Compile and Run Sample
(Use floe-examples project as a template for creating your project)
floe-examples is compiled along with floe-core during the previous step.
#to run the sample HelloWorld Application. (go to the floe2 home directory)
chmod a+x bin/floe.py
bin/floe.py jar floe-examples/target/floe-examples-0.1-SNAPSHOT.jar edu.usc.pgroup.HelloWorldApp
Download and Build Floe2
Follow same instructions as Local Mode
Configure Floe2 and run in psuedo distributed mode
Compile and Run Sample
(Use floe-examples project as a template for creating your project)
floe-examples is compiled along with floe-core during the previous step.
chmod a+x bin/floe.py
#Start zookeeper. (For convenience, a dev version of zookeeper is provided with Floe2. Do not use this for production environment.)
bin/floe.py dev-zookeeper &
#Start coordinator
bin/floe.py coordinator &
#Start multiple containers on a single machine. Each with it own range of network ports.
#Note that the floe.flake.port config property is used to specify the starting port number to be used by flakes in that container.
#You may use any number of -Dconfig.property=value options to override properties mentioned in the config file.
bin/floe.py container -Dfloe.flake.port=7000 &
bin/floe.py container -Dfloe.flake.port=8000 &
bin/floe.py container -Dfloe.flake.port=9000 &
#to run the sample HelloWorld Application. (go to the floe2 home directory)
bin/floe.py jar floe-examples/target/floe-examples-0.1-SNAPSHOT.jar edu.usc.pgroup.floe.examples.WordCountApp
#all logs are appended to the file alllogs.log, use tail to see the ouput of the above command.
tail -f alllogs.log
In this mode different components, includig Zookeeper, are run on different machine.
Download and Build Floe2 on EACH machine that hosts Floe components
Follow same instructions as Local Mode
Download and Run Zookeeper ensemble
Note: This is a quick start guide to run zookeeper in a standalone mode. For production it is suggested that you create an ensemble of atleast three dedicated zookeeper machines. For details see: Zookeer Getting Started
tar xvzf zookeeper-3.4.6.tar.gz
cd zookeeper-3.4.6
#Create a file conf/zoo.cfg and add the following contents:
tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
#start zookeeper
bin/zkServer.sh start
Note 2: Zookeeper can quickly consume your disk space. For long running jobs Zookeeper storage must be managed externally (for data dirctory and logs). See maintenance instructions on the zookeeper admin page
Configure Floe2 and run distributed mode
Edit conf/floe.properties (change it on the Coodinator node and COPY it to all container nodes whenever a change is made)
Known Issue: After changing a config value, you have to run ``mvn install'' again. This will be fixed in the coming version.
Compile and Run Sample
(Use floe-examples project as a template for creating your project)
floe-examples is compiled along with floe-core during the previous step.
chmod a+x bin/floe.py
#On the coordinator node Start coordinator
bin/floe.py coordinator &
#Start multiple containers, ONE per Machine
#You may use any number of -Dconfig.property=value options to override properties mentioned in the config file.
bin/floe.py container
#to run the sample Application (from the coordinator node or a client node), run:
bin/floe.py jar floe-examples/target/floe-examples-0.1-SNAPSHOT.jar edu.usc.pgroup.floe.examples.WordCountApp
#For different components, all logs are appended to the file alllogs.log locally, log in to individual machine and use tail to see the ouput of the above command.
tail -f alllogs.log