hadoop 多機安裝

環境：10台 ubuntu 10.04
    hostname          
     node1        namenode & jobtracker & datanode & tasktracker
     node2        datanode & tasktracker
     node3        datanode & tasktracker
     node4        datanode & tasktracker
     node5        datanode & tasktracker
     node6        datanode & tasktracker
     node7        datanode & tasktracker
     node8        datanode & tasktracker
     node9        datanode & tasktracker
     node10      datanode & tasktracker

每一台  創建  username(hadooper)  userpasswd(*****) usergroup(hadooper)

並把改使用者設定為具有車籍使用者的權限 ＝>sudo adduser username admin

step0: node1~node10 再第二裝網卡設定ip
step1:   設定兩台機器ssh登入免密碼     (再node1做)

  ssh-keygen -t rsa -f ~/.ssh/id_rsa -P "" 

  cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys 

  scp -r ~/.ssh node2:~/（在node1做 複製到 node2~node10）

step2:安裝java

   (node1~node10)     
sudo apt-get purge java-gcj-compat 

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner" 

sudo apt-get update 

sudo apt-get install sun-java6-jdk sun-java6-plugin 

sudo update-java-alternatives -s java-6-sun

step3:下載安裝Hadoop

~$ cd /opt
/opt$ sudo wget http://ftp.twaren.net/Unix/Web/apache/hadoop/core

     /hadoop-0.20.2/hadoop-0.20.2.tar.gz
/opt$ sudo tar zxvf hadoop-0.20.2.tar.gz
/opt$ sudo mv hadoop-0.20.2/ hadoop
/opt$ sudo chown -R 使用者帳號:使用者群組 hadoop
/opt$ sudo mkdir /var/hadoop
/opt$ sudo chown -R 使用者帳號:使用者群組 /var/hadoop 

step4: 設定 hadoop-env.sh

/opt$ cd hadoop/
/opt/hadoop$ gedit conf/hadoop-env.sh

export JAVA_HOME=/usr/lib/jvm/java-6-sun
export HADOOP_HOME=/opt/hadoop
export HADOOP_CONF_DIR=/opt/hadoop/conf
export HADOOP_LOG_DIR=/tmp/hadoop/logs
export HADOOP_PID_DIR=/tmp/hadoop/pids 

step5: 設定 hadoop-site.xml

/opt/hadoop# gedit conf/core-site.xml 

<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://node1:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/var/hadoop/hadoop-${user.name}</value>
  </property>
</configuration>  

/opt/hadoop# gedit conf/hdfs-site.xml 

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>2</value>
  </property>
</configuration>  

/opt/hadoop# gedit conf/mapred-site.xml 

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>node1:9001</value>
  </property>
</configuration>  

step6: 設定masters及slaves

 /opt/hadoop$ gedit conf/slaves

step7: 將第一台設定的資料copy到另外幾台

ssh node2~node10

/opt/hadoop$ "sudo mkdir /opt/hadoop"
/opt/hadoop$ "sudo chown -R 使用者帳號:使用者群組 /opt/hadoop"
/opt/hadoop$ "sudo mkdir /var/hadoop"
/opt/hadoop$ "sudo chown -R 使用者帳號:使用者群組 /var/hadoop" 

/opt/hadoop$ scp -r /opt/hadoop/* node2:/opt/hadoop/

step8: 格式化HDFS

/opt/hadoop$ bin/hadoop namenode -format 

step9: 啟動Hadoop(在node1上執行)

/opt/hadoop$ bin/start-dfs.sh

 /opt/hadoop$ /opt/hadoop/bin/start-mapred.sh