Detailed installation and configuration of hadoop2.7.2 under ubuntu15.10

There are many Hadoop installation tutorials on Linux on the Internet and in many books, but readers who have tried will find that even if they follow these tutorials step by step, they will still go wrong somewhere. The main reason is version issues. In particular, the versions in the books are generally older, so it is not recommended to follow the tutorials in the books, but they can be used as a reference. The following is the installation process that I have personally experienced. I will try to write down the principles that I know, so that even if there are problems in some steps, it is easy to solve them once you know the reasons. The only thing to worry about is that if there are problems, you still don’t know where the problems are.

The following codes are all entered in the Ubuntu command window (terminal):

1. Tool preparation:

1.1 Update apt-get: It is convenient to use apt-get to install software in the command line window;

sudo apt-get update

1.2 Install vim: configuration file.

sudo apt-get install vim

Depending on the output, you need to enter "Y" to continue.

2. Create a hadoop user:

2.1 Add hadoop user: Create hadoop user and use /bin/bash as shell

sudo useradd -m hadoop -s /bin/bash

2.2 Set the password and enter it twice as prompted:

sudo passwd hadoop

2.3 Add administrator privileges to the hadoop user:

sudo adduser hadoop sudo

3. Install SSH and configure SSH password-free login

Both cluster and single-node modes require SSH login (similar to remote login, you can log in to a Linux host and run commands on it). Ubuntu has an SSH client installed by default. In addition, you need to install an SSH server:

sudo apt-get install openssh-server

After installation, you can log in to the machine using the following command:

ssh localhost

4. Install Java environment

4.1 Install JDK:

sudo apt-get install openjdk-8-jre openjdk-8-jdk

4.2 Configure JAVA_HOME

dpkg -L openjdk-8-jdk | grep '/bin/javac'

This command will output a path. Remove "/bin/javac" from the end of the path and the remaining path is the correct path. If the output path is /usr/lib/jvm/java-8-openjdk/bin/javac, the path we need is /usr/lib/jvm/java-8-openjdk.

If the output here is wrong, how do we know the address of JAVA_HOME? It's very simple, just type:

java -verbose

Output: usr/lib/jvm/java-8-openjdk/jre/lib/rt.jar

The previous usr/lib/jvm/java-8-openjdk is the address of JAVA_HOME.

Then, open the /etc/profile file using the command sudo gedit /etc/profile:

sudo gedit /etc/profile

Then add the following three lines at the end of the file:

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk
CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

Then press ctrl+s to save. Then execute the following code to make the changes take effect:

$source ~/.bashrc

If JAVA_HOME still does not change, it may be that the file is not effective. Run

$source /etc/profile

Check JAVA_HOME again and you can see that it is the path we set in the file.

If the above settings are correct, $JAVA_HOME/bin/java -version will output the java version information, and the output result is the same as java -version, as shown in the figure below, then the configuration is successful:

5. Install Hadoop

5.1hadoop program download: http://mirror.bit.edu.cn/apache/hadoop/common/

It is recommended to download hadoop-2.xytar.gz under "stable".

5.2 Hadoop Installation

Enter the command:

sudo tar -zxf ~/Downloads/hadoop-2.7.2.tar.gz -C /usr/local  
cd /usr/local/
sudo mv ./hadoop-2.7.2/ ./hadoop      
sudo chown -R hadoop ./hadoop

Hadoop is available for use after decompression. Enter the following command to check whether Hadoop is available. If successful, the Hadoop version information will be displayed:

cd /usr/local/hadoop
./bin/hadoop version

5.3 Hadoop Click Configuration

The default mode of Hadoop is non-distributed mode and can run without any additional configuration. Non-distributed means a single Java process, which is convenient for debugging.

The configuration has not been completed yet. As I have been very busy recently, I will configure it when I have some free time in a couple of days.

Summarize

The above is the full content of this article. I hope that the content of this article will have certain reference learning value for your study or work. Thank you for your support of 123WORDPRESS.COM. If you want to learn more about this, please check out the following links