Detailed tutorial on installing CentOS, JDK and Hadoop on VirtualBox

1. Prerequisites

1.1 Supported Platforms

GNU/Linux is the platform on which the product is developed and runs. Hadoop has been verified on a cluster system consisting of 2000 GNU/Linux hosts.
So first we need to download the Linux ISO installation package and install it. For specific installation, please refer to my previous blog:
Installation of VirtualBox virtual machine and CentOS system [Details]

1.2 jdk and hadoop installation packages

You can go to the official website to download or download the resources I uploaded. The specific version I use is shown in the figure below:

Link: https://pan.baidu.com/s/1XTZfzl0t79Dtz0-jmQrQuQ Extraction code: pkwy

1.3 Xshell 7 and Xftp 7 tool installation

The main function is to use Xshell 7 to perform various operations on the virtual machine and use Xftp 7 to transfer the local jdk and hadoop installation packages to the virtual machine

Both tools can be downloaded from the official website. If you choose personal use, you can download them for free.

2. Specific installation steps

2.1 Network Configuration

It should be noted that when installing CentOS and setting up network connections in the previous blog, both gateways must be opened, as follows:
Network setup before startup

Then click Start. When you proceed to the following steps, make sure to turn on both gateways and manually configure the second network.

Click Configure, manually configure Ipv4, fill in the address in the box, and click Save.

After installing CentOS, click Start. After logging in to the root account in the terminal, enter the ifconfig command to check the network configuration. If it does not exist, execute the command yum install net-tools.x86_64. If the y/n prompt pops up, select y.

implement

vi /etc/sysconfig/network-scripts/ifcfg-enp0s8

Change onboot=no to yes
Click the insert key (i.e. the i key on the keyboard), move up and down to where you want to change it, press esc to exit the edit mode, and then press :wq to save and exit. Note that:
As expected, the network has been configured.
Restart the virtual machine or restart the network card service

service network restart

After logging in, enter the ifconfig command again. There will be feedback information. Write down the IP address in the box as shown in the figure.

2.2 Xshell connection

Open Xshell, create a new connection, and write the IP address into the box below to establish a connection with the virtual machine locally. All subsequent operations on the virtual machine can be performed in the Xshell terminal.

2.3 Xftp transmission

Click the button below to create a new file transfer and automatically call Xftp for file transfer

Create a new java subfolder under the root folder on the right, find the local file, right-click -> Transfer, and then upload the two compressed packages to the virtual machine.

2.4 jdk environment configuration

Enter the following command to check whether JDK is installed by default. If there is no feedback information, it means it is not installed.

use

> cd corresponding folder path

Change the directory to the java subfolder,
Use the following command to start decompressing the jdk compressed package

Configure environment variables through /etc/profile

[root@bogon ~]# vi /etc/profile

Enter edit mode (i), add the following code, then save and exit (esc+:+wq)

#Finally add the following content, pay attention to check your own path and jdk version export JAVA_HOME=/root/java/jdk-16
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

To make the /etc/profile file take effect immediately after modification, you can use the following command:

[root@bogon ~]# source /etc/profile

Check whether the installation is successful:

[root@bogon ~]# java -version 

2.5 ssh service configuration

ssh must be installed and sshd must be running so that the Hadoop script can be used to manage the remote Hadoop daemon. Check whether the ssh service is installed:

> [root@bogon ~]# rpm -qa|grep ssh

If it has been installed, the corresponding version will be displayed

If not installed, install it using the following command

[root@bogon ~]# yum install openssh-server

Enable sshd service

[root@bogon ~]# sudo service sshd start

In order to avoid having to manually start the sshd service every time you start CentOS, you can add the sshd service to the auto-start list and enter

[root@bogon ~]# systemctl enable sshd.service

You can enter

[root@bogon ~]# systemctl list-unit-files | grep sshd

Check whether the sshd service is enabled.

2.6 Hadoop Configuration

To obtain a Hadoop distribution, download the latest stable release from one of Apache's mirror servers.
Here we use the stable version hadoop-1.0.3.tar.gz in the installation package
Just like the previous decompression of JDK, decompress the hadoop-1.0.3.tar.gz file in the java folder. The specific operation is to switch the terminal to the java path and use the following command to decompress the compressed package.

tar -zxvf hadoop-1.0.3.tar.gz

The decompression results are as follows

Try the following command:

[root@bogon java]# cd /root/java/hadoop-1.0.3
[root@bogon hadoop-1.0.3]# bin/hadoop

The usage documentation of the hadoop script will be displayed

2.7 Hadoop stand-alone mode operation method

By default, Hadoop is configured to run as a standalone Java process in non-distributed mode. This is very helpful for debugging.
The following command runs the hadoop program once and gets the output.

The following example takes a copy of the unpacked conf directory as input, searches for and displays entries that match a given regular expression, and writes the output to the specified output directory:

[root@bogon hadoop-1.0.3]# mkdir input 
[root@bogon hadoop-1.0.3]# cp conf/*.xml input 
[root@bogon hadoop-1.0.3]# bin/hadoop jar hadoop-examples-1.0.3.jar grep input output 'dfs[az.]+'
[root@bogon hadoop-1.0.3]# cat output/*

The following shows the results of running the hadoop program

This is the end of this article about the detailed tutorial on how to install CentOS and JDK and Hadoop on VirtualBox. For more information about how to install JDK and Hadoop on VirtualBox and CentOS, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!