VMware configuration hadoop to achieve pseudo-distributed graphic tutorial

2. Prepare the Linux network environment

1.0 Click the VMware shortcut, right-click to open the file location -> double-click vmnetcfg.exe -> VMnet1 host-only -> modify subnet ip to set the network segment: 192.168.150.0 subnet mask: 255.255.255.0 -> apply -> ok

Go back to Windows --> Open Network and Sharing Center -> Change adapter settings -> Right click VMnet1 -> Properties -> Double-click IPv4 -> Set Windows IP: 192.168.150.110 Subnet mask: 255.255.255.0 -> Click OK

On the virtualization software--My Computer -> select the virtual machine -> right click -> settings -> network adapter -> host only -> ok

3. Configure hostname

Change the host name (permanently change)
Modify the hosts file

4. Modify the IP in centos

Modify through the Linux graphical interface (strongly recommended)

Enter the Linux graphical interface -> right click on the two small computers in the upper right corner -> click Edit connections -> select the current network System eth0 -> click the edit button -> select IPv4 -> select manual for method -> click the add button -> add IP: 192.168.150.110 Subnet mask: 255.255.255.0 Gateway: 192.168.150.1 -> apply

5. Permanently turn off the firewall

Check the firewall status

Stop the firewall

Permanently turn off the firewall

6. Unzip JDK

Modify the environment variables and source them

7. Unzip Hadoop

8. Modify configuration files (5)

First: hadoop-env.sh, under /cloud/hadoop-2.8.5/etc/hadoop

Modify about line 27. To see the line number, you can vi the file, press esc, enter a colon, enter set nu and press Enter, and the line number will be displayed.

export JAVA_HOME=/usr/java/jdk1.8.0_181 

Second: core-site.xml

<configuration>
		<!-- Specify the communication address of the HDFS boss (namenode) -->
		<property>
				<name>fs.defaultFS</name>
				<value>hdfs://itcast01:9000</value>
		</property>
		<!-- Specify the storage path for files generated by hadoop during runtime-->
		<property>
				<name>hadoop.tmp.dir</name>
				<value>/cloud/hadoop-2.8.5/tmp</value>
		</property>
	</configuration> 

Third: mapred-site.xml.template needs to be renamed: mv mapred-site.xml.template mapred-site.xml

	<configuration>
<!-- Notify the framework MR to use YARN -->
<property>
		<name>mapreduce.framework.name</name>
		<value>yarn</value>
</property>
		</configuration> 

After finishing here, enter and modify the name of the xml file

mv mapred-site.xml.template mapred-site.xml

Fourth: hdfs-site.xml

	<configuration>
<!-- Set the number of HDFS replicas -->
<property>
		<name>dfs.replication</name>
		<value>1</value>
</property>
		</configuration> 

Fifth: yarn-site.xml

	<configuration>
<!-- The way reducer gets data is mapreduce_shuffle -->
<property>
	<name>yarn.nodemanager.aux-services</name>
	<value>mapreduce_shuffle</value>
</property>
		</configuration> 

9. Add hadoop to environment variables

vim /etc/profile

	export JAVA_HOME=/usr/java/jdk1.8.0_181
export HADOOP_HOME=/cloud/hadoop-2.8.5
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin
	
		source /etc/profile 

10. Format HDFS (namenode) when using it for the first time (forgot to take a screenshot)

hadoop namenode -format

11. Start Hadoop

Start start-all
Enter ./start-all.sh in /cloud/hadoop/sbin/

Then keep entering the password and yes

12. Enter jps to check whether the configuration is successful

There are 6 things in the red box, and none of them can be missing, otherwise something is not configured properly. According to the experience of my classmates, either the network is not configured properly, or the 5 configuration files above are not written properly. If you can't find them, the simplest, crudest, troublesome but effective method is to install the machine from the beginning. I succeeded in one go.

Enter the browser in centos

Enter the centos ip port number 50070 from the browser to see the following picture

Enter the centos ip port number 8088 from the browser and the following picture will appear, success

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.