How to install and use Ubuntu Docker

1. Automatic installation using the official installation script

Manual Installation

Uninstall old versions

2. Install using Docker repository

Install Docker Engine-Community

3. Install using Shell script

Uninstall Docker

2. Create a container

5. Building a big data environment in Docker

Install MySQL

Install JDK

Install Hadoop

Install Hive

Install Sqoop

Docker Engine-Community supports the following Ubuntu versions:

Xenial 16.04 (LTS)
Bionic 18.04 (LTS)
Cosmic 18.10
Disco 19.04
Other updated versions...

Docker Engine - Community supports x86_64 (or amd64), armhf, arm64, s390x (IBM Z), and ppc64le (IBM Power) architectures.

1. Automatic installation using the official installation script

The installation command is as follows:

curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

You can also use the domestic daocloud one-click installation command:

curl -sSL https://get.daocloud.io/docker | sh

Manual Installation

Uninstall old versions

Older versions of Docker were called docker, docker.io, or docker-engine. If installed, uninstall them:

$ sudo apt-get remove docker docker-engine docker.io containerd runc

The Docker Engine - Community package is currently called docker-ce.

Install Docker Engine-Community. There are two ways to do this.

2. Install using Docker repository

Before installing Docker Engine-Community for the first time on a new host, you need to set up the Docker repository. Afterwards, you can install and update Docker from the repository.

Setting up a warehouse
Update the apt package index.

$ sudo apt-get update

Install apt dependency packages to obtain the repository via HTTPS:

$ sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

Add Docker's official GPG key:

$ curl -fsSL https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/gpg | sudo apt-key add -

9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88 Verify that you now have the key with the fingerprint by searching for the last 8 characters of the fingerprint.

$ sudo apt-key fingerprint 0EBFCD88
   
pub rsa4096 2017-02-22 [SCEA]
      9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid [ unknown] Docker Release (CE deb) <[email protected]>
sub rsa4096 2017-02-22 [S]

Use the following command to set up a stable version repository

$ sudo add-apt-repository \
   "deb [arch=amd64] https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu/ \
  $(lsb_release -cs) \
  stable"

Install Docker Engine-Community

Update the apt package index.

$ sudo apt-get update

Install the latest version of Docker Engine - Community and containerd, or go to the next step to install a specific version:

$ sudo apt-get install docker-ce docker-ce-cli containerd.io

To install a specific version of Docker Engine - Community, list the available versions in the repository and choose one to install. List the versions available in your repository:

$ apt-cache madison docker-ce

  docker-ce | 5:18.09.1~3-0~ubuntu-xenial | https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu xenial/stable amd64 Packages
  docker-ce | 5:18.09.0~3-0~ubuntu-xenial | https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu xenial/stable amd64 Packages
  docker-ce | 18.06.1~ce~3-0~ubuntu | https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu xenial/stable amd64 Packages
  docker-ce | 18.06.0~ce~3-0~ubuntu | https://mirrors.ustc.edu.cn/docker-ce/linux/ubuntu xenial/stable amd64 Packages
  ...

Use the version string in the second column to install a specific version, for example 5:18.09.1~3-0~ubuntu-xenial.

$ sudo apt-get install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING> containerd.io

Test whether Docker is installed successfully. Enter the following command. If the following information is printed, the installation is successful:

$ sudo docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete Digest: sha256:c3b4ada4687bbaa170745b3e4dd8ac3f194ca95b2d0518b417fb47e5879d9b5f
Status: Downloaded newer image for hello-world:latest


Hello from Docker!
This message shows that your installation appears to be working correctly.


To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon creates a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.


To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash


Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/


For more examples and ideas, visit:
 https://docs.docker.com/get-started/

3. Install using Shell script

Docker provides convenience scripts on get.docker.com and test.docker.com that will quickly install edge and test versions of Docker Engine-Community. The source code of the script is in the docker-install repository. These scripts are not recommended for use in production environments, and you should be aware of the potential risks before using them:

The script needs to be run as root or with sudo privileges. Therefore, you should carefully review and audit your scripts before running them.
These scripts attempt to detect your Linux distribution and version and configure the package management system for you. Additionally, the script does not allow you to customize any installation parameters. This can result in an unsupported configuration from either a Docker perspective or from the perspective of your own organization's guidelines and standards.
These scripts will install all dependencies and suggestions for package managers without asking for confirmation. This may install a large number of packages, depending on the current configuration of the host.
The script does not provide an option to specify which version of Docker to install; instead, it installs the latest version released in the edge channel.
Do not use the convenience script if you have already installed Docker on your host using some other mechanism.

This example uses a script from get.docker.com to install the latest version of Docker Engine - Community on Linux. To install the latest testing version, use test.docker.com instead. In each command below, replace every occurrence of get with test.

$ curl -fsSL https://get.docker.com -o get-docker.sh
$ sudo sh get-docker.sh

If you want to use Docker as a non-root user, you should consider adding your user to the docker group using something like:

$ sudo usermod -aG docker your-user

Uninstall Docker

Delete the installation package:

sudo apt-get purge docker-ce

Delete images, containers, configuration files, etc.:

sudo rm -rf /var/lib/docker

2. Create a container

Pull the image: sudo docker pull centos:7

insert image description here

Start and create the container:

sudo docker run -itd --privileged --name singleNode -h singleNode \
-p 2222:22 \
-p 3306:3306 \
-p 50070:50070 \
-p 8088:8088 \
-p 8080:8080 \
-p 10000:10000 \
-p 60010:60010 \
-p 9092:9092 \
centos:7 /usr/sbin/init

insert image description here

Enter the container: sudo docker exec -it singleNode /bin/bash

insert image description here

At this point, you have successfully entered the container

5. Building a big data environment in Docker

Installing Components

yum clean all
yum -y install unzip bzip2-devel vim bashname

Configuring SSH password-free login

yum install -y openssh openssh-server openssh-clients openssl openssl-devel 
ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' 
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# Start the SSH service systemctl start sshd

Set the time zone

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

insert image description here

If there is a firewall, you need to turn it off:

systemctl stop firewalld
systemctl disable firewalld

Create a folder:

#Store the decompressed package mkdir -p /opt/install

Exit the container: exit

Upload the big data component package to a directory in Ubuntu and create a directory named software

Copy the component package in Ubuntu to the container

sudo docker cp /home/zy/software/ singleNode:/opt/

Reenter the container

sudo docker exec -it singleNode /bin/bash

Install MySQL

Enter the installation package path

cd /opt/software

Unzip the package

tar zxvf MySQL-5.5.40-1.linux2.6.x86_64.rpm-bundle.tar -C /opt/install

Install Dependencies

yum -y install libaio perl

Install the server and client

#First go to the path where MySQL is unzipped cd /opt/install
#Install the server rpm -ivh MySQL-server-5.5.40-1.linux2.6.x86_64.rpm
#Install the client rpm -ivh MySQL-client-5.5.40-1.linux2.6.x86_64.rpm

Start and configure MySQL

#Step 1: Start systemctl start mysql
#Step 2: Configure username and password /usr/bin/mysqladmin -u root password 'root'
#Step 3: Enter mysql
mysql -uroot -proot 
#Step 4: Configuration> update mysql.user set host='%' where host='localhost';
> delete from mysql.user where host<>'%' or user='';
> flush privileges;
#Installation complete quit

Install JDK

Unzip the package

tar zxvf /opt/software/jdk-8u171-linux-x64.tar.gz -C /opt/install/

Create a soft link

#Alias: java
ln -s /opt/install/jdk1.8.0_171 /opt/install/java

Configure environment variables:

vi /etc/profile
#Add the following configuration information export JAVA_HOME=/opt/install/java
export PATH=$JAVA_HOME/bin:$PATH

Effective configuration file:

source /etc/profile

Check the Java version:

java -version

Install Hadoop

Unzip the package

tar zxvf /opt/software/hadoop-2.6.0-cdh5.14.2.tar_2.gz -C /opt/install/

Create a soft link

ln -s /opt/install/hadoop-2.6.0-cdh5.14.2 /opt/install/hadoop

Configure core-site.xml

vi core-site.xml
-------------------------------------------
<configuration>
  <property>
    <name>fs.defaultFS</name>
    <value>hdfs://singleNode:9000</value>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/opt/install/hadoop/data/tmp</value>
  </property>
</configuration>
-------------------------------------------

Configure hdfs-site.xml

vi hdfs-site.xml
-------------------------------------------
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property
</configuration>
-------------------------------------------

Configure mapred-site.xml

vi mapred-site.xml.template
-------------------------------------------
<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.address</name>
    <value>singleNode:10020</value>
  </property>
  <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>singleNode:19888</value>
  </property>
</configuration>
-------------------------------------------

Configure yarn-site.xml

vi yarn-site.xml
-------------------------------------------
<configuration>
	<property>
		<name>yarn.nodemanager.aux-services</name>
		<value>mapreduce_shuffle</value>
	</property>
	<property>
		<name>yarn.resourcemanager.hostname</name>
		<value>singleNode</value>
	</property>
	<property>
		<name>yarn.log-aggregation-enable</name>
		<value>true</value>
	</property>
	<property>
		<name>yarn.log-aggregation.retain-seconds</name>
		<value>604800</value>
	</property>
</configuration>
-------------------------------------------

Configure hadoop-env.sh

vi hadoop-env.sh
-------------------------------------------
export JAVA_HOME=/opt/install/java
-------------------------------------------

Configure mapred-env.sh

vi mapred-env.sh
-------------------------------------------
export JAVA_HOME=/opt/install/java
-------------------------------------------

Configure yarn-env.sh

vi yarn-env.sh
-------------------------------------------
export JAVA_HOME=/opt/install/java
-------------------------------------------

Configuring slaves

export HADOOP_HOME=/opt/install/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH

HDFS Formatting

hdfs namenode -format

Start Hadoop Services

start-all.sh

Web View

#Address 192.168.**.**:50070

insert image description here

Install Hive

Unzip the package

tar zxvf /opt/software/hive-1.1.0-cdh5.14.2.tar.gz -C /opt/install/

Create a soft link

ln -s /opt/install/hive-1.1.0-cdh5.14.2 /opt/install/hive

Modify the configuration file:

#Go to the configuration file path cd /opt/install/hive/conf/

Modify hive-site.xml

vi hive-site.xml
-------------------------------------------
<configuration>
	<property>
		<name>hive.metastore.warehouse.dir</name>
		<value>/home/hadoop/hive/warehouse</value>
	</property>
	<property>
		<name>javax.jdo.option.ConnectionURL</name>
		<value>jdbc:mysql://singleNode:3306/hive?createDatabaseIfNotExist=true</value>
	</property>
	<property>
		<name>javax.jdo.option.ConnectionDriverName</name>
		<value>com.mysql.jdbc.Driver</value>
	</property>
	<property>
		<name>javax.jdo.option.ConnectionUserName</name>
		<value>root</value>
	</property>
	<property>
		<name>javax.jdo.option.ConnectionPassword</name>
		<value>root</value>
	</property>
	<property>
		<name>hive.exec.scratchdir</name>
		<value>/home/hadoop/hive/data/hive-${user.name}</value>
		<description>Scratch space for Hive jobs</description>
	</property>

	<property>
		<name>hive.exec.local.scratchdir</name>
		<value>/home/hadoop/hive/data/${user.name}</value>
		<description>Local scratch space for Hive jobs</description>
	</property>
</configuration>
-------------------------------------------

Modify hive-env.sh.template

vi hive-env.sh.template
-------------------------------------------
HADOOP_HOME=/opt/install/hadoop
-------------------------------------------

Adding Dependencies

cp /opt/software/mysql-connector-java-5.1.31.jar /opt/install/hive/lib/

Add environment variables

vi /etc/profile
#Add the following configuration information export HIVE_HOME=/opt/install/hive
export PATH=$HIVE_HOME/bin:$PATH

Start the service

nohup hive --service metastore &
nohup hive --service hiveserver2 &

View the process: jps

Install Sqoop

Unzip the package

tar zxvf /opt/software/sqoop-1.4.6-cdh5.14.2.tar.gz -C /opt/install/

Create a soft link

ln -s /opt/install/sqoop-1.4.6-cdh5.14.2 /opt/install/sqoop

Modify sqoop-env-template.sh

cd /opt/install/sqoop/conf/
vi sqoop-env-template.sh
-------------------------------------------
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/opt/install/hadoop

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/opt/install/hadoop

#Set the path to where bin/hive is available
export HIVE_HOME=/opt/install/hive
-------------------------------------------

Add dependency packages

cp /opt/software/mysql-connector-java-5.1.31.jar /opt/install/sqoop/lib/
cp /opt/software/java-json.jar /opt/install/sqoop/lib/

Add environment variables

vi /etc/profile
#Add the following configuration information export SQOOP_HOME=/opt/install/sqoop
export PATH=$SQOOP_HOME/bin:$PATH

View Version

sqoop version

This is the end of this article about how to install and use Ubuntu Docker. For more information about Ubuntu Docker installation and use, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!

You may also be interested in: