Tutorial on installing Elasticsearch 7.6.2 in Docker

Tutorial on installing Elasticsearch 7.6.2 in Docker

Install Docker

You have to install Docker, no further instructions.

Install Elasticsearch

Note: The version used is 7.6.2, you can choose other versions

Pull the image

docker pull elasticsearch:7.6.2

Start the container

docker run --restart=always -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" \

-e ES_JAVA_OPTS="-Xms512m -Xmx512m" \

--name='elasticsearch' --cpuset-cpus="1" -m 2G -d elasticsearch:7.6.2

illustrate:

1. -v /opt/hanlp:/opt/hanlp If hanlp word segmentation is used, the word library needs to be mounted

2. ES_JAVA_OPTS can set parameters

3. Single node startup

Access address: http://172.18.63.211:9200

Plugin Installation

Install ik tokenizer

Download the corresponding version: elasticsearch-analysis-ik

Why install IK, lightweight. A configured vocabulary can also be used for Chinese word segmentation. HanLP is heavyweight and has many built-in algorithms, so it is not suitable for single word segmentation.

# Offline installation, download the corresponding plug-in zip
# https://github.com/medcl/elasticsearch-analysis-ik
docker cp /opt/elasticsearch-analysis-ik-7.6.2.zip elasticsearch:/opt
docker exec -it elasticsearch bash
cd plugins/
mkdir analysis -ik
unzip -d /usr/share/elasticsearch/plugins/analysis-ik/ /opt/elasticsearch-analysis-ik-7.6.2.zip 
exit
docker restart elasticsearch

Custom vocabulary

Custom Dictionary

Remote vocabulary

Common maintenance commands

# View all index information GET /_cat/indices?pretty
# Node monitoring GET /_cat/health?pretty
# Which plugins are installed? GET _cat/plugins

Other commands will be sorted out when they are used. This article mainly talks about installation and deployment.

Monitoring and development tool Kibana

Kibana is an open source analytics and visualization platform designed for Elasticsearch. You can use Kibana to search, view, and interact with data stored in Elasticsearch indices. You can easily perform advanced data analysis and visualization, presenting it in the form of icons.

Our server IP is 172.18.63.211

docker run --restart=always --link elasticsearch:elasticsearch --name kibana -p 5601:5601 -d kibana:7.6.2

Enter the container and modify the configuration file kibana.yml

docker exec -it kibana bash
vi config/kibana.yml
########################
#Specify the address of es elasticsearch.hosts: ["http://172.18.63.211:9200"]
# Chinese i18n.locale: "zh-CN"
# Modify the optional server.host for external network access: "0.0.0.0"
exit
########################
docker restart kibana

Open address: http://172.18.63.211:5601

Test word segmentation tool

POST _analyze
{
 "text": "Primers for detecting the sulfonylurea herbicide resistance gene BnALS3R in Brassica napus and its application",
 "analyzer": "hanlp"
}

Added index library

PUT achievement
{
 "settings": {
 "number_of_shards": 1,
 "number_of_replicas": 1
 }
}

PUT achievement/_mapping
{
 "properties": {
 "id": {
  "type": "text"
 },
 "owner": {
  "type": "text"
 },
 "title": {
  "type": "text",
  "analyzer": "hanlp"
 },
 "description": {
  "type": "text",
  "analyzer": "hanlp"
 },
 "update_time":{
  "type": "date"
 }
 }
}

Data synchronization Logstash

Used to collect, parse and convert logs, synchronize data, etc.

Install

docker pull logstash:7.5.0

Configuration file directory

mkdir -p /usr/local/logstash/config
cd /usr/local/logstash/config
touch logstash.yml
vi log4j2.properties
#####Add the following content logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = debug
#####
vi pipelines.yml
####
- pipeline.id: logstash-match
 path.config: "/usr/share/logstash/config/*.conf"
 pipeline.workers: 3
####

At the same time, you need to put the MySQL driver package into the configuration file.

Then create a configuration file

Here is an example of periodically synchronizing MySQL data to es. *

# logstash-mysql-es.conf
input{
 jdbc{
 jdbc_driver_class => "com.mysql.cj.jdbc.Driver"
 jdbc_connection_string => "jdbc:mysql://172.18.63.211:3306/open_intelligence?characterEncoding=utf8&serverTimezone=Asia/Shanghai"
 jdbc_user => "docker"
 jdbc_password => "docker@12345"
 jdbc_paging_enabled => true
 jdbc_page_size => 10000
 jdbc_fetch_size => 10000
 connection_retry_attempts => 3
 connection_retry_attempts_wait_time => 1
 jdbc_pool_timeout => 5
 use_column_value => true
 tracking_column => "update_time"
 tracking_column_type => "timestamp"
 record_last_run => true
 last_run_metadata_path => "/usr/share/logstash/mysql/goods_achievement"
 statement => "select * from goods_achievement where update_time > :sql_last_value"
 schedule => "* */30 * * * *"
 }
}

filter{
 mutate {
 split => { "feature1" => ";" }
 }
 mutate {
 split => { "feature2" => ";" }
 }
 mutate {
 split => { "feature3" => ";" }
 }
}

output {
 elasticsearch
 document_id => "%{id}"
 index => "goods_achievement"
 hosts => ["http://172.18.63.211:9200"]
 }
}

start up

docker run -d -p 5044:5044 -p 9600:9600 -it \
-e TZ=Asia/Shanghai \
--name logstash --restart=always \
-v /usr/local/logstash/config/:/usr/share/logstash/config/ \
-v /usr/local/logstash/mysql/:/usr/share/logstash/mysql/ \
--privileged=true \
logstash:7.6.2

If an error is reported

Error: com.mysql.cj.jdbc.Driver not loaded. :jdbc_driver_library is not set, are you sure you included the proper driver client libraries in your classpath?

You can try to copy the driver, mysql-connector-java-xxxx-bin.jar, to the logstash directory\logstash-core\lib\jars

like:

cd /usr/local/logstash/config

docker cp mysql-connector-java-8.0.17.jar logstash:/usr/share/logstash/logstash-core/lib/jars

Detection Profile

bin/logstash -f /usr/local/logstash/config/mysql-es-patent.conf -t

Done, you can start developing.

The above tutorial on installing Elasticsearch 7.6.2 in Docker is all I have to share with you. I hope it can give you a reference. I also hope that you will support 123WORDPRESS.COM.

You may also be interested in:
  • Problems and solutions for installing ElasticSearch and Kibana in Docker
  • How to install Elasticsearch7.6 cluster in docker and set password
  • How to install ElasticSearch on Docker in one article
  • Implementation of Docker deployment of ElasticSearch and ElasticSearch-Head
  • Teach you how to install elasticsearch and head plug-ins using docker

<<:  In-depth analysis of MySQL lock blocking

>>:  Detailed explanation of Vue transition effects and animation transition usage examples

Recommend

How to use crontab to add scheduled tasks in Linux

Preface The Linux system is controlled by the sys...

Pure CSS code to achieve flow and dynamic line effects

Ideas: An outer box sets the background; an inner...

Analysis of several situations where MySQL index fails

1. Best left prefix principle - If multiple colum...

JS ES6 asynchronous solution

Table of contents Initially using the callback fu...

Detailed explanation of the pitfalls of MySQL 8.0

I updated MySQL 8.0 today. The first problem: Nav...

Detailed analysis of MySQL master-slave replication

Preface: In MySQL, the master-slave architecture ...

Common Linux English Error Chinese Translation (Newbies Must Know)

1.command not found command not found 2. No such ...

Example of implementing a virtual list in WeChat Mini Program

Table of contents Preface analyze Initial Renderi...

Tutorial on installing Ceph distributed storage with yum under Centos7

Table of contents Preface Configure yum source, e...

Centos8 builds nfs based on kdc encryption

Table of contents Configuration nfs server (nfs.s...

Detailed tutorial on deploying Django project under CentOS

Basic Environment Pagoda installation service [Py...

Summary of vue's webpack -v error solution

Xiaobai learned about Vue, then learned about web...