theluyuan/linux-tutorial
1
0
Fork 0
mirror of https://github.com/dunwu/linux-tutorial.git synced 2024-04-15 19:55:24 +08:00
Code Issues Projects Releases Wiki Activity
e3e645a29a
linux-tutorial/linux/soft/elastic/elastic-quickstart.html
Travis CI User e3e645a29a deploy
2021-05-13 17:44:54 +08:00

101 lines
41 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Elastic 技术栈之快速入门 | LINUX-TUTORIAL</title>
<meta name="generator" content="VuePress 1.8.2">
<link rel="icon" href="/linux-tutorial/favicon.ico">
<meta name="description" content="数据库教程">
<link rel="preload" href="/linux-tutorial/assets/css/0.styles.45d9d031.css" as="style"><link rel="preload" href="/linux-tutorial/assets/js/app.79a38eea.js" as="script"><link rel="preload" href="/linux-tutorial/assets/js/4.fb6e0f89.js" as="script"><link rel="preload" href="/linux-tutorial/assets/js/54.e78d2776.js" as="script"><link rel="preload" href="/linux-tutorial/assets/js/5.cb43ecfb.js" as="script"><link rel="prefetch" href="/linux-tutorial/assets/js/10.7933187b.js"><link rel="prefetch" href="/linux-tutorial/assets/js/11.b9b41530.js"><link rel="prefetch" href="/linux-tutorial/assets/js/12.70a5dba8.js"><link rel="prefetch" href="/linux-tutorial/assets/js/13.857dcc43.js"><link rel="prefetch" href="/linux-tutorial/assets/js/14.5a603a55.js"><link rel="prefetch" href="/linux-tutorial/assets/js/15.d217acb7.js"><link rel="prefetch" href="/linux-tutorial/assets/js/16.ad565eae.js"><link rel="prefetch" href="/linux-tutorial/assets/js/17.d43e9f56.js"><link rel="prefetch" href="/linux-tutorial/assets/js/18.aa00ff43.js"><link rel="prefetch" href="/linux-tutorial/assets/js/19.43ce44b3.js"><link rel="prefetch" href="/linux-tutorial/assets/js/20.5618e1ff.js"><link rel="prefetch" href="/linux-tutorial/assets/js/21.1c5a41d7.js"><link rel="prefetch" href="/linux-tutorial/assets/js/22.fbe9fdf1.js"><link rel="prefetch" href="/linux-tutorial/assets/js/23.a4fb0e74.js"><link rel="prefetch" href="/linux-tutorial/assets/js/24.e3a23b69.js"><link rel="prefetch" href="/linux-tutorial/assets/js/25.9896afe9.js"><link rel="prefetch" href="/linux-tutorial/assets/js/26.96164082.js"><link rel="prefetch" href="/linux-tutorial/assets/js/27.391033bb.js"><link rel="prefetch" href="/linux-tutorial/assets/js/28.703f74c2.js"><link rel="prefetch" href="/linux-tutorial/assets/js/29.02a952cb.js"><link rel="prefetch" href="/linux-tutorial/assets/js/30.7e13628f.js"><link rel="prefetch" href="/linux-tutorial/assets/js/31.c4652f75.js"><link rel="prefetch" href="/linux-tutorial/assets/js/32.05d2cbec.js"><link rel="prefetch" href="/linux-tutorial/assets/js/33.3b265df8.js"><link rel="prefetch" href="/linux-tutorial/assets/js/34.26330a03.js"><link rel="prefetch" href="/linux-tutorial/assets/js/35.417d706d.js"><link rel="prefetch" href="/linux-tutorial/assets/js/36.0ed775e0.js"><link rel="prefetch" href="/linux-tutorial/assets/js/37.34430c74.js"><link rel="prefetch" href="/linux-tutorial/assets/js/38.87d5e0ff.js"><link rel="prefetch" href="/linux-tutorial/assets/js/39.7b648b3e.js"><link rel="prefetch" href="/linux-tutorial/assets/js/40.3b7a219e.js"><link rel="prefetch" href="/linux-tutorial/assets/js/41.e727eee9.js"><link rel="prefetch" href="/linux-tutorial/assets/js/42.0134c187.js"><link rel="prefetch" href="/linux-tutorial/assets/js/43.175e982f.js"><link rel="prefetch" href="/linux-tutorial/assets/js/44.72d90888.js"><link rel="prefetch" href="/linux-tutorial/assets/js/45.d49955bd.js"><link rel="prefetch" href="/linux-tutorial/assets/js/46.a9c290ec.js"><link rel="prefetch" href="/linux-tutorial/assets/js/47.cc639f04.js"><link rel="prefetch" href="/linux-tutorial/assets/js/48.98c78321.js"><link rel="prefetch" href="/linux-tutorial/assets/js/49.a7c3afed.js"><link rel="prefetch" href="/linux-tutorial/assets/js/50.22d8c542.js"><link rel="prefetch" href="/linux-tutorial/assets/js/51.28055fcd.js"><link rel="prefetch" href="/linux-tutorial/assets/js/52.f8103df5.js"><link rel="prefetch" href="/linux-tutorial/assets/js/53.76541550.js"><link rel="prefetch" href="/linux-tutorial/assets/js/55.3ce3079c.js"><link rel="prefetch" href="/linux-tutorial/assets/js/56.832958c9.js"><link rel="prefetch" href="/linux-tutorial/assets/js/57.961ce896.js"><link rel="prefetch" href="/linux-tutorial/assets/js/58.6d6fbc82.js"><link rel="prefetch" href="/linux-tutorial/assets/js/59.d5e48112.js"><link rel="prefetch" href="/linux-tutorial/assets/js/6.c8f4721c.js"><link rel="prefetch" href="/linux-tutorial/assets/js/60.7927b23b.js"><link rel="prefetch" href="/linux-tutorial/assets/js/61.ee233f24.js"><link rel="prefetch" href="/linux-tutorial/assets/js/62.6ba50cc7.js"><link rel="prefetch" href="/linux-tutorial/assets/js/63.9cbf9f2b.js"><link rel="prefetch" href="/linux-tutorial/assets/js/64.0be148a4.js"><link rel="prefetch" href="/linux-tutorial/assets/js/65.c520257e.js"><link rel="prefetch" href="/linux-tutorial/assets/js/66.f2335390.js"><link rel="prefetch" href="/linux-tutorial/assets/js/67.e5737218.js"><link rel="prefetch" href="/linux-tutorial/assets/js/68.46427a01.js"><link rel="prefetch" href="/linux-tutorial/assets/js/69.450417bb.js"><link rel="prefetch" href="/linux-tutorial/assets/js/7.046e5a1b.js"><link rel="prefetch" href="/linux-tutorial/assets/js/70.072034d2.js"><link rel="prefetch" href="/linux-tutorial/assets/js/8.77fb8967.js"><link rel="prefetch" href="/linux-tutorial/assets/js/9.ebfa537e.js"><link rel="prefetch" href="/linux-tutorial/assets/js/vendors~flowchart.20a64d45.js"><link rel="prefetch" href="/linux-tutorial/assets/js/vendors~notification.ea176280.js">
<link rel="stylesheet" href="/linux-tutorial/assets/css/0.styles.45d9d031.css">
</head>
<body>
<div id="app" data-server-rendered="true"><div class="theme-container"><header class="navbar"><div class="sidebar-button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"><path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path></svg></div> <a href="/linux-tutorial/" class="home-link router-link-active"><img src="images/dunwu-logo-100.png" alt="LINUX-TUTORIAL" class="logo"> <span class="site-name can-hide">LINUX-TUTORIAL</span></a> <div class="links"><div class="search-box"><input aria-label="Search" autocomplete="off" spellcheck="false" value=""> <!----></div> <nav class="nav-links can-hide"><div class="nav-item"><a href="/linux-tutorial/linux/cli/" class="nav-link">
Linux 命令
</a></div><div class="nav-item"><a href="/linux-tutorial/linux/ops/" class="nav-link">
Linux 运维
</a></div><div class="nav-item"><a href="/linux-tutorial/linux/soft/" class="nav-link router-link-active">
Linux 软件运维
</a></div><div class="nav-item"><a href="/linux-tutorial/docker/" class="nav-link">
Docker 教程
</a></div><div class="nav-item"><a href="https://github.com/dunwu/blog" target="_blank" rel="noopener noreferrer" class="nav-link external">
🎯 博客
<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div> <a href="https://github.com/dunwu/linux-tutorial" target="_blank" rel="noopener noreferrer" class="repo-link">
Github
<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></nav></div></header> <div class="sidebar-mask"></div> <aside class="sidebar"><nav class="nav-links"><div class="nav-item"><a href="/linux-tutorial/linux/cli/" class="nav-link">
Linux 命令
</a></div><div class="nav-item"><a href="/linux-tutorial/linux/ops/" class="nav-link">
Linux 运维
</a></div><div class="nav-item"><a href="/linux-tutorial/linux/soft/" class="nav-link router-link-active">
Linux 软件运维
</a></div><div class="nav-item"><a href="/linux-tutorial/docker/" class="nav-link">
Docker 教程
</a></div><div class="nav-item"><a href="https://github.com/dunwu/blog" target="_blank" rel="noopener noreferrer" class="nav-link external">
🎯 博客
<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></div> <a href="https://github.com/dunwu/linux-tutorial" target="_blank" rel="noopener noreferrer" class="repo-link">
Github
<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></nav> <ul class="sidebar-links"><li><section class="sidebar-group depth-0"><p class="sidebar-heading open"><span>Elastic 技术栈之快速入门</span> <!----></p> <ul class="sidebar-links sidebar-group-items"><li><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#概念" class="sidebar-link">概念</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#elk-是什么" class="sidebar-link">ELK 是什么</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#为什么使用-elk" class="sidebar-link">为什么使用 ELK ？</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#elastic-架构" class="sidebar-link">Elastic 架构</a></li></ul></li><li><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#安装" class="sidebar-link">安装</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#准备" class="sidebar-link">准备</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#elasticsearch" class="sidebar-link">Elasticsearch</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#logstash" class="sidebar-link">Logstash</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#kibana" class="sidebar-link">Kibana</a></li><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#安装-faq" class="sidebar-link">安装 FAQ</a></li></ul></li><li><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#使用" class="sidebar-link">使用</a><ul class="sidebar-sub-headers"><li class="sidebar-sub-header"><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#java-应用输出日志到-elk" class="sidebar-link">Java 应用输出日志到 ELK</a></li></ul></li><li><a href="/linux-tutorial/linux/soft/elastic/elastic-quickstart.html#资料" class="sidebar-link">资料</a><ul class="sidebar-sub-headers"></ul></li></ul></section></li></ul> </aside> <main class="page"> <div class="theme-default-content content__default"><h1 id="elastic-技术栈之快速入门"><a href="#elastic-技术栈之快速入门" class="header-anchor">#</a> Elastic 技术栈之快速入门</h1> <h2 id="概念"><a href="#概念" class="header-anchor">#</a> 概念</h2> <h3 id="elk-是什么"><a href="#elk-是什么" class="header-anchor">#</a> ELK 是什么</h3> <p>ELK 是 elastic 公司旗下三款产品 <a href="https://www.elastic.co/products/elasticsearch" target="_blank" rel="noopener noreferrer">ElasticSearch<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 、<a href="https://www.elastic.co/products/logstash" target="_blank" rel="noopener noreferrer">Logstash<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 、<a href="https://www.elastic.co/products/kibana" target="_blank" rel="noopener noreferrer">Kibana<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 的首字母组合。</p> <p><a href="https://www.elastic.co/products/elasticsearch" target="_blank" rel="noopener noreferrer">ElasticSearch<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 是一个基于 <a href="http://lucene.apache.org/core/documentation.html" target="_blank" rel="noopener noreferrer">Lucene<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 构建的开源，分布式，RESTful 搜索引擎。</p> <p><a href="https://www.elastic.co/products/logstash" target="_blank" rel="noopener noreferrer">Logstash<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 传输和处理你的日志、事务或其他数据。</p> <p><a href="https://www.elastic.co/products/kibana" target="_blank" rel="noopener noreferrer">Kibana<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 将 Elasticsearch 的数据分析并渲染为可视化的报表。</p> <h3 id="为什么使用-elk"><a href="#为什么使用-elk" class="header-anchor">#</a> 为什么使用 ELK ？</h3> <p>对于有一定规模的公司来说，通常会很多个应用，并部署在大量的服务器上。运维和开发人员常常需要通过查看日志来定位问题。如果应用是集群化部署，试想如果登录一台台服务器去查看日志，是多么费时费力。</p> <p>而通过 ELK 这套解决方案，可以同时实现日志收集、日志搜索和日志分析的功能。</p> <h3 id="elastic-架构"><a href="#elastic-架构" class="header-anchor">#</a> Elastic 架构</h3> <p><img src="https://www.elastic.co/guide/en/logstash/current/static/images/deploy3.png" alt="img"></p> <blockquote><p><strong>说明</strong></p> <p>以上是 ELK 技术栈的一个架构图。从图中可以清楚的看到数据流向。</p> <p><a href="https://www.elastic.co/products/beats" target="_blank" rel="noopener noreferrer">Beats<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 是单一用途的数据传输平台，它可以将多台机器的数据发送到 Logstash 或 ElasticSearch。但 Beats 并不是不可或缺的一环，所以本文中暂不介绍。</p> <p><a href="https://www.elastic.co/products/logstash" target="_blank" rel="noopener noreferrer">Logstash<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 是一个动态数据收集管道。支持以 TCP/UDP/HTTP 多种方式收集数据（也可以接受 Beats 传输来的数据），并对数据做进一步丰富或提取字段处理。</p> <p><a href="https://www.elastic.co/products/elasticsearch" target="_blank" rel="noopener noreferrer">ElasticSearch<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 是一个基于 JSON 的分布式的搜索和分析引擎。作为 ELK 的核心，它集中存储数据。</p> <p><a href="https://www.elastic.co/products/kibana" target="_blank" rel="noopener noreferrer">Kibana<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a> 是 ELK 的用户界面。它将收集的数据进行可视化展示（各种报表、图形化数据），并提供配置、管理 ELK 的界面。</p></blockquote> <h2 id="安装"><a href="#安装" class="header-anchor">#</a> 安装</h2> <h3 id="准备"><a href="#准备" class="header-anchor">#</a> 准备</h3> <p>ELK 要求本地环境中安装了 JDK 。如果不确定是否已安装，可使用下面的命令检查：</p> <div class="language-bash extra-class"><pre class="language-bash"><code>java -version
</code></pre></div><blockquote><p><strong>注意</strong></p> <p>本文使用的 ELK 是 6.0.0，要求 jdk 版本不低于 JDK8。</p> <p>友情提示：安装 ELK 时，三个应用请选择统一的版本，避免出现一些莫名其妙的问题。例如：由于版本不统一，导致三个应用间的通讯异常。</p></blockquote> <h3 id="elasticsearch"><a href="#elasticsearch" class="header-anchor">#</a> Elasticsearch</h3> <p>安装步骤如下：</p> <ol><li><a href="https://www.elastic.co/downloads/elasticsearch" target="_blank" rel="noopener noreferrer">elasticsearch 官方下载地址<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>下载所需版本包并解压到本地。</li> <li>运行 <code>bin/elasticsearch</code> （Windows 上运行 <code>bin\elasticsearch.bat</code>）</li> <li>验证运行成功：linux 上可以执行 <code>curl http://localhost:9200/</code> ；windows 上可以用访问 REST 接口的方式来访问 http://localhost:9200/</li></ol> <blockquote><p><strong>说明</strong></p> <p>Linux 上可以执行下面的命令来下载压缩包：</p> <div class="language- extra-class"><pre class="language-text"><code>curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.0.0.tar.gz
</code></pre></div><p>Mac 上可以执行以下命令来进行安装：</p> <div class="language- extra-class"><pre class="language-text"><code>brew install elasticsearch
</code></pre></div><p>Windows 上可以选择 MSI 可执行安装程序，将应用安装到本地。</p></blockquote> <h3 id="logstash"><a href="#logstash" class="header-anchor">#</a> Logstash</h3> <p>安装步骤如下：</p> <ol><li><p>在 <a href="https://www.elastic.co/downloads/logstash" target="_blank" rel="noopener noreferrer">logstash 官方下载地址<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>下载所需版本包并解压到本地。</p></li> <li><p>添加一个 <code>logstash.conf</code> 文件，指定要使用的插件以及每个插件的设置。举个简单的例子：</p> <div class="language- extra-class"><pre class="language-text"><code>input { stdin { } }
output {
elasticsearch { hosts =&gt; [&quot;localhost:9200&quot;] }
stdout { codec =&gt; rubydebug }
}
</code></pre></div></li> <li><p>运行 <code>bin/logstash -f logstash.conf</code> （Windows 上运行<code>bin/logstash.bat -f logstash.conf</code>）</p></li></ol> <h3 id="kibana"><a href="#kibana" class="header-anchor">#</a> Kibana</h3> <p>安装步骤如下：</p> <ol><li>在 <a href="https://www.elastic.co/downloads/kibana" target="_blank" rel="noopener noreferrer">kibana 官方下载地址<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a>下载所需版本包并解压到本地。</li> <li>修改 <code>config/kibana.yml</code> 配置文件，设置 <code>elasticsearch.url</code> 指向 Elasticsearch 实例。</li> <li>运行 <code>bin/kibana</code> （Windows 上运行 <code>bin\kibana.bat</code>）</li> <li>在浏览器上访问 http://localhost:5601</li></ol> <h3 id="安装-faq"><a href="#安装-faq" class="header-anchor">#</a> 安装 FAQ</h3> <h4 id="elasticsearch-不允许以-root-权限来运行"><a href="#elasticsearch-不允许以-root-权限来运行" class="header-anchor">#</a> elasticsearch 不允许以 root 权限来运行</h4> <p>**问题：**在 Linux 环境中，elasticsearch 不允许以 root 权限来运行。</p> <p>如果以 root 身份运行 elasticsearch，会提示这样的错误：</p> <div class="language- extra-class"><pre class="language-text"><code>can not run elasticsearch as root
</code></pre></div><p>**解决方法：**使用非 root 权限账号运行 elasticsearch</p> <div class="language-bash extra-class"><pre class="language-bash"><code><span class="token comment"># 创建用户组</span>
<span class="token function">groupadd</span> elk
<span class="token comment"># 创建新用户，-g elk 设置其用户组为 elk，-p elk 设置其密码为 elk</span>
<span class="token function">useradd</span> elk -g elk -p elk
<span class="token comment"># 更改 /opt 文件夹及内部文件的所属用户及组为 elk:elk</span>
<span class="token function">chown</span> -R elk:elk /opt <span class="token comment"># 假设你的 elasticsearch 安装在 opt 目录下</span>
<span class="token comment"># 切换账号</span>
<span class="token function">su</span> elk
</code></pre></div><h4 id="vm-max-map-count-不低于-262144"><a href="#vm-max-map-count-不低于-262144" class="header-anchor">#</a> vm.max_map_count 不低于 262144</h4> <p><strong>问题：</strong><code>vm.max_map_count</code> 表示虚拟内存大小，它是一个内核参数。elasticsearch 默认要求 <code>vm.max_map_count</code> 不低于 262144。</p> <div class="language- extra-class"><pre class="language-text"><code>max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
</code></pre></div><p><strong>解决方法：</strong></p> <p>你可以执行以下命令，设置 <code>vm.max_map_count</code> ，但是重启后又会恢复为原值。</p> <div class="language- extra-class"><pre class="language-text"><code>sysctl -w vm.max_map_count=262144
</code></pre></div><p>持久性的做法是在 <code>/etc/sysctl.conf</code> 文件中修改 <code>vm.max_map_count</code> 参数：</p> <div class="language- extra-class"><pre class="language-text"><code>echo &quot;vm.max_map_count=262144&quot; &gt; /etc/sysctl.conf
sysctl -p
</code></pre></div><blockquote><p><strong>注意</strong></p> <p>如果运行环境为 docker 容器，可能会限制执行 sysctl 来修改内核参数。</p> <p>这种情况下，你只能选择直接修改宿主机上的参数了。</p></blockquote> <h4 id="nofile-不低于-65536"><a href="#nofile-不低于-65536" class="header-anchor">#</a> nofile 不低于 65536</h4> <p><strong>问题：</strong> <code>nofile</code> 表示进程允许打开的最大文件数。elasticsearch 进程要求可以打开的最大文件数不低于 65536。</p> <div class="language- extra-class"><pre class="language-text"><code>max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
</code></pre></div><p><strong>解决方法：</strong></p> <p>在 <code>/etc/security/limits.conf</code> 文件中修改 <code>nofile</code> 参数：</p> <div class="language- extra-class"><pre class="language-text"><code>echo &quot;* soft nofile 65536&quot; &gt; /etc/security/limits.conf
echo &quot;* hard nofile 131072&quot; &gt; /etc/security/limits.conf
</code></pre></div><h4 id="nproc-不低于-2048"><a href="#nproc-不低于-2048" class="header-anchor">#</a> nproc 不低于 2048</h4> <p><strong>问题：</strong> <code>nproc</code> 表示最大线程数。elasticsearch 要求最大线程数不低于 2048。</p> <div class="language- extra-class"><pre class="language-text"><code>max number of threads [1024] for user [user] is too low, increase to at least [2048]
</code></pre></div><p><strong>解决方法：</strong></p> <p>在 <code>/etc/security/limits.conf</code> 文件中修改 <code>nproc</code> 参数：</p> <div class="language- extra-class"><pre class="language-text"><code>echo &quot;* soft nproc 2048&quot; &gt; /etc/security/limits.conf
echo &quot;* hard nproc 4096&quot; &gt; /etc/security/limits.conf
</code></pre></div><h4 id="kibana-no-default-index-pattern-warning"><a href="#kibana-no-default-index-pattern-warning" class="header-anchor">#</a> Kibana No Default Index Pattern Warning</h4> <p>**问题：**安装 ELK 后，访问 kibana 页面时，提示以下错误信息：</p> <div class="language- extra-class"><pre class="language-text"><code>Warning No default index pattern. You must select or create one to continue.
...
Unable to fetch mapping. Do you have indices matching the pattern?
</code></pre></div><p>这就说明 logstash 没有把日志写入到 elasticsearch。</p> <p><strong>解决方法：</strong></p> <p>检查 logstash 与 elasticsearch 之间的通讯是否有问题，一般问题就出在这。</p> <h2 id="使用"><a href="#使用" class="header-anchor">#</a> 使用</h2> <p>本人使用的 Java 日志方案为 slf4j + logback，所以这里以 logback 来讲解。</p> <h3 id="java-应用输出日志到-elk"><a href="#java-应用输出日志到-elk" class="header-anchor">#</a> Java 应用输出日志到 ELK</h3> <p><strong>修改 logstash.conf 配置</strong></p> <p>首先，我们需要修改一下 logstash 服务端 logstash.conf 中的配置</p> <div class="language- extra-class"><pre class="language-text"><code>input {
# stdin { }
tcp {
# host:port就是上面appender中的 destination，
# 这里其实把logstash作为服务，开启9250端口接收logback发出的消息
host =&gt; &quot;127.0.0.1&quot; port =&gt; 9250 mode =&gt; &quot;server&quot; tags =&gt; [&quot;tags&quot;] codec =&gt; json_lines
}
}
output {
elasticsearch { hosts =&gt; [&quot;localhost:9200&quot;] }
stdout { codec =&gt; rubydebug }
}
</code></pre></div><blockquote><p><strong>说明</strong></p> <p>这个 input 中的配置其实是 logstash 服务端监听 9250 端口，接收传递来的日志数据。</p></blockquote> <p>然后，在 Java 应用的 pom.xml 中引入 jar 包：</p> <div class="language-xml extra-class"><pre class="language-xml"><code><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>dependency</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>groupId</span><span class="token punctuation">&gt;</span></span>net.logstash.logback<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>groupId</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>artifactId</span><span class="token punctuation">&gt;</span></span>logstash-logback-encoder<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>artifactId</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>version</span><span class="token punctuation">&gt;</span></span>4.11<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>version</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>dependency</span><span class="token punctuation">&gt;</span></span>
</code></pre></div><p>接着，在 logback.xml 中添加 appender</p> <div class="language-xml extra-class"><pre class="language-xml"><code><span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>appender</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>LOGSTASH<span class="token punctuation">&quot;</span></span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>net.logstash.logback.appender.LogstashTcpSocketAppender<span class="token punctuation">&quot;</span></span><span class="token punctuation">&gt;</span></span>
<span class="token comment">&lt;!--
destination 是 logstash 服务的 host:port，
相当于和 logstash 建立了管道，将日志数据定向传输到 logstash
--&gt;</span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>destination</span><span class="token punctuation">&gt;</span></span>127.0.0.1:9250<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>destination</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>encoder</span> <span class="token attr-name">charset</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>UTF-8<span class="token punctuation">&quot;</span></span> <span class="token attr-name">class</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>net.logstash.logback.encoder.LogstashEncoder<span class="token punctuation">&quot;</span></span><span class="token punctuation">/&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>appender</span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>logger</span> <span class="token attr-name">name</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>io.github.dunwu.spring<span class="token punctuation">&quot;</span></span> <span class="token attr-name">level</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>TRACE<span class="token punctuation">&quot;</span></span> <span class="token attr-name">additivity</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>false<span class="token punctuation">&quot;</span></span><span class="token punctuation">&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;</span>appender-ref</span> <span class="token attr-name">ref</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">&quot;</span>LOGSTASH<span class="token punctuation">&quot;</span></span> <span class="token punctuation">/&gt;</span></span>
<span class="token tag"><span class="token tag"><span class="token punctuation">&lt;/</span>logger</span><span class="token punctuation">&gt;</span></span>
</code></pre></div><p>大功告成，此后，<code>io.github.dunwu.spring</code> 包中的 TRACE 及以上级别的日志信息都会被定向输出到 logstash 服务。</p> <p><img src="http://upload-images.jianshu.io/upload_images/3101171-cd876d79a14955b0.png" alt="img"></p> <h2 id="资料"><a href="#资料" class="header-anchor">#</a> 资料</h2> <ul><li><p><a href="https://www.elastic.co/guide/index.html" target="_blank" rel="noopener noreferrer">elastic 官方文档<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></p></li> <li><p><a href="https://github.com/elastic/elasticsearch" target="_blank" rel="noopener noreferrer">elasticsearch github<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></p></li> <li><p><a href="https://github.com/elastic/logstash" target="_blank" rel="noopener noreferrer">logstash github<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></p></li> <li><p><a href="https://github.com/elastic/kibana" target="_blank" rel="noopener noreferrer">kibana github<span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></a></p></li></ul></div> <footer class="page-edit"><div class="edit-link"><a href="https://github.com/dunwu/linux-tutorial/edit/master/docs/linux/soft/elastic/elastic-quickstart.md" target="_blank" rel="noopener noreferrer">帮助我们改善此页面！</a> <span><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" focusable="false" x="0px" y="0px" viewBox="0 0 100 100" width="15" height="15" class="icon outbound"><path fill="currentColor" d="M18.8,85.1h56l0,0c2.2,0,4-1.8,4-4v-32h-8v28h-48v-48h28v-8h-32l0,0c-2.2,0-4,1.8-4,4v56C14.8,83.3,16.6,85.1,18.8,85.1z"></path> <polygon fill="currentColor" points="45.7,48.7 51.3,54.3 77.2,28.5 77.2,37.2 85.2,37.2 85.2,14.9 62.8,14.9 62.8,22.9 71.5,22.9"></polygon></svg> <span class="sr-only">(opens new window)</span></span></div> <div class="last-updated"><span class="prefix">上次更新:</span> <span class="time">a year ago</span></div></footer> <!----> </main></div><div class="global-ui"><!----><!----></div></div>
<script src="/linux-tutorial/assets/js/app.79a38eea.js" defer></script><script src="/linux-tutorial/assets/js/4.fb6e0f89.js" defer></script><script src="/linux-tutorial/assets/js/54.e78d2776.js" defer></script><script src="/linux-tutorial/assets/js/5.cb43ecfb.js" defer></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink