Hi,
我在 AWS 開了一台 EC2 架設 ELK8.9,
OS為 Amazon linux2,
使用 YUM 的方式架設,
以下為安裝連結。
https://www.elastic.co/guide/en/elasticsearch/reference/8.9/rpm.html
到這邊檢查curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
都沒有問題,
問題是,我在我的電腦上,
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://$EC2-IP:9200 -k
設定這個居然能夠回我
{
"name" : "Cp8oag6",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
"version" : {
"number" : "8.9.2",
"build_type" : "tar",
"build_hash" : "f27399d",
"build_flavor" : "default",
"build_date" : "2016-03-30T09:51:41.449Z",
"build_snapshot" : false,
"lucene_version" : "9.7.0",
"minimum_wire_compatibility_version" : "1.2.3",
"minimum_index_compatibility_version" : "1.2.3"
},
"tagline" : "You Know, for Search"
}
即使我在 /etc/elasticsearch.yml
有以下設定
network.host=localhost
# http.host=0.0.0.0
發現我的電腦還是可以去戳,我覺得這不是安全的,
我想正常情況是,ES只能透過內網去訪問他,而不能讓我這台不在VPC的外網電腦能夠訪問。
不知道還有甚麼參數是我沒注意到的。
補充 /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 12-09-2023 09:29:04
#
# --------------------------------------------------------------------------------
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ip-172-31-27-25.ap-northeast-1.compute.internal"]
# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: _site_
# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0
測試另外一台同VPC的VM
[root@ip-172-31-29-89 conf.d]# curl -k -u elastic:$我是密碼 https://172.31.27.25:9200
{
"name" : "ip-172-31-27-25.ap-northeast-1.compute.internal",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "BfwzZ50BQ7CCJ09NUCKgiw",
"version" : {
"number" : "8.10.0",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "e338da74c79465dfdc204971e600342b0aa87b6b",
"build_date" : "2023-09-07T08:16:21.960703010Z",
"build_snapshot" : false,
"lucene_version" : "9.7.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
電腦測試
curl -k -u elastic:$我是密碼 https://$我是Server外網IP:9200
{
"name" : "ip-172-31-27-25.ap-northeast-1.compute.internal",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "BfwzZ50BQ7CCJ09NUCKgiw",
"version" : {
"number" : "8.10.0",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "e338da74c79465dfdc204971e600342b0aa87b6b",
"build_date" : "2023-09-07T08:16:21.960703010Z",
"build_snapshot" : false,
"lucene_version" : "9.7.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
Elasticsearch 有它自己的一些專用變數, 可以用在 YAML 設定中:
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html
例如:
_local_
專指本機上的 loopback 位址, 例如: 127.0.0.1
_site_
專指本機上的本地網段, 例如: 192.168.0.1 (這樣萬一你的 VM 換內網 IP 就不用重新設定)
_global_
任何一個公開的 Internet 位址
_[networkInterface]_
綁定特定網卡的網址, 如: en0
0.0.0.0
專指任何一個 Internet 位址
感謝雷神
那個網址我有翻到,只是爬網路大部分範例都是
0.0.0.0(全開) or localhost,ELK 那些參數我再設定看看。
而連結的最上面也有寫到說,就是設定 network.* 跟 httpd.*,
By default Elasticsearch binds only to localhost which means it cannot be accessed remotely
而 /etc/elasticsearch/elasticsearch.yml 原本註解的設定
# network.host=localhost
我才想說綁 localhost 就好了
127.0.0.1是專指本機,你要在內網用,假設內網網段是192.168.0.X,你只綁定在127.0.0.1要怎麼跟你內網網段溝通?
而且內網其他電腦要存取你的ELK的時候,打的網址也是127.0.0.1,會指向client自己。
localhost本質上也只是個域名,雲端因為架構的問題,你不會知道localhost背後指向的是不是就是127.0.0.1。
localhost本質上也只是個域名,雲端因為架構的問題,你不會知道localhost背後指向的是不是就是127.0.0.1。
雲端架構上跟從virtualbox有什麼不同嘛?
[root@ip-172-31-27-25 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost6 localhost6.localdomain6
[root@ip-172-31-27-25 ~]# ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=255 time=0.024 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=255 time=0.034 ms
^C
--- localhost ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1013ms
rtt min/avg/max/mdev = 0.024/0.029/0.034/0.005 ms
看起來 localhost 也是指向 127.0.0.1,
我有修改 http.host: site 測試上外網還是碰的到。
[root@ip-172-31-27-25 ~]# cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 12-09-2023 09:29:04
#
# --------------------------------------------------------------------------------
# Enable security features
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
enabled: true
keystore.path: certs/http.p12
# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
enabled: true
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ip-172-31-27-25.ap-northeast-1.compute.internal"]
# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: _site_
# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0
#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
由於篇幅過長,我將這些設定檔案,放在原本問題區域,比較好閱讀。
我後來解開這個問題了,給兩位大大參考,
http.host: _site_
肯定是不行的,因為走 eth0會導致外網IP也能存取。我的解法。
手動新增 ENI,將 EC2 綁定上去,手動新增的 ENI 不會有外部IP在上面。
然後http.host: _eth1_
or http.host: _eth1的ip_
這時候外部網路就進不了了。
附圖實驗結果, ip 跟 密碼就不遮了,這台是測試機,全部都會砍掉。