iT邦幫忙

0

在 AWS 架 ELK8.9 ,ES外部能夠訪問,希望能夠禁止

  • 分享至 

  • xImage

Hi,
我在 AWS 開了一台 EC2 架設 ELK8.9,
OS為 Amazon linux2,
使用 YUM 的方式架設,
以下為安裝連結。
https://www.elastic.co/guide/en/elasticsearch/reference/8.9/rpm.html

到這邊檢查
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200

都沒有問題,
問題是,我在我的電腦上,
curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://$EC2-IP:9200 -k
設定這個居然能夠回我

{
  "name" : "Cp8oag6",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "AT69_T_DTp-1qgIJlatQqA",
  "version" : {
    "number" : "8.9.2",
    "build_type" : "tar",
    "build_hash" : "f27399d",
    "build_flavor" : "default",
    "build_date" : "2016-03-30T09:51:41.449Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "1.2.3",
    "minimum_index_compatibility_version" : "1.2.3"
  },
  "tagline" : "You Know, for Search"
}

即使我在 /etc/elasticsearch.yml
有以下設定

network.host=localhost
# http.host=0.0.0.0

發現我的電腦還是可以去戳,我覺得這不是安全的,
我想正常情況是,ES只能透過內網去訪問他,而不能讓我這台不在VPC的外網電腦能夠訪問。
不知道還有甚麼參數是我沒注意到的。

補充 /etc/elasticsearch/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 12-09-2023 09:29:04
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ip-172-31-27-25.ap-northeast-1.compute.internal"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: _site_

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

測試另外一台同VPC的VM

[root@ip-172-31-29-89 conf.d]# curl -k -u elastic:$我是密碼 https://172.31.27.25:9200
{
  "name" : "ip-172-31-27-25.ap-northeast-1.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "BfwzZ50BQ7CCJ09NUCKgiw",
  "version" : {
    "number" : "8.10.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "e338da74c79465dfdc204971e600342b0aa87b6b",
    "build_date" : "2023-09-07T08:16:21.960703010Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

電腦測試

 curl -k -u elastic:$我是密碼 https://$我是Server外網IP:9200
{
  "name" : "ip-172-31-27-25.ap-northeast-1.compute.internal",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "BfwzZ50BQ7CCJ09NUCKgiw",
  "version" : {
    "number" : "8.10.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "e338da74c79465dfdc204971e600342b0aa87b6b",
    "build_date" : "2023-09-07T08:16:21.960703010Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

1 個回答

0
Ray
iT邦大神 1 級 ‧ 2023-09-12 22:10:16
最佳解答
  1. 你的 EC2 沒有綁任何 AWS 的安全群組 (Security Group) 嗎? 有綁的話, AWS Security Group 裡面沒開 9200 port 的連入規則, 外界誰也連不進去的.
  2. 請不要用 localhost 這個名稱來指定本機, 因為有時雲主機的 localhost 名稱其實並不是被解析到 127.0.0.1. 你不如直接寫 IP 127.0.0.1 還比較保險.
  3. 要給內網使用, 也不該把 Network host 綁在 127.0.0.1 上, 因為這會連內網都無法使用.

Elasticsearch 有它自己的一些專用變數, 可以用在 YAML 設定中:
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html
例如:
_local_
專指本機上的 loopback 位址, 例如: 127.0.0.1
_site_
專指本機上的本地網段, 例如: 192.168.0.1 (這樣萬一你的 VM 換內網 IP 就不用重新設定)
_global_
任何一個公開的 Internet 位址
_[networkInterface]_
綁定特定網卡的網址, 如: en0
0.0.0.0
專指任何一個 Internet 位址

看更多先前的回應...收起先前的回應...

感謝雷神

  1. 因為在練習,所以 SG 設定全開,專注在配置 ELK 系統本身。
    1. 可以細講一點嗎,對 lookback ip,沒那麼懂,我不明白會阻塞的原因,

那個網址我有翻到,只是爬網路大部分範例都是
0.0.0.0(全開) or localhost,ELK 那些參數我再設定看看。

而連結的最上面也有寫到說,就是設定 network.* 跟 httpd.*,

By default Elasticsearch binds only to localhost which means it cannot be accessed remotely

而 /etc/elasticsearch/elasticsearch.yml 原本註解的設定

# network.host=localhost 

我才想說綁 localhost 就好了

froce iT邦大師 1 級 ‧ 2023-09-13 08:13:49 檢舉

127.0.0.1是專指本機,你要在內網用,假設內網網段是192.168.0.X,你只綁定在127.0.0.1要怎麼跟你內網網段溝通?
而且內網其他電腦要存取你的ELK的時候,打的網址也是127.0.0.1,會指向client自己。

localhost本質上也只是個域名,雲端因為架構的問題,你不會知道localhost背後指向的是不是就是127.0.0.1。

localhost本質上也只是個域名,雲端因為架構的問題,你不會知道localhost背後指向的是不是就是127.0.0.1。

雲端架構上跟從virtualbox有什麼不同嘛?

[root@ip-172-31-27-25 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost6 localhost6.localdomain6
[root@ip-172-31-27-25 ~]# ping localhost
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=255 time=0.024 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=255 time=0.034 ms
^C
--- localhost ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1013ms
rtt min/avg/max/mdev = 0.024/0.029/0.034/0.005 ms

看起來 localhost 也是指向 127.0.0.1,
我有修改 http.host: site 測試上外網還是碰的到。

[root@ip-172-31-27-25 ~]# cat /etc/elasticsearch/elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 12-09-2023 09:29:04
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["ip-172-31-27-25.ap-northeast-1.compute.internal"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: _site_

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------

由於篇幅過長,我將這些設定檔案,放在原本問題區域,比較好閱讀。

我後來解開這個問題了,給兩位大大參考,

  1. 限制 SG 確實是最快的解法,如果給其他人維護大家也直觀能想到 SG
  2. 由於 AWS 網路是由 ENI 再到 EC2 的,如果開一台機器不做任何更動,ip a 網卡會有兩張,一張是 lookback,一張是 eth0 上面有內網 ip,這時候的網路走法是 internet -> ENI -> ec2(eth0)。
    但是 eth0 同時也有負責外網的部分,如果設定
    http.host: _site_ 肯定是不行的,因為走 eth0會導致外網IP也能存取。

我的解法。
手動新增 ENI,將 EC2 綁定上去,手動新增的 ENI 不會有外部IP在上面。
然後http.host: _eth1_ or http.host: _eth1的ip_
這時候外部網路就進不了了。

附圖實驗結果, ip 跟 密碼就不遮了,這台是測試機,全部都會砍掉。
https://ithelp.ithome.com.tw/upload/images/20230914/20160038x3Sc9QdjaQ.png

Ray iT邦大神 1 級 ‧ 2023-09-14 17:02:55 檢舉

/images/emoticon/emoticon42.gif

我要發表回答

立即登入回答