定时删除Wazuh日志索引

前情

生产和办公环境部署的Wazuh系统，因为对接的大量的Agent，每日的数据量在1.2GB左右。为减少实际运行成本并根据实际使用情况，原始日志导入一份到日志服务器外，Wazuh的安全日志只保留一个自然月。

脚本

使用Shell脚本来实现定时删除Wazuh的ElasticSearch索引的功能：

#!/bin/bash
#本脚本用于按月份清理ES存储数据
#获取本月份与索引中的日期进行比较，本月份前的索引数据将被删除。
#删除前要确认

ES_URL=http://127.0.0.1:9200
NOW_Year=`date +%Y` 
NOW_Mon=`date +%m`
ALLLINES=`/usr/bin/curl -s -XGET $ES_URL/_cat/indices?v| grep wazuh-alerts-4.x-*`
 
echo
echo "THIS IS WHAT SHOULD BE DELETED FOR ES:"
echo
 
echo "${ALLLINES}" | while read line
do
  # echo ${line}
  FORMATEDLINE=`echo ${line} |awk '{ print $3 }' | awk -F'-' '{ print $4 }' | cut -c 6-7 `
  # echo ${FORMATEDLINE}
  if [ "${FORMATEDLINE}" != "${NOW_Mon}" ]
  then
    echo "$ES_URL/wazuh-alerts-4.x-$NOW_Year.$FORMATEDLINE.*"
  fi
done
 
echo
echo -n "if this make sence, Y to continue N to exit [Y/N]:"
read INPUT
if [ "${INPUT}" == "Y" ] || [ "${INPUT}" == "y" ] || [ "${INPUT}" == "yes" ] || [ "${INPUT}" == "YES" ]
then
  echo "${ALLLINES}" | while read line
  do
    FORMATEDLINE=`echo ${line} |awk '{ print $3 }' | awk -F'-' '{ print $4 }' | cut -c 6-7 `
    if [ "${FORMATEDLINE}" != "${NOW_Mon}" ]
    then
      /usr/bin/curl -XDELETE "$ES_URL/wazuh-alerts-4.x-$NOW_Year.$FORMATEDLINE.*"
      sleep 1
      fi
  done
else
  echo SCRIPT CLOSED BY USER, BYE ...
  exit
fi

使用Crontab来实现定时删除指定数据：

#!/bin/bash
#本脚本用于定时按月份清理Wazuh存储数据
#获取本月份与索引中的日期进行比较，本月份前的索引数据将被删除。
# crontab参数为： 50 23 28-31 * * /opt/file/del_es_data.sh

ES_URL=http://127.0.0.1:9200
NOW_Year=`date +%Y`
NOW_Mon=`date +%m`
ALLLINES=`/usr/bin/curl -s -XGET $ES_URL/_cat/indices?v| grep wazuh-alerts-4.x-*`

echo "${ALLLINES}" | while read line
do
  FORMATEDLINE=`echo ${line} |awk '{ print $3 }' | awk -F'-' '{ print $4 }' | cut -c 6-7 `
  if [ "${FORMATEDLINE}" != "${NOW_Mon}" ]
  then
    /usr/bin/curl -XDELETE "$ES_URL/wazuh-alerts-4.x-$NOW_Year.$FORMATEDLINE.*"
  fi
done