使用Rclone进行文件备份 | Word Count: 966 | Reading Time: 4mins | Post Views:
环境概述
本地环境为由Beijing、shanghai、guangzhou三台Linux主机构成的数据备份源；
远端备份目的为本地自建S3存储和紫光云存储；
使用rclone将本地多主机文件备份到远端存储；
具体工具有：rclone、minio和sftp。
主机地址
主机名
角色
IP地址
Rclone
同步工具
192.168.174.150
Minio
开源S3服务器
192.168.174.160
UNicloud
公网S3服务器
*
Beijing
备份站点
192.168.174.141
Shanghai
备份站点
192.168.174.142
Guangzhou
备份站点
192.168.174.143
Minio的建立 Mnino是一个轻巧、快速的开源S3服务器，其官方主页为：minio https://min.io/
# 下载二进制执行文件 wget https: # 添加可执行权限 chmod +x minio # 复制到指定路径 cp minio /usr/local/bin/ # 建立程序运行账户 useradd -r minio -s /sbin/nologin # 对程序属主进行配置 chown minio:minio /usr/local/bin/minio # 配置程序启动参数 touch /etc/default /minio cat>/etc/default /minio<<EOF MINIO_VOLUMES="/usr/local/share/minio/" MINIO_OPTS="-C /etc/minio --address 192.168.174.160:9000" MINIO_ACCESS_KEY="minio" MINIO_SECRET_KEY="miniostorage" EOF # 配置systemctl管理程序 touch minio.service cat >> minio.service <<EOF [Unit] Description=MinIO Documentation=https: Wants=network-online.target After=network-online.target AssertFileIsExecutable=/usr/local/bin/minio [Service] WorkingDirectory=/usr/local/ User=minio Group=minio EnvironmentFile=/etc/default /minio ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi" ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES # Let systemd restart this service always Restart=always # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=65536 # Disable timeout logic and wait until process is stopped TimeoutStopSec=infinity SendSIGKILL=no [Install] WantedBy=multi-user.target # Built for ${project.name}-${project.version} (${project.name}) EOF # 复制配置文件到指定位置 cp minio.service /etc/systemd/system systemctl daemon-reload systemctl enable minio.service --now # 配置防火墙 firewall-cmd --permanent --add-service={ssh,http,https} firewall-cmd --permanent --add-port=9000 /tcp firewall-cmd --reload
建立桶存储并上传文件 [root@minio ~]# ls /usr/local/share/minio/ test [root@minio ~]# ls /usr/local/share/minio/test/ This is MinIO.mp4
接下来，我们可以使用官方客户端mc来访问我们建立的桶：
wget http://dl.minio.org.cn/client/mc/release/linux-amd64/mc chmod +x mc ./mc config host add minio http://192.168.174.160:9000 minio miniostroage [root@minio ~]# ./mc ls minio [2020-06-01 23:50:56 CST] 0B test/ [root@minio ~]# ./mc ls minio/test/ [2020-06-01 23:51:01 CST] 8.7MiB This is MinIO.mp4
Rclone部署 rclone的部署很简单，从官方网站下载安装rclone包即可。
wget https://downloads.rclone.org/v1.52.0/rclone-v1.52.0-linux-amd64.rpm yum localinstall -y rclone-v1.52.0-linux-amd64.rpm
然后，使用“rclone config”进行配置即可。配置完成的配置文件如下：
[minio] type = s3 provider = Minio env_auth = false access_key_id = minio secret_access_key = miniostorage region = us-east-1 endpoint = http://192.168.174.160:9000 [unicloud] type = s3 provider = Other env_auth = false access_key_id = __ secret_access_key = ______ endpoint = ______.unicloudsrv.com acl = private
接下来，我们进行测试：
[root@rclone ~]# rclone lsd minio: -1 2020-06-01 23:50:56 -1 test [root@rclone ~]# rclone ls minio:test 9117336 This is MinIO.mp4 [root@rclone ~]# rclone copy rclone-v1.52.0-linux-amd64.rpm minio:test [root@rclone ~]# rclone ls minio:test 9117336 This is MinIO.mp4 13825921 rclone-v1.52.0-linux-amd64.rpm
备份其他上传
各个服务器需要开启时间同步服务并配置rclone主机同各个备份主机的ssh免密登录；
使用rclone config 建立sftp类型的配置文件；
测试[root@beijing opt]# touch beijing/192.168.174.141 [root@beijing opt]# hostname >> ./beijing/192.168.174.141 [root@rclone ~]# rclone copy beijing:/opt/beijing minio:/test [root@rclone ~]# rclone ls minio:/test/ 9 192.168.174.141 9117336 This is MinIO.mp4 13825921 rclone-v1.52.0-linux-amd64.rpm # 在公有云新建桶 [root@rclone ~]# rclone mkdir unicloud:sujx [root@rclone ~]# rclone lsd unicloud: -1 2020-05-27 14:16:27 -1 backup-ucd-public-prod -1 2020-05-27 14:16:22 -1 backup-ucd-public-test -1 2020-05-20 09:20:30 -1 crmbak -1 2020-05-25 16:36:46 -1 fs-pcf -1 2020-05-25 16:36:35 -1 fs-pcf-temp -1 2020-06-01 19:28:14 -1 sujx # 在本地S3和远端S3 同步 [root@rclone ~]# rclone sync minio:test unicloud:sujx [root@rclone ~]# rclone ls unicloud:sujx 9 192.168.174.141 9117336 This is MinIO.mp4 13825921 rclone-v1.52.0-linux-amd64.rpm
