在Windows中使用Docker搭建大模型RAG

本文将使用Windows环境来构建基于Qwen2-7b大模型的小型本地知识库。涉及到的内容包括：

Windows环境的Docker部署；
Qwen2-7b大模型的获取；
基于Ollama的大模型部署；
Open-webui的前台展示

2024.07 摄于广州黄埔·鱼珠码头 909功勋试验舰

系统准备

部署WSL

需要先启用“适用于 Linux 的 Windows 子系统”可选功能，然后才能在 Windows 上安装 Linux 分发。

以管理员身份打开 PowerShell（“开始”菜单 >“PowerShell” >单击右键 >“以管理员身份运行”），然后输入以下命令：

# 启用WSL
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
# 安装虚拟化
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
# 下载内核包并安装
https://wslstorestorage.blob.core.windows.net/wslblob/wsl_update_x64.msi

# 重启主机
Restart-Computer

# 升级wsl，否则会有"WslRegisterDistribution failed with error: 0x800701bc"报错
wsl --update
# 将wsl版本默认为2
wsl --set-default-version 2

# 安装默认Ubuntu
wsl --install

配置Ubuntu

# 禁用开机提示
touch .hushlogin

# 系统更新
sudo apt update -y
sudo apt install -y vim htop iotop lnav
sudo apt install -y wget curl git

显卡信息

sujx@alienware:~$ nvidia-smi
Wed Aug 21 15:56:03 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.73.01              Driver Version: 552.12         CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4050 ...    On  |   00000000:01:00.0  On |                  N/A |
| N/A   49C    P8              5W /   45W |     832MiB /   6141MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

安装Docker

下载安装文件
安装完成
配置代理

配置CUDA

大模型部署

下载模型

# 通过构建魔搭下载器来获取大模型
docker pull python:3.10-slim
docker run --rm -it -v `pwd`:/models python:3.10-slim bash
sed -i 's/snapshot.debian.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list.d/debian.sources
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
cd /models
pip install modelscope

# 1.5B下载
modelscope download --model=qwen/Qwen2-1.5B-Instruct-GGUF --local_dir . qwen2-1_5b-instruct-q5_k_m.gguf
# 7B下载
modelscope download --model=qwen/Qwen2-7B-Instruct-GGUF --local_dir . qwen2-7b-instruct-q5_k_m.gguf

Ollama部署

docker pull ollama/ollama:0.3.6

# 默认 CPU 模式运行
# docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

# Nvidia GPU 模式运行
docker run -d --gpus=all -v .\ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:0.3.6

# 进入Docker
docker exec -it ollama bash
mkdir ollama
cd ollama/
mv ../.ollama/qwen2-7b-instruct-q5_k_m.gguf ./
echo "FROM ./qwen2-7b-instruct-q5_k_m.gguf" > Modelfile
cd
ollama create custom_qwen2-7b -f ~/.ollama/Modelfile

导入

# 模型转换
root@055b6d9444a9:~/.ollama# ollama create custom_qwen2-7b -f ~/.ollama/Modelfile
transferring model data 100%
using existing layer sha256:7aeb11bb9b36d47625bd61514a3ac085826537144b375bb1d66b42738127e425
using autodetected template chatml
using existing layer sha256:f02dd72bb2423204352eabc5637b44d79d17f109fdb510a7c51455892aa2d216
using existing layer sha256:78a522a17dc3f34b94876c309ad592f7ff4ff1e029777eeac93774d69dd11bc9
writing manifest
success

# 查看模型信息
root@055b6d9444a9:~# ollama show custom_qwen2-7b
  Model
        arch                    qwen2
        parameters              7.6B
        quantization            Q5_K_M
        context length          32768
        embedding length        3584

# 转换完成的模型       
root@055b6d9444a9:~# du -hs ~/.ollama/models/
5.1G    /root/.ollama/models/

演示

docker pull soulteary/ollama:0.3.0-fix
docker run --gpus all -d -v .\ollama:/root/.ollama -p 11434:11434 --name qwen2 soulteary/ollama:0.3.0-fix 

sujx@alienware:~$ curl http://192.168.1.158:11434/api/generate -d '{
  "model": "custom_qwen2-7b",
  "prompt":"天空为什么那么蓝?"
}'
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.363290951Z","response":"天空","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.422390576Z","response":"之所以","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.482391648Z","response":"呈现","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.542272685Z","response":"蓝色","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.603602263Z","response":"，","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.664370661Z","response":"主要是","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.721054611Z","response":"由于","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.779412138Z","response":"大气","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.8385991Z","response":"中的","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.896342916Z","response":"气体","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:41.95431912Z","response":"和","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.010744209Z","response":"微","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.068093671Z","response":"小","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.12515422Z","response":"颗粒","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.182568345Z","response":"对","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.239366307Z","response":"太阳","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.297282361Z","response":"光","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.353023896Z","response":"的","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.410579873Z","response":"散","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.46952615Z","response":"射","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.525942839Z","response":"作用","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.583992362Z","response":"。","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.64235722Z","response":"太阳","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.701759517Z","response":"光","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.759330313Z","response":"是由","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.817788108Z","response":"各种","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.874783492Z","response":"不同","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.935173302Z","response":"波","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:42.992110059Z","response":"长","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.052014607Z","response":"的","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.112184454Z","response":"光线","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.170437415Z","response":"组成的","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.230450769Z","response":"复合","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.288458072Z","response":"光","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.347953774Z","response":"，","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.407144815Z","response":"包含了","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.466536397Z","response":"从","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.524233057Z","response":"紫","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.582222113Z","response":"到","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.63959678Z","response":"红","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.697854731Z","response":"的各种","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.755939647Z","response":"颜色","done":false}
{"model":"custom_qwen2-7b","created_at":"2024-08-21T18:09:43.813553529Z","response":"。","done":false}

前端部署

MaxKB

MaxKB是国内开源的一款基于 LLM 大语言模型的开源知识库问答系统，旨在成为企业的最强大脑。

启动

# 通过Docker拉取镜像
docker pull cr2.fit2cloud.com/1panel/maxkb
# 拉起服务
docker run -d --name=Maxkb -p 8080:8080 -v ~/.maxkb:/var/lib/postgresql/data cr2.fit2cloud.com/1panel/maxkb

# 默认登录信息
http://127.0.0.1:8080
用户名：admin
默认密码：MaxKB@123..

应用

添加应用

配置模型

1
2
3

这里需要注意两点：
1. 添加大模型的地址不能使用127.0.0.1或者localhost，可以用实际主机获取地址
2. 由于我们使用的是自定义过的大模型，需要配置自定义模型名称

测试模型
另外问题测试
后台管理
添加知识库
导入Ansible面试题
关联应用和知识库
自我展示

Open-webUI

open-webUI 是另一个著名的LLM前端项目，它是一个类似chatGPT风格的、自托管的大模型前端项目，可以对接Ollama这样的对话大模型，也可以对接SD这样的文生图大模型、TTS-OpenedAI这样的文生语音大模型。

启动

# 拉取镜像
docker pull ghcr.io/open-webui/open-webui:main
# 拉起服务
docker run -d -p 3000:8080 -e OLLAMA_BASE_URL=http://172.16.10.110:11434 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main