安装满血版本的deepseek蒸馏模型+AnythingLLM+Milvus

1)install docker 服务

sudo apt update
sudo apt upgrade -y
sudo apt install -y ca-certificates curl gnupg

为了确保下载的 Docker 软件包的完整性和安全性，需要添加 Docker 的官方 GPG 密钥：
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

将 Docker 的官方软件源添加到系统的软件源列表中：
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
再次更新软件包列表
sudo apt update
运行以下命令来安装 Docker 引擎、Docker CLI 和 containerd（一个容器运行时）
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

docker-ce：Docker 社区版引擎。
docker-ce-cli：Docker 社区版的命令行界面。
containerd.io：容器运行时。
docker-buildx-plugin：用于构建 Docker 镜像的扩展插件。
docker-compose-plugin：用于管理多容器 Docker 应用的插件。

验证 Docker 安装

sudo docker run hello-world

默认情况下，运行 Docker 命令需要使用 sudo 权限。为了避免每次都输入 sudo，可以将当前用户添加到 docker 用户组中：

sudo usermod -aG docker $USER

(base) dragon@dragonquant:/mnt/mydriver1/share$ sudo docker ps -a
CONTAINER ID   IMAGE         COMMAND    CREATED              STATUS                          PORTS     NAMES
79782b895d1c   hello-world   "/hello"   About a minute ago   Exited (0) About a minute ago             wonderful_mcclintock

证明可以运行这个命令；

2)install milvus 矢量数据库

wget https://github.com/milvus-io/milvus/releases/download/v2.5.4/milvus-standalone-docker-compose.yml -O docker-compose.yml

修改配置文件
说明：anythingllm对接milvus时需要提供milvus账号密码，因此需要修改docker-compose.yml文件中的username和password字段。

sudo vi docker-compose.yml

如果docker-compose 有问题

sudo curl -L “https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname-s)-$(uname -m)” -o /usr/local/bin/docker-compose

直接下载并运行

environment:
  MINIO_ACCESS_KEY: minioadmin
  MINIO_SECRET_KEY: minioadmin

yaml 配置如下：

version: '3.5'

services:
  etcd:
    container_name: milvus-etcd
    image: quay.io/coreos/etcd:v3.5.16
    environment:
      - ETCD_AUTO_COMPACTION_MODE=revision
      - ETCD_AUTO_COMPACTION_RETENTION=1000
      - ETCD_QUOTA_BACKEND_BYTES=4294967296
      - ETCD_SNAPSHOT_COUNT=50000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/etcd:/etcd
    command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
    healthcheck:
      test: ["CMD", "etcdctl", "endpoint", "health"]
      interval: 30s
      timeout: 20s
      retries: 3

  minio:
    container_name: milvus-minio
    image: minio/minio:RELEASE.2023-03-20T20-16-18Z
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    ports:
      - "9001:9001"
      - "9000:9000"
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/minio:/minio_data
    command: minio server /minio_data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  standalone:
    container_name: milvus-standalone
    image: milvusdb/milvus:v2.5.4
    command: ["milvus", "run", "standalone"]
    security_opt:
    - seccomp:unconfined
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - ${DOCKER_VOLUME_DIRECTORY:-.}/volumes/milvus:/var/lib/milvus
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9091/healthz"]
      interval: 30s
      start_period: 90s
      timeout: 20s
      retries: 3
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - "etcd"
      - "minio"

networks:
  default:
    name: milvus

3)install ollama

curl -fsSL https://ollama.com/install.sh | sh

下载向量模型

ollama pull nomic-embed-text

查看测试

ollama list

配置ollama 环境
通过调用sudo vim /etc/systemd/system/ollama.service编辑systemd服务。这将打开一个编辑器。
对于每个环境变量，在[Service]部分下添加一行Environment：

[Service]
Environment=“OLLAMA_HOST=0.0.0.0”

4)下载安装AnythingLLM

docker pull mintplexlabs/anythingllm

进行前置操作

export STORAGE_LOCATION=$HOME/anythingllm && \
sudo mkdir -p $STORAGE_LOCATION && \
sudo touch "$STORAGE_LOCATION/.env"

针对/home/dragon/anythingllm 的权限进行改写

sudo chown -R dragon:dragon /home/dragon/anythingllm
sudo chmod -R 777 /home/dragon/anythingllm

就是为了docker 可以修改和读取这些文件
drwxr-xr-x 2 root root 4.0K 2月 10 13:20 anythingllm
drwxrwxrwx 3 dragon dragon 4.0K 12月 16 22:10 anythingllm

启动docker 服务

export STORAGE_LOCATION=$HOME/anythingllm && \
sudo mkdir -p $STORAGE_LOCATION && \
sudo touch "$STORAGE_LOCATION/.env" && \
docker run -d -p 3001:3001 \
--cap-add SYS_ADMIN \
-v ${STORAGE_LOCATION}:/app/server/storage \
-v ${STORAGE_LOCATION}/.env:/app/server/.env \
-e STORAGE_DIR="/app/server/storage" \
--restart always \
mintplexlabs/anythingllm

访问localhost端口，已经开始服务了，然后进行配置

配置向量数据库

可以添加管理员账户等操作：