Milvus安装

Milvus安装

安装Docker

官网安装

在Docker中安装Milvus

在 Ubuntu/CentOS 上安装 Milvus

第一步 确认 Docker 状态
确认 Docker daemon 正在运行：

$ docker info

如果无法正常打印 Docker 相关信息，请启动 Docker daemon.

提示：在 Linux 上，Docker 命令前面需加 sudo。若要在没有 sudo 情况下运行 Docker 命令，请创建 docker 组并添加用户。更多详情，请参阅 Linux 安装后步骤。

第二步 拉取 Milvus 镜像
拉取仅需 CPU 的镜像：

$ docker pull milvusdb/milvus:1.0.0-cpu-d030521-1ea92e

注意：如果您在拉取镜像时速度过慢或一直失败，请参考操作常见问题中提供的解决办法。

第三步 下载配置文件

$ mkdir -p /home/$USER/milvus/conf
$ cd /home/$USER/milvus/conf
$ wget https://raw.githubusercontent.com/milvus-io/milvus/v1.0.0/core/conf/demo/server_config.yaml
$ wget https://raw.githubusercontent.com/milvus-io/milvus/v1.0.0/core/conf/demo/log_config.conf

注意：万一您遇到无法通过 wget 命令正常下载配置文件的情况，您也可以在 /home/$USER/milvus/conf 路径下创建 server_config.yaml和 log_config.conf 文件，然后复制粘贴 server config 文件 和 log config 文件的内容。

server_config.yaml

# Copyright (C) 2019-2020 Zilliz. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software distributed under the License
# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
# or implied. See the License for the specific language governing permissions and limitations under the License.

version: 0.5

#----------------------+------------------------------------------------------------+------------+-----------------+
# Cluster Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | If runinng with Mishards, set true, otherwise false.       | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# role                 | Milvus deployment role: rw / ro                            | role       | rw              |
#----------------------+------------------------------------------------------------+------------+-----------------+
cluster:
  enable: false
  role: rw

#----------------------+------------------------------------------------------------+------------+-----------------+
# General Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# time_zone            | Use UTC-x or UTC+x to specify a time zone.                 | Timezone   | UTC+8           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# meta_uri             | URI for metadata storage, using SQLite (for single server  | URL        | sqlite://:@:/   |
#                      | Milvus) or MySQL (for distributed cluster Milvus).         |            |                 |
#                      | Format: dialect://username:password@host:port/database     |            |                 |
#                      | Keep 'dialect://:@:/', 'dialect' can be either 'sqlite' or |            |                 |
#                      | 'mysql', replace other texts with real values.             |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
general:
  timezone: UTC+8
  meta_uri: sqlite://:@:/

#----------------------+------------------------------------------------------------+------------+-----------------+
# Network Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# bind.address         | IP address that Milvus server monitors.                    | IP         | 0.0.0.0         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# bind.port            | Port that Milvus server monitors. Port range (1024, 65535) | Integer    | 19530           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# http.enable          | Enable web server or not.                                  | Boolean    | true            |
#----------------------+------------------------------------------------------------+------------+-----------------+
# http.port            | Port that Milvus web server monitors.                      | Integer    | 19121           |
#                      | Port range (1024, 65535)                                   |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
network: 
  bind.address: 0.0.0.0
  bind.port: 19530
  http.enable: true
  http.port: 19121

#----------------------+------------------------------------------------------------+------------+-----------------+
# Storage Config       | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# path                 | Path used to save meta data, vector data and index data.   | Path       | /var/lib/milvus |
#----------------------+------------------------------------------------------------+------------+-----------------+
# auto_flush_interval  | The interval, in seconds, at which Milvus automatically    | Integer    | 1 (s)           |
#                      | flushes data to disk.                                      |            |                 |
#                      | 0 means disable the regular flush.                         |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
storage:
  path: /var/lib/milvus
  auto_flush_interval: 1

#----------------------+------------------------------------------------------------+------------+-----------------+
# WAL Config           | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | Whether to enable write-ahead logging (WAL) in Milvus.     | Boolean    | true            |
#                      | If WAL is enabled, Milvus writes all data changes to log   |            |                 |
#                      | files in advance before implementing data changes. WAL     |            |                 |
#                      | ensures the atomicity and durability for Milvus operations.|            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# recovery_error_ignore| Whether to ignore logs with errors that happens during WAL | Boolean    | false           |
#                      | recovery. If true, when Milvus restarts for recovery and   |            |                 |
#                      | there are errors in WAL log files, log files with errors   |            |                 |
#                      | are ignored. If false, Milvus does not restart when there  |            |                 |
#                      | are errors in WAL log files.                               |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# buffer_size          | Sum total of the read buffer and the write buffer in MBs.  | Integer    | 256 (MB)        |
#                      | buffer_size must be in range [64, 4096] (MB).              |            |                 |
#                      | If the value you specified is out of range, Milvus         |            |                 |
#                      | automatically uses the boundary value closest to the       |            |                 |
#                      | specified value. It is recommended you set buffer_size to  |            |                 |
#                      | a value greater than the inserted data size of a single    |            |                 |
#                      | insert operation for better performance.                   |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# path                 | Location of WAL log files.                                 | String     |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
wal:
  enable: true
  recovery_error_ignore: false
  buffer_size: 256MB
  path: /var/lib/milvus/wal

#----------------------+------------------------------------------------------------+------------+-----------------+
# Cache Config         | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# cache_size           | The size of CPU memory used for caching data for faster    | Integer    | 4 (GB)          |
#                      | query. The sum of 'cpu_cache_capacity' and                 |            |                 |
#                      | 'insert_buffer_size' must be less than system memory size. |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# insert_buffer_size   | Buffer size used for data insertion.                       | Integer    | 1 (GB)          |
#                      | The sum of 'insert_buffer_size' and 'cpu_cache_capacity'   |            |                 |
#                      | must be less than system memory size.                      |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# preload_collection   | A comma-separated list of collection names that need to    | StringList |                 |
#                      | be pre-loaded when Milvus server starts up.                |            |                 |
#                      | '*' means preload all existing tables (single-quote or     |            |                 |
#                      | double-quote required).                                    |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
cache:
  cache_size: 4GB
  insert_buffer_size: 1GB
  preload_collection:

#----------------------+------------------------------------------------------------+------------+-----------------+
# GPU Config           | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | Enable GPU resources or not.                               | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# cache_size           | The size of GPU memory per card used for cache.            | Integer    | 1 (GB)          |
#----------------------+------------------------------------------------------------+------------+-----------------+
# gpu_search_threshold | A Milvus performance tuning parameter. This value will be  | Integer    | 1000            |
#                      | compared with 'nq' to decide if the search computation will|            |                 |
#                      | be executed on GPUs only.                                  |            |                 |
#                      | If nq >= gpu_search_threshold, the search computation will |            |                 |
#                      | be executed on GPUs only;                                  |            |                 |
#                      | if nq < gpu_search_threshold, the search computation will  |            |                 |
#                      | be executed on CPUs only.                                  |            |                 |
#                      | The SQ8H index is special, if nq < gpu_search_threshold,   |            |                 |
#                      | the search will be executed on both CPUs and GPUs.         |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# search_resources     | The list of GPU devices used for search computation.       | DeviceList | gpu0            |
#                      | Must be in format gpux.                                    |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# build_index_resources| The list of GPU devices used for index building.           | DeviceList | gpu0            |
#                      | Must be in format gpux.                                    |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
gpu:
  enable: false
  cache_size: 1GB
  gpu_search_threshold: 1000
  search_devices:
    - gpu0
  build_index_devices:
    - gpu0

#----------------------+------------------------------------------------------------+------------+-----------------+
# FPGA Config           | Description                                               | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | Use FPGA devices or not.                                   | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# search_devices       | The list of FPGA devices used for search computation.      | DeviceList | fpga0           |
#                      | Must be in format fpgax.                                   |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
fpga:
   enable: false
   search_devices:
     - fpga0

#----------------------+------------------------------------------------------------+------------+-----------------+
# Logs Config          | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# level                | Log level in Milvus. Must be one of debug, info, warning,  | String     | debug           |
#                      | error, fatal                                               |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# trace.enable         | Whether to enable trace level logging in Milvus.           | Boolean    | true            |
#----------------------+------------------------------------------------------------+------------+-----------------+
# path                 | Absolute path to the folder holding the log files.         | String     |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# max_log_file_size    | The maximum size of each log file, size range [512, 4096]  | Integer    | 1024 (MB)       |
#----------------------+------------------------------------------------------------+------------+-----------------+
# log_rotate_num       | The maximum number of log files that Milvus keeps for each | Integer    | 0               |
#                      | logging level, num range [0, 1024], 0 means unlimited.     |            |                 |
#----------------------+------------------------------------------------------------+------------+-----------------+
# log_to_stdout        | Whether to write logs to standard output in Milvus.        | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# log_to_file          | Whether to write logs to files in Milvus                   | Boolean    | true            |
#----------------------+------------------------------------------------------------+------------+-----------------+
logs:
  level: debug
  trace.enable: true
  path: /var/lib/milvus/logs
  max_log_file_size: 1024MB
  log_rotate_num: 0
  log_to_stdout: false
  log_to_file: true

#----------------------+------------------------------------------------------------+------------+-----------------+
# Metric Config        | Description                                                | Type       | Default         |
#----------------------+------------------------------------------------------------+------------+-----------------+
# enable               | Enable monitoring function or not.                         | Boolean    | false           |
#----------------------+------------------------------------------------------------+------------+-----------------+
# address              | Pushgateway address                                        | IP         | 127.0.0.1       +
#----------------------+------------------------------------------------------------+------------+-----------------+
# port                 | Pushgateway port, port range (1024, 65535)                 | Integer    | 9091            |
#----------------------+------------------------------------------------------------+------------+-----------------+
metric:
  enable: false
  address: 127.0.0.1
  port: 9091

log config

* GLOBAL:
    FORMAT                  =   "%datetime | %level | %logger | %msg"
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-global.log"
    ENABLED                 =   true
    TO_FILE                 =   true
    TO_STANDARD_OUTPUT      =   false
    SUBSECOND_PRECISION     =   3
    PERFORMANCE_TRACKING    =   false
    MAX_LOG_FILE_SIZE       =   209715200 ## Throw log files away after 200MB
* DEBUG:
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-debug.log"
    ENABLED                 =   true
* WARNING:
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-warning.log"
* TRACE:
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-trace.log"
* VERBOSE:
    FORMAT                  =   "%datetime{%d/%M/%y} | %level-%vlevel | %msg"
    TO_FILE                 =   false
    TO_STANDARD_OUTPUT      =   false
## Error logs
* ERROR:
    ENABLED                 =   true
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-error.log"
* FATAL:
    ENABLED                 =   true
    FILENAME                =   "/var/lib/milvus/logs/milvus-%datetime{%y-%M-%d-%H:%m}-fatal.log"

第四步 启动 Milvus Docker 容器

$ docker run -d --name milvus_cpu_1.0.0 \
-p 19530:19530 \
-p 19121:19121 \
-p 9091:9091 \
-v D:\milvus\db:/var/lib/milvus/db \
-v D:\milvus\conf:/var/lib/milvus/conf \
-v D:\milvus\logs:/var/lib/milvus/log \
-v D:\milvus\wal:/var/lib/milvus/wal milvusdb/milvus:1.0.0-cpu-d030521-1ea92e

上述命令中用到的 docker run 参数定义如下：

-d: 运行 container 到后台并打印 container id。
--name: 为 container 分配一个名字。
-p: 暴露 container 端口到 host。
-v: 将路径挂载至 container。

最后，确认 Milvus 运行状态：

$ docker ps

如果 Milvus 服务没有正常启动，您可以执行以下命令查询错误日志。

$ docker ps -a
$ docker logs <milvus container id>

在 Windows 上安装 Milvus

在 Windows 上安装 Milvus 的步骤和在 Ubuntu/CentOS 上几乎一样，只不过 第三步 和 第四步 略有不同。

在第三步中，不同于使用 wget 来获取配置文件，使用 Windows 安装时，建议在 C 盘，或其它合适的位置创建 milvus 文件夹，下面包含 db，conf，logs 和 wal 等文件夹，然后分别复制 server config 文件 和 log config 文件的内容到您在 C:\milvus\conf 路径下创建的server_config.yaml 和 log_config.conf 文件中。

第四步，启动 Docker 容器，将 Milvus 文件映射到您本地的文件路径。下面的命令是在命令提示符中运行的：

$ docker run -d --name milvus_cpu_1.0.0 ^ 
-p 19530:19530 ^
-p 19121:19121 ^
-p 9091:9091 ^
-v D:\milvus\db:/var/lib/milvus/db ^
-v D:\milvus\conf:/var/lib/milvus/conf ^
-v D:\milvus\logs:/var/lib/milvus/log ^
-v D:\milvus\wal:/var/lib/milvus/wal milvusdb/milvus:1.0.0-cpu-d030521-1ea92e