Sphinx 2.3.2-beta 笔记

最新推荐文章于 2022-01-12 20:40:40 发布
最新推荐文章于 2022-01-12 20:40:40 发布
阅读量334 收藏
点赞数
文章标签： sphinx 搜索引擎
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/big_cheng/article/details/122112520
版权

目录

版权

本文为原创, 遵循 CC 4.0 BY-SA 版权协议, 转载需注明出处: https://blog.csdn.net/big_cheng/article/details/122112520.

2.3.2-beta

Sphinx 虽然功能强大, 但上手有一些’坑’, 希望本文对您有帮助.

环境(CentOS-7, virtualbox虚机) 无法成功运行目前最新的Sphinx 3.4.1(*** Oops, indexer crashed! Please send the following report to developers. …), 所以选用了http://sphinxsearch.com/downloads/archive/ 的最新版:

rpm -Uhv sphinx-2.3.2-1.rhel7.x86_64.rpm

注: 帮助文档在.rpm 内usr\share\doc\sphinx-2.3.2\sphinx.html - 建议您事先阅读(内容很多).

mysql lib

环境是10.6.3-MariaDB 而非mysql, 开始时总是连不到mysql 数据库, 尝试了yum install mariadb-devel/mariadb-libs/mysql-community-libs/mysql-community-client/mysql-community-devel 都不行.

https://sphinxsearch.com/forum/view.html?id=15745

# 因为libmysqlclient.so link的目标不存在
cd /usr/lib64
rm libmysqlclient.so
ln -s libmariadbd.so.19 libmysqlclient.so
ls libmysql* -l

在3x文档"Installing indexer SQL drivers on Linux" 里提到具体名称: “libmysqlclient.so or libmariadb.so” - Sphinx 按顺序尝试加载这2个lib.

服务启动失败

似乎启动服务命令运行成功, 且日志里无错误, 但服务状态总是出错: “searchd.service start operation timed out”.

改为手动启动:

# 修改索引文件权限
cd /var/lib/sphinx
chown sphinx.sphinx *
ls -l

# 手工su sphinx 后启动、停止ok, .pid文件创建、删除正常
searchd --config /etc/sphinx/sphinx.conf
searchd --config /etc/sphinx/sphinx.conf --stopwait

中文(ngram)

sphinx.conf
    source
        sql_query_pre = SET NAMES utf8
    index
        stopwords  = /etc/sphinx/stopwords-innodb.txt  /etc/sphinx/stopwords-myisam.txt
        ngram_len = 1
        ngram_chars = U+3000..U+2FA1F
example.sql
    + SET NAMES 'utf8'
    CREATE TABLE ... DEFAULT CHARSET=utf8;

见文档12.1.11. sql_query_pre: “Sphinx accepts only UTF-8 texts”.

stopwords 借用mysql 的: mysql-5.7 Chapter-12 functions.html “12.10 Full-Text Search Functions”.

ngram_chars = U+3000…U+2FA1F
https://www.unicode.org/versions/Unicode14.0.0/ch02.pdf “2.9 Details of Allocation”. CJK 没有一个单一连续的范围, 上面范围(Plane 0)里也包含非CJK 字符.

main+delta (plain+plain)

main 有记录1,2, delta 也有记录1,2, 查询时delta 将覆盖main 的:

# conf "source srcDelta" 注掉"sql_query_killlist = select 2" 行

# (sphinx用户)
indexer --all # 重建索引
searchd --config /etc/sphinx/sphinx.conf # 启动searchd
# (root用户) mysql -h0 -P9306
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from delta;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)
# 4条记录都匹配'cat'. 因为delta 在后, 所以相同id 只返回delta 的记录(main 的记录被屏蔽):
MySQL [(none)]> select * from main,delta where match('cat');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)
# delta 记录2 被"deleted"条件过滤掉后, main 记录2 不再被屏蔽:
MySQL [(none)]> select * from main,delta where match('cat') and deleted=0;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

可见准确理解是: main和delta 均匹配后, 2组结果里重复的记录选用delta的 - delta不能完全屏蔽main(实质是delta or main).

把delta 合并到main:

# (sphinx用户) (因为searchd 正在使用索引文件, 加"--rotate")
indexer --merge main delta --rotate
# (root用户)
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from main where match('中');
Empty set (0.001 sec)

MySQL [(none)]> select * from main where match('日');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)

可见delta 记录的field/attr 都覆盖到了main.

kill-list, main+delta (plain+plain)

见文档"12.1.16. sql_query_killlist".

main 有记录1,2. delta 有记录1,2, kill-list 含记录2:

# conf "source srcDelta" 启用"sql_query_killlist = select 2" 行

# 注意: test数据库delta表仍有记录1,2 (2未删).

# (sphinx用户)
cd /var/lib/sphinx # sphinx 索引文件目录
searchd --config /etc/sphinx/sphinx.conf --stopwait # 停searchd
rm * # 删除所有索引文件
indexer --all # 重建所有索引
searchd --config /etc/sphinx/sphinx.conf # 启动searchd
# (root用户) mysql -h0 -P9306
# delta 仍有记录1,2(且查询时屏蔽了main 记录):
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from delta;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from main,delta where match('cat');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)
# 但是delta 记录2 被过滤掉时, main 记录2 仍会被屏蔽:
MySQL [(none)]> select * from main,delta where match('cat') and deleted=0;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
+------+------+------+---------+
1 row in set (0.001 sec)

MySQL [(none)]> select * from main,delta where match('中');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
+------+------+------+---------+
1 row in set (0.001 sec)

MySQL [(none)]> select * from main,delta where addr='bar';
Empty set (0.000 sec)

可见delta kill-list 彻底屏蔽了main 的相关记录, 无论delta 本身返回这些记录否(可理解为delta or (main - delta.killlist)).

合并delta 到main:

# (sphinx用户)
indexer --merge main delta --rotate
# (root用户)
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.000 sec)

MySQL [(none)]> select * from main where match('中');
Empty set (0.001 sec)

MySQL [(none)]> select * from main where match('日');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)

delta 完全覆盖了main.

另, 测试delta 仅含记录1 且kill-list 含记录2 (数据库表里删除记录2; conf 启用"sql_query_killlist = select 2" 行), 合并后main 仅含记录1(33-baz) - 即delta kill-list 始终有效, 无论delta 本身有无这些记录.

main+rt (plain+rt)

rt 的优势是可以在线增删改且立即反映(plain 只能在线改attr), 但存在rt or main 问题: rt 记录不匹配时, 重复的main 记录(代表修改前的旧值) 只要匹配仍能返回:

# (sphinx用户) 重建:
searchd --config /etc/sphinx/sphinx.conf --stopwait
rm *
indexer --all
searchd --config /etc/sphinx/sphinx.conf
# (root用户) mysql -h0 -P9306
# rt 初始为空:
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from rt;
Empty set (0.002 sec)
# rt 新增记录1 - 立即反映:
MySQL [(none)]> replace into rt (id, title, content, age, addr, deleted) values (1, 'rt one', '日本 cat', 33, 'baz', 0);
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from rt;
+------+------+---------+------+
| id   | age  | deleted | addr |
+------+------+---------+------+
|    1 |   33 |       0 | baz  |
+------+------+---------+------+
1 row in set (0.002 sec)
# main+rt 也是rt or main:
MySQL [(none)]> select * from main,rt where match('cat');
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   33 | baz  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.002 sec)

MySQL [(none)]> select * from main,rt where match('cat') and addr!='baz';
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

通常不希望也能搜到修改前的旧内容. 可以在修改rt 记录同时也修改main 旧记录(置一个标志attr) - 查询时总是检查此标志:

# 因为rt 记录1 最新, 弃用main 记录1:
MySQL [(none)]> update main set deleted=1 where id=1;
Query OK, 1 row affected (0.001 sec)
# 查询时总是检查弃用标志:
MySQL [(none)]> select * from main,rt where deleted=0 and match('cat') and addr!='baz';
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
1 row in set (0.001 sec)

如此rt 可以完全屏蔽重复的main 记录(即使之后rt 记录又删除了).

注1-rt 不能合并到plain:

bash-4.2$ indexer --merge main rt --rotate
......
using config file '/etc/sphinx/sphinx.conf'...
merging index 'rt' into index 'main'...
FATAL: failed to merge index 'rt' into index 'main': source index preload failed: failed to open /var/lib/sphinx/rt.sph: No such file or directory

注2-rt 磁盘结构实际是append-only:
见文档"4.3. RT index internals": rt 增删改先写到一个唯一的内存chunk, 内存chunk 满了后再写到磁盘chunk. 所有磁盘chunk 不会更新只会append, rt 也使用plain kill-list 的方式来屏蔽旧记录. 所以rt 的实时可更新是有代价的 - 过多的删/改会导致磁盘碎片和搜索变慢.
见文档"8.36. OPTIMIZE INDEX syntax" rt 碎片整理.

--merge-dst-range

见文档"3.13. Index merging" 和"7.1. indexer command reference".

delta 合并到main(rt 不能参与合并) 时可以指定main 记录的过滤条件: 先过滤main 再合并.
测试案例: main 记录1,2, 1更新为deleted=1. delta 无记录1, 记录2 deleted=1. 合并时指定"deleted 0 0". 结果无记录1, 记录2合并了:

# conf "source srcDelta" 注掉"sql_query_killlist" 行

# 确认test数据库(非sphinx索引) 记录如下:
MariaDB [test]> select * from main;
+----+------------+---------------------------+-----+------+---------+
| id | title      | content                   | age | addr | deleted |
+----+------------+---------------------------+-----+------+---------+
|  1 | main 1 one | this 中 国 abc cat from   |  11 | foo  |       0 |
|  2 | main 1 two | this中国def cats cannot   |  22 | bar  |       0 |
+----+------------+---------------------------+-----+------+---------+
2 rows in set (0.001 sec)

MariaDB [test]> select * from delta;
+----+-------------+------------+-----+------+---------+
| id | title       | content    | age | addr | deleted |
+----+-------------+------------+-----+------+---------+
|  2 | delta 1 two | 日本cats   |  44 | xyz  |       1 |
+----+-------------+------------+-----+------+---------+
1 row in set (0.001 sec)

# (sphinx用户)
searchd --config /etc/sphinx/sphinx.conf --stopwait
rm *
indexer --all
searchd --config /etc/sphinx/sphinx.conf
# (root用户) mysql -h0 -P9306
MySQL [(none)]> update main set deleted=1 where id=1;
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       1 |
|    2 |   22 | bar  |       0 |
+------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> select * from delta;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
1 row in set (0.001 sec)
# (sphinx用户) 执行合并:
indexer --merge main delta --merge-dst-range deleted 0 0 --rotate
# (root用户)
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    1 |   11 | foo  |       0 |
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
2 rows in set (0.001 sec)

Oops! 为什么记录1 没有被过滤掉? 原来"update main deleted=1" 时仅仅更新了内存的attr缓存 - 并未写到磁盘, 看起来merge 时只读了磁盘!
从头再来:

# 同上......

# attr缓存写到磁盘:
MySQL [(none)]> flush attributes;
+------+
| tag  |
+------+
|    1 |
+------+
1 row in set (0.502 sec)
# (sphinx用户) 执行合并:
indexer --merge main delta --merge-dst-range deleted 0 0 --rotate
# (root用户)
MySQL [(none)]> select * from main;
+------+------+------+---------+
| id   | age  | addr | deleted |
+------+------+------+---------+
|    2 |   44 | xyz  |       1 |
+------+------+------+---------+
1 row in set (0.001 sec)

现在正常: main 记录1 在合并前先被条件过滤掉, 而不满足条件的delta 记录2 没有被过滤, 可见过滤条件仅用于原目标索引(main).

与kill-list 的关系

前面章节讲过delta kill-list 在合并时也能屏蔽main 记录, 看起来这2种都是合并时过滤的不同方式, 适用于不同场合: kill-list 完全不触动main, 而--merge-dst-range 需要更新main attr(到磁盘).

设计测试案例-改用kill-list: main 记录1,2(deleted均0). delta 无记录1, 记录2 deleted=1, kill-list 含1. 合并时不指定过滤条件. 结果相同.

事务

经试验Sphinx 的事务与db 事务差异较大. Update:

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

# (autocommit 设不设置似乎没区别)
MySQL [(none)]> set autocommit=0;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> begin;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> update rt set deleted=3 where id=1;
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       3 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

# 直接kill 进程:
ps -ef|grep searchd
kill xxxx yyyy
ps -ef|grep searchd # 确认已kill

searchd --config /etc/sphinx/sphinx.conf

# mysql -h0 -P9306
MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       3 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.002 sec)

未commit 的值保留下来了, 可见update 是立即生效. Update rollback:

MySQL [(none)]> begin;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> update rt set deleted=3 where id=1;
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       3 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> rollback;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> exit
Bye

# mysql -h0 -P9306

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       3 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

改为3, rollback 后退出重连, 查询到仍是3. 可见update 仍是立即生效的, rollback 对它无效.

Replace:

MySQL [(none)]> begin;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> replace into rt values (1, 'delta 1 one', '日 本 cat', '日 本 cat', 33, 'baz', 3);
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> commit;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       3 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

replace 后在事务内查到的仍是旧值, commit 后才能查到新值. 文档"8.11. BEGIN, COMMIT, and ROLLBACK syntax" 最后一句"Overly isolated means that the changes are not only invisible to the concurrent transactions" 就是这个意思.

Replace rollback:

MySQL [(none)]> begin;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> replace into rt values (1, 'delta 1 one', '日 本 cat', '日 本 cat', 33, 'baz', 3);
Query OK, 1 row affected (0.001 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

MySQL [(none)]> rollback;
Query OK, 0 rows affected (0.000 sec)

MySQL [(none)]> select * from rt;
+------+-------------+------+------+---------+
| id   | content_str | age  | addr | deleted |
+------+-------------+------+------+---------+
|    1 | 日 本 cat   |   33 | baz  |       0 |
|    2 | 日本cats    |   44 | xyz  |       1 |
+------+-------------+------+------+---------+
2 rows in set (0.001 sec)

replace rollback 前后均是旧值. 之前是事务内看不到新值, 之后说明rollback 成功了.

试验结果总结:

  • update 不支持事务, 总是直接生效了: begin => update => 查询值已生效 => 再commit/rollback无作用
  • replace 支持事务: begin => replace => 查询值未变 => commit/rollback 后再查起作用了
  • delete 支持
  • (单笔)insert 支持

附录

sphinx.conf

(注: 未经优化)

source srcMain
{
	type			= mysql

	sql_host		= localhost
	sql_user		= root
	sql_pass		= 1
	sql_db			= test
	sql_port		= 3306	# optional, default is 3306
	
	sql_query_pre = SET NAMES utf8

	sql_query		= \
		SELECT id, title, content, age, addr, deleted \
		FROM main

	sql_attr_uint		= age
	sql_attr_string		= addr
	sql_attr_uint = deleted
}

index main
{
	source			= srcMain
	path			= /var/lib/sphinx/main
	morphology  = stem_en
	stopwords  = /etc/sphinx/stopwords-innodb.txt  /etc/sphinx/stopwords-myisam.txt
	ngram_len = 1
	ngram_chars = U+3000..U+2FA1F
}


source srcDelta : srcMain {
  sql_query = select id, title, content, age, addr, deleted \
    from delta
  sql_query_killlist = select 2
}

index delta : main {
  source = srcDelta
  path = /var/lib/sphinx/delta
}


index rt
{
	type			= rt
	rt_mem_limit		= 128M

	path			= /var/lib/sphinx/rt
	morphology  = stem_en
	stopwords  = /etc/sphinx/stopwords-innodb.txt  /etc/sphinx/stopwords-myisam.txt
	ngram_len = 1
	ngram_chars = U+3000..U+2FA1F

	rt_field = title
	rt_field = content
	rt_attr_uint   = age
	rt_attr_string = addr
	rt_attr_uint   = deleted
}


indexer
{
	mem_limit		= 128M
}


searchd
{
	listen			= 9306:mysql41
	log			= /var/log/sphinx/searchd.log
	query_log		= /var/log/sphinx/query.log
	read_timeout		= 5
	max_children		= 30
	pid_file		= /var/run/sphinx/searchd.pid
	seamless_rotate		= 1
	preopen_indexes		= 1
	unlink_old		= 1
	workers			= threads # for RT to work
	binlog_path		= /var/lib/sphinx/
}

example.sql

SET NAMES 'utf8';
use test;


DROP TABLE IF EXISTS test.main;
CREATE TABLE test.main
(
	id          INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT,
	title       VARCHAR(255) NOT NULL,
	content     TEXT NOT NULL,
	age         INTEGER NOT NULL,
	addr       VARCHAR(24) NOT NULL,
	deleted    INTEGER NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

REPLACE INTO test.main ( id, title, content, age, addr, deleted ) VALUES
	( 1, 'main 1 one', 'this 中 国 abc cat from', 11, 'foo', 0 ),
	( 2, 'main 1 two', 'this中国def cats cannot', 22, 'bar', 0 );


DROP TABLE IF EXISTS test.delta;
CREATE TABLE test.delta
(
	id          INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT,
	title       VARCHAR(255) NOT NULL,
	content     TEXT NOT NULL,
	age         INTEGER NOT NULL,
	addr       VARCHAR(24) NOT NULL,
	deleted    INTEGER NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

replace into test.delta (id, title, content, age, addr, deleted) VALUES
	(1, 'delta 1 one', '日 本 cat', 33, 'baz', 0),
	(2, 'delta 1 two', '日本cats', 44, 'xyz', 1 );

CentOS-8 环境

经试验CentOS-8 环境(MariaDB-10.6) 可如下安装2.3.2-beta:

# install pg|odbc 依赖
yum install postgresql-libs unixODBC
# 忽略目前缺的依赖, 强制安装:
rpm -Uhv -i --nodeps sphinx-2.3.2-1.rhel7.x86_64.rpm
# 缺的依赖直接ln 现有的文件:
# (缺依赖运行searchd 会报错例如: error while loading shared libraries: libmysqlclient.so.18: ...)
# (可从CentOS-7环境拷贝, 文件版本可略有差异)
cd /usr/lib64
ln -s libmariadb.so.3 libmysqlclient.so.18
ln -s libssl.so.1.1.1g libssl.so.10
ln -s libcrypto.so.1.1.1g libcrypto.so.10

安装rpm后会提示:
[/usr/lib/tmpfiles.d/searchd.conf:1] Line references path below legacy directory /var/run/, updating /var/run/sphinx → /run/sphinx; please update the tmpfiles.d/ drop-in file accordingly.
这个好像是建议弃用/var/run/sphinx 改用/run/sphinx, 但因为sphinx.conf “pid_file” 及searchd.service 里都用老的路径, 试验时没改.

可修改/usr/lib/systemd/system/searchd.service 改用root 账号启动服务:

User=root
Group=root
# ExecStartPre=/bin/chown sphinx.sphinx /var/run/sphinx

参考

使用Sphinx 2.3.2-beta 实现全文搜索(rt 方案)

big_cheng
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
（迁移）sphinx使用笔记
倪时鸿的博客
03-18 339
教程链接 http://ishare.iask.sina.com.cn/f/35lxVVGIqXs.html 强制关闭 killall -9 searchd 建立索引 /usr/local/sphinx/bin/indexer -c /usr/local/sphinx/etc/sphinx.conf --all 更新索引 /usr/local/sphinx/bin/indexer -...
sphinx-2.3.2-beta
12-01
Sphinx-2.3.2-beta 版本是 Sphinx 的一个开发版本，提供了比稳定版更多的新特性和改进，同时可能包含一些未解决的bug。在深入探讨这个版本之前，我们先来理解一下 Sphinx 的基本概念。 Sphinx 主要用于创建技术文档...
欢迎使用CSDN-markdown编辑器
u014379440的博客
01-06 200
Sphinx笔记下载sphinx 地址 sphinx的目录结构使用Sphinx1、首先得有数据 2、建立Sphinx配置文件 3、生成索引 4、启动Sphinx 5、使用之（调用api或search.exe程序进行查询）具体如下2，关于sphinx配置文件 #源定义 需要对数据库里面的哪些数据建立全文索引 diqu,这个diqu可以自定义 source
sphinx索引中关于indexer里的mem_limit配置项的大小限制
西西工作室
08-12 822
今天在进行sphinx索引的时候遇到报错：WARNING: collect_hits: mem_limit=-2097152 kb too low, increasing to -118752 kb，排查后发现是mem_limit配置项的问题，indexer索引中的配置项mem_limit是建立索引时的索引内存限制值，此值并不是越大越好，今天在修改配置的时候，考虑到服务器内存足够大，我就把这个配置改成了2G即2048M，然后在索引的时候就碰到了下面的报错，详细如下： [kermit@kermit sphin
Sphinx奇怪的报错indexer crashed
walle的博客
06-03 339
担心sphinx的内存不够用，把conf里面的mem_limit加到了4000M，结果报错了： 改到小一点的800M就不报错 官方文档中找到这么一段话，原文和译文如下： mem_limit----------------- Indexing RAM usage limit. Optional, default is 32M.Enforced memory usage limit that the indexer will not go above. Can bespecified in b..
Coreseek:indexer crashed不解之谜
weixin_33757911的博客
01-29 82
前两天浩哥让我再把Coreseek的索引再做一次，由于需求那边有点变化，要把索引的公司名字显示出来，就在配置文件中面加入了sql_field_string：字符串字段.。 这个属性特别好用，由于它不仅能够当做属性过滤并且还能够可全文搜索，可返回原始文本信息。 然后写好文件，建立索引，建索引的时候出现了这么一个匪夷所思的问题Oops, indexer crashed!真当是莫名其妙。 ...
sphinx4-1.0beta6-bin.zip
12-27
这个"**sphinx4-1.0beta6-bin.zip**"文件是Sphinx4的1.0 beta 6版本的二进制发行包，包含预编译的库和必要的资源，使得用户可以直接在自己的项目中集成Sphinx4而无需从源码编译。 **Sphinx4概述：** Sphinx4是基于...
sphinx4-1.0beta6-src
02-20
这个"**sphinx4-1.0beta6-src**"是Sphinx4的源代码版本，相对应的"**sphinx4-1.0beta6-bin**"则是编译好的二进制版本，通常包含了可以直接运行的jar文件和其他必要的资源文件。在源代码版本中，用户可以深入理解其...
OPEN SPHiNX C-- Compiler-开源
05-14
Open SPHiNX C--编译器是基于Peter Cellik创建的“ SPHiNX C--”编程语言的一组编译器和库。
cmusphinx-zh-cn-5.2.zip
07-06
cmu sphinx语音识别，中文语音包，官网下载重新打的zip包
使用Sphinx 2.3.2-beta 实现全文搜索(rt 方案)
big_cheng的博客
01-12 360
使用Sphinx 2.3.2-beta 实现全文搜索 (rt)
MySQL主从同步show master status; Empty set (0.01 sec)主库无master状态问题解决
热门推荐
lucky_ykcul的博客
10-29 1万+
在进行MySQL主从复制过程中出现主库master无状态的问题 MariaDB [(none)]> show master status; Empty set (0.01 sec) 查看master配置文件 [root@server1 local]# vim /etc/my.cnf log-bin=mysql-bin #启用二进制日志 server-id=1 正常情况下配置了log-b...
php sphinx 高效率搜索引擎配置教程
csdn265的博客
01-22 886
php sphinx 高效率搜索引擎配置教程，有需要的朋友可以参考下。 php sphinx 高效率搜索引擎配置教程，有需要的朋友可以参考下。 1.下载PHP客户端安装: http://pecl.php.net/package/sphinx vim sphinxclient.c                     找到 void sock_close ( int soc
coreseek3.2.14（sphinx）搜索引擎的安装及简单配置
qq_35118483的博客
11-01 3519
转自 http://blog.csdn.net/qq157962718/article/details/50985562 做了部分修改 架构使用图例 优势 数据库搜索，如果搜索范围包括全部正文数据，往往性能很低，用户响应时间长（分钟级，经常超时）；而全文检索可以在0.X秒（毫秒级）的时间内将结果反馈给用户。数据库搜索，一个用户在搜索，往往会影响其他用户对应用系统进行操作
Sphinx 2.2.3 安装和配置，英文数字中文搜索
风.foxwho(神秘狐)
07-25 5301
注：项目中中文分词比较
sphinx教程2__安装、配置和使用
clh604的专栏
06-24 8824
从sphinx网站下载sphinx源码包，当前稳定版本是：http://sphinxsearch.com/downloads/sphinx-0.9.9.tar.gz  一、安装       步骤如下： 放在任意目录下，解压sphinx源码包: $ tar xzvf sphinx-0.9.9.tar.gz        $ cd sphinx 运行configure配
Sphinx参考手册(九)
cbhjerry的博客
05-12 1176
9. sphinx.conf选项参考
更新sphinx 索引属性方法 -UpdateAttributes 使用
萧刚
06-25 3298
//sphinx把商品上架(deleted为0) public static function upProSphinx($proid) { $sphinxclient = new SphinxClient (); $sphinxclient->UpdateAttributes ( "products", array("deleted"), array($proid=>array(0))
sphinx kill-list设置
cleanfield的专栏
11-15 3207
官方文档： ******************************************************************************************************************* 11.1.16. sql_query_killlist：Kill-list查询 用于得到Kill-list的查询。可选选项，默认为空（不设定查询）
sphinx-build: command not found
最新发布
01-24
根据提供的引用内容，当出现"sphinx-build: command not found"错误时，可能是因为没有安装Sphinx或者没有正确设置SPHINXBUILD环境变量。以下是解决该问题的步骤： 1. 确保已经安装了Sphinx。可以使用以下命令检查Sphinx是否已安装： ```shell sphinx-build --version ``` 如果未安装Sphinx，可以使用以下命令安装： ```shell pip install sphinx ``` 2. 如果已经安装了Sphinx但仍然出现"sphinx-build: command not found"错误，可能是因为SPHINXBUILD环境变量没有正确设置。可以按照以下步骤设置SPHINXBUILD环境变量： - 打开终端并输入以下命令： ```shell which sphinx-build ``` 这将显示Sphinx的安装路径，例如：/usr/local/bin/sphinx-build。 - 打开终端配置文件（例如.bashrc、.bash_profile或.zshrc）并添加以下行： ```shell export SPHINXBUILD=/usr/local/bin/sphinx-build ``` - 保存并关闭终端配置文件，然后重新加载配置文件或重新启动终端。 - 再次运行sphinx-build命令，应该不再出现"sphinx-build: command not found"错误。 请注意，确保替换上述命令中的路径为实际的Sphinx安装路径。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值