一、关于sphinx搜索进程searchd启动工作进程数及mysql查询方式
即sphinx的searchd服务模式可以是多处理模式MPM,如下为启动一个searchd之后,可以看到随即searchd即启动了10个工作进程。publish:November 10, 2017 -Friday
[root@localhost test]# ps -ef | grep searchd
root 10232 1 0 14:23 pts/5 00:00:00 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10233 10232 37 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10234 10232 44 14:23 pts/5 00:00:03 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10235 10232 37 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10236 10232 25 14:23 pts/5 00:00:01 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10237 10232 27 14:23 pts/5 00:00:01 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10238 10232 37 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10239 10232 36 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10240 10232 34 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10241 10232 29 14:23 pts/5 00:00:02 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
root 10242 10232 21 14:23 pts/5 00:00:01 /usr/local/coreseek4.1/bin/searchd -c /usr/local/coreseek4.1/etc/news.conf
这是为什么呢?sphinx的searchd配置中有一项workers配置,这项配置决定了sphinx的searchd工作进程的启动方式和个数。workers配置的值有如下几种:
1,none: 所有请求串行处理。
searchd进程启动后不会再启动其它工作进程。但因为它是串行处理,因此如果有一个mysql端连接上了sphinx.则其它端要连接mysql必须等待直到这个连接断开。
2,fork:执行sphinx查询的时候创建一个新的子进程
它会在你执行sphinx查询的时候创建一个新的子进程来处理每个传入请求。工作进程的个数受max_children控制。达到了这个限制之后再使用mysql连接sphinx就会报错:
ERROR 2013 (HY000): Lost connection to MySQL server at reading initial communication packet, system error: 0
3,prefork:预先fork一个工作进程数
searchd会在启动时,预先fork一个工作进程数,再将传入的请求传递给子进程处理。一般启动10个
4,threads:创建一个新的线程来处理每个传入请求
将创建一个新的线程来处理每个传入请求
建议测试环境配置为fork,max_children配置3-5个即可,线上环境配置成prefork,max_children配置100即可,当然也还需要结合业务需要进行调整。上面的结果即是配置成了prefork。
另外sphinx的searchd服务提供轮换功能,即--rotate。为防止在需要预取大量数据的索引时停止搜索响应。有一项相关配置seamless_rotate,当然它默就是1(即启用无缝seamless轮换),还有一项配置关于索引轮换成功之后,是否删除以.old为扩展名的索引拷贝。默认是删除。如下:
seamless_rotate = 1
unlink_old = 1
之前有一篇文章: sphinx发生系统错误 1067进程意外终止问题 及 完整sphinx配置文件详细解释说明-CSDN博客 就涉及sph文件全都是命名成.old的文件的错误。
提完sphinx的搜索服务后,记一点sphinx提供的mysql方式的连接查询。php也可以直接在程序代码里像连接mysql一样连接sphinx进行SQL查询。
searchd
{
listen = 192.168.162.11:9310
listen = 192.168.162.11:9311:mysql41 #添加此行配置
}
即只需要在searchd中增加一行配置文件,加入一个协议为mysql41的listener。不过sphinx并不完全支持所有SQL查询样式,只支持一些简单语法。如下:
[root@localhost controller]# mysql -h 192.168.162.11 -P 9311
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 2.0.2-dev (r2922)
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> select title from news where match('中国人民') limit 5;
+----------+--------+-----------------------------------------+
| id | weight | title |
+----------+--------+-----------------------------------------+
| 10123463 | 12590 | 中国人民大学:城市发展导论 |
| 12123153 | 12590 | 中国人民解放军进行曲 |
| 12222119 | 12590 | 中国人民解放军进行曲 |
| 12156663 | 12590 | 中国人民解放军进行曲 |
| 12789530 | 12590 | 中国人民解放军进行曲 |
+----------+--------+-----------------------------------------+
5 rows in set (0.03 sec)
mysql>
二、coreseek以xml文件作索引源时提示sphinx/coreseek xmlpipe2 support NOT compiled
今天在使用php对库进行大量查询处理生成一个较大的xml文件,然后准备使用此xml文件作为coreseek sphinx索引数据来源时遇到报错sphinx/coreseek xmlpipe2 support NOT compiled。即是提示加载xml数据源所需要的xmlpipe2支持未编译进来 publish:October 30, 2017 -Monday。
WARNING: source 'maindata': xmlpipe2 support NOT compiled in. To use xmlpipe2, install missing XML libraries.
解决办法很简单:
yum install -y expat-devel
安装xml的支持扩展,如其描述:The expat-devel package contains the libraries, include files and documentation to develop XML applications with expat.
[onlinedev@kermit data]$ yum info expat-devel
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* epel: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Installed Packages
Name : expat-devel
Arch : x86_64
Version : 2.0.1
Release : 13.el6_8
Size : 476 k
Repo : installed
From repo : base
Summary : Libraries and header files to develop applications using expat
URL : http://www.libexpat.org/
License : MIT
Description : The expat-devel package contains the libraries, include files and documentation
: to develop XML applications with expat.
但有个麻烦的事,安装expat-devel之后需要重新安装sphinx/coreseek,所以最好一开始就知道这一点。从而避免再次安装coreseek.