linux下安装中文检索软件 sphinx/coreseek-CSDN博客

本文链接：https://blog.csdn.net/w786572258/article/details/50620468

注意：前提要先安装好mysql

一、下载coreseek-3.2.14.tar.gz，解压

命令会吧，不多说。

二、编译安装mmseg、coreseek(其实是一个中文分词词典)

1、进入coreseek-3.2.14里面的mmseg目录,执行命令：

./configure --prefix=/usr/local/mmseg3

make

执行后最后一段报错："cannot input file: src/Makefile.in" 这里要automake 一下，如果没有automake命令则自己用yum安装一下。

重新make,这次就没报错了，继续

make install

2、回到上级目录的csft目录，编译安装coreseek,执行命令：

./configure --prefix=/usr/local/coreseek --with-mysql=/usr/local/mysql --with-mmseg=/usr/local/mmseg3 --with-mmseg-includes=/usr/local/mmseg3/include/mmseg/ --with-mmseg-libs=/usr/local/mmseg3/lib/

make

make install

三、建立mysql的测试表数据

注意：表必须要有主键，我建了一个t2表

四、配置csft.conf

复制 /usr/local/soreseek/etc/sphinx.conf.dist ，改名为：csft.conf（一定要这个名字，默认查找这个配置文件）。
打开csft.conf，修改（列出一小部分,参考而已）：
sql_host = localhost
sql_user = root
sql_pass = root
sql_db = test
sql_port = 3306

sql_sock = /tmp/mysql.sock

sql_query_pre = SET NAMESutf8
sql_query_pre = SET SESSION query_cache_type=OFF

sql_query = select * from t2

#sql_attr_uint = group_id

#sql_attr_timestamp = date_added

sql_query_info = SELECT * from t2

#stopwords = G:\data\stopwords.txt
#wordforms = G:\data\wordforms.txt
# exceptions = /data/exceptions.txt
#charset_type = zh_cn.utf-8

#词典路径
charset_dictpath = /usr/local/mmseg3/etc/

#这一段必须关闭
#charset_table = 0..9, A..Z->a..z, _, a..z, U+410..U+42F->U+430..U+44F, U+430..U+44F

ngram_len =0

五、php调用coreseek(sphinx)

1、libsphinxclient 安装（PHP模块需要）
cd api/libsphinxclient
./configure –prefix=/usr/local/sphinx
make 时会出现以下错误：

libsphinxclient # make
make  all-am
make[1]: Entering directory `/work/setup/sphinx-1.10-beta/api/libsphinxclient'
/bin/sh ./libtool --mode=compile --tag=CC gcc -DHAVE_CONFIG_H -I. -I. -I.     -g -O2 -c -o sphinxclient.lo sphinxclient.c
mkdir .libs
 gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c sphinxclient.c  -fPIC -DPIC -o .libs/sphinxclient.o
sphinxclient.c: In function 'net_connect_get':
sphinxclient.c:1358: error: 'SO_NOSIGPIPE' undeclared (first use in this function)
sphinxclient.c:1358: error: (Each undeclared identifier is reported only once
sphinxclient.c:1358: error: for each function it appears in.)
make[1]: *** [sphinxclient.lo] Error 1
make[1]: Leaving directory `/work/setup/sphinx-1.10-beta/api/libsphinxclient'
make: *** [all] Error 2

请打开sphinxclient.c，将以下代码：

#ifndef _WIN32
        if ( setsockopt ( sock, SOL_SOCKET, SO_NOSIGPIPE, (void *)&optval, (sock
len_t)sizeof(optval) ) < 0 )
        {
                set_error ( client, "setsockopt() failed: %s", sock_error() );
                return -1;
        }
#endif

修改为：

#ifndef _WIN32
#ifdef SO_NOSIGPIPE
        if ( setsockopt ( sock, SOL_SOCKET, SO_NOSIGPIPE, (void *)&optval, (sock
len_t)sizeof(optval) ) < 0 )
        {
                set_error ( client, "setsockopt() failed: %s", sock_error() );
                return -1;
        }
#endif
#endif

如果出现以下错误：

gcc -g -O2 -o test test.o  .libs/libsphinxclient.a
.libs/libsphinxclient.a(sphinxclient.o): In function `set_error':
/opt/software/sphinx-1.10-beta/api/libsphinxclient/sphinxclient.c:359: undefined reference to `_vsnprintf'
collect2: ld returned 1 exit status
make[1]: *** [test] Error 1
make[1]: Leaving directory `/opt/software/sphinx-1.10-beta/api/libsphinxclient'
make: *** [all] Error 2

则注释掉头部的：

#define vsnprintf _vsnprintf

就可以了。

接着：
make install

2、php运行

<?php
    include "sphinxapi.php"; //这个文件在coreseek安装包的api文件夹里面
    $sphinx = new SphinxClient();
    //print_r($sphinx);
    //phpinfo();
    //
    $sphinx->SetServer("localhost", 9312);
    $sphinx->SetMatchMode(SPH_MATCH_ANY);


    // $result = $sphinx->query("php","test1");
    $result = $sphinx->query("caocao","*");
    echo '<pre>';
    //print_r($result);
?>

这是另一个比较全的代码

<?php
    $keyword = $_REQUEST['keyword'];
    include "sphinxapi.php"; //这个文件在coreseek安装包的api文件夹里面
    $sphinx = new SphinxClient();
    
    $sphinx->SetServer("localhost", 9312);
    $sphinx->SetMatchMode(SPH_MATCH_ANY);
    // $result = $sphinx->query("php","test1");
    $result = $sphinx->query("$keyword","*");
    //echo "<pre>";
    //print_r($result);
    //取出文档id
    $ids = implode(',',array_keys($result['matches']));
    //echo $ids;
    //查询mysql
    $conn = mysql_connect('localhost', 'root', 'root');
    mysql_query("set names utf8");
    mysql_select_db('test');
    $res = mysql_query("SELECT * FROM t2 WHERE id in($ids)");
    $arr = array();
    while($row = mysql_fetch_assoc($res)) {
        $arr[] = $row;
    }
    //echo '<pre>';
    //print_r($arr);

    //查询结果
    echo "<h1>查询结果：共搜索到{$result['total_found']}篇</h1>";
    $opts = array(
        "before_match" => "<font style='color:red'>",
        "after_match" => "</font>",
    );
    foreach($arr as $k=>$v) {
        //高亮显示
        $v = $sphinx->buildExcerpts($v, 'main', $keyword, $opts);
        
        //var_dump($v);
        //exit;
        echo "<b>第{$v[0]}篇帖子</b><br>";
        echo "名字：{$v['1']}<br>";
        echo "内容：{$v['2']}<br>";
        echo "标题：{$v['3']}<br>";
        echo "<hr>";
    }
   
?>

六、配置sphinx的主索引、增量索引

实现目的： mysql每插入一条新数据需要使用indexer --all --rotate 更新索引才能同步数据，现在的设计模式是这样：每5分钟更新一下增量索引、每天凌晨3点更新主索引（所有）。

1、在mysql建立计数器

2、修改csft.con主数据源、增量数据源、主数据索引、增量数据索引

  
  
   
   source main{
  
  
  
  
   
   sql_query_pre                   = replace into sph_counter select 1,max(id) from t2
  
  
  
  
   
   }
  
  
  
  
   
   
source delta : main
{
        #sql_ranged_throttle                    = 100
        sql_query_pre                   = SET NAMES utf8
        sql_query = select * from t2 where id > (select max_doc_id from sph_counter where counter_id=1)
}
  
  
  
  
   
   
index main
{
        source                  = main
        path                    = /usr/local/coreseek/var/data/main

  
  
  
  
   
   }
  
  
  
  
   
   
index delta : main
{
        source=delta
        path                    = /usr/local/coreseek/var/data/delta
        morphology              = stem_en
}

3、编写sell脚本配合crontab

main.sh

 
 
  
  #!/bin/bash
#main.sh
/usr/local/coreseek/bin/indexer --all --rotate >> /usr/local/coreseek/var/log/main.log

delta.sh

 
 
  
  #!/bin/bash
  
  

  
  #delta.sh
  
  

  
  /usr/local/coreseek/bin/indexer delta --rotate >> /usr/local/coreseek/var/log/delta.log

执行命令

 
 
  
  crontab -e
 
 
 
 
  
  写入
 
 
 
 
  
  */5 * * * * /usr/local/coreseek/init/delta.sh
00 03 * * * /usr/local/coreseek/init/main.sh