Django之站内搜索-Solr,Haystack

最新推荐文章于 2020-03-11 16:12:07 发布

weixin_34216107

最新推荐文章于 2020-03-11 16:12:07 发布

阅读量272

点赞数

文章标签： python java 开发工具

原文链接：http://www.cnblogs.com/similarface/p/5412701.html

版权

java -version 不多说 solr 是java 开发的

java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

Download Solr:

http://archive.apache.org/dist/lucene/solr/

下载－－》解压－－》solr-4.10.4/example

SimilarFacedeMacBook-Pro:example similarface$ java -jar start.jar

长长的log我也是醉了

图形化真好看

SimilarFacedeMacBook-Pro:solr similarface$ pwd
/Users/similarface/Downloads/solr-4.10.4/example/solr
SimilarFacedeMacBook-Pro:solr similarface$ tree newblog/
newblog/
├── conf
│   ├── _rest_managed.json
│   ├── lang
│   │   └── stopwords_en.txt
│   ├── protwords.txt
│   ├── schema.xml
│   ├── solrconfig.xml
│   ├── stopwords.txt
│   └── synonyms.txt
├── core.properties
└── data

SimilarFacedeMacBook-Pro:solr similarface$ cat newblog/conf/solrconfig.xml
<?xml version="1.0" encoding="utf-8" ?>
<config>
    <luceneMatchVersion>LUCENE_36</luceneMatchVersion>
        <requestHandler name="/select" class="solr.StandardRequestHandler"
                            default="true"/>
                                <requestHandler name="/update" class="solr.UpdateRequestHandler"/>
                                    <requestHandler name="/admin" class="solr.admin.AdminHandlers"/>
                                        <requestHandler name="/admin/ping" class="solr.PingRequestHandler">
                                                <lst name="invariants">
                                                            <str name="qt">search</str>
                                                                        <str name="q">*:*</str>
                                                                                </lst>
                                                                                    </requestHandler>
                                                                                    </config>

schema.xml默认写下面的：

<?xml version="1.0" ?>
   <schema name="default" version="1.5">
   </schema>

上面怎么做，bitch ，mkdir and vim

pip install django-haystack==2.4.0

pip install pysolr==3.3.2

settings.py

［haystack ,HAYSTACK_CONNECTIONS］

INSTALLED_APPS = [
　　 ...
    'haystack',
    ...
]

HAYSTACK_CONNECTIONS = {
       'default': {
           'ENGINE': 'haystack.backends.solr_backend.SolrEngine',
           'URL': 'http://127.0.0.1:8983/solr/myblog'
       },
}

#coding:utf-8
__author__ = 'similarface'

from haystack import indexes
from .models import Post
class PostIndex(indexes.SearchIndex,indexes.Indexable):
    '''
    给文章Post 建立索引
    '''
    text = indexes.CharField(document=True, use_template=True)
    publish = indexes.DateTimeField(model_attr='publish')
    def get_model(self):
        return Post
    def index_queryset(self, using=None):
        return self.get_model().published.all()

SimilarFacedeMacBook-Pro:templates similarface$ tree search/
search/
└── indexes
    └── myblog
        └── post_text.txt

SimilarFacedeMacBook-Pro:templates similarface$ pwd
/Users/similarface/PycharmProjects/StudyDjango/myblog/templates
SimilarFacedeMacBook-Pro:templates similarface$ cat search/indexes/myblog/post_text.txt
{{ object.title }}
{{ object.tags.all|join:", " }}
{{ object.content }}

SimilarFacedeMacBook-Pro:StudyDjango similarface$ python manage.py build_solr_schema
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pytz/__init__.py:29: UserWarning: Module email was already imported from /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/__init__.pyc, but /Library/Python/2.7/site-packages is being added to sys.path
  from pkg_resources import resource_stream
/Library/Python/2.7/site-packages/django/core/management/base.py:265: RemovedInDjango110Warning: OptionParser usage for Django management commands is deprecated, use ArgumentParser instead
  RemovedInDjango110Warning)

/Library/Python/2.7/site-packages/haystack/management/commands/build_solr_schema.py:56: RemovedInDjango110Warning: render() must be called with a dict, not a Context.
  return t.render(c)




Save the following output to 'schema.xml' and place it in your Solr configuration directory.
--------------------------------------------------------------------------------------------

<?xml version="1.0" ?>
<!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<schema name="default" version="1.5">
  <types>
    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
    <fieldtype name="binary" class="solr.BinaryField"/>

    <!-- Numeric field types that manipulate the value into
         a string value that isn't human-readable in its internal form,
         but with a lexicographic ordering the same as the numeric ordering,
         so that range queries work correctly. -->
    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
    <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
    <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
    <fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
    <fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
    <fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
    <fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>

    <fieldType name="tint" class="solr.TrieIntField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
    <fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
    <fieldType name="tlong" class="solr.TrieLongField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>

    <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
    <!-- A Trie based date field for faster date range queries and date faceting. -->
    <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>

    <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
    <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
    <fieldtype name="geohash" class="solr.GeoHashField"/>

    <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
        <!-- in this example, we will only use synonyms at query time
        <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
        -->
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>

    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
      <analyzer type="index">
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="lang/stopwords_en.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.EnglishPossessiveFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
          <filter class="solr.EnglishMinimalStemFilterFactory"/>
        -->
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="lang/stopwords_en.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.EnglishPossessiveFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
          <filter class="solr.EnglishMinimalStemFilterFactory"/>
        -->
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
    </fieldType>

    <fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
      <analyzer>
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
      </analyzer>
    </fieldType>

    <fieldType name="ngram" class="solr.TextField" >
      <analyzer type="index">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" />
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>

    <fieldType name="edge_ngram" class="solr.TextField" positionIncrementGap="1">
      <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front" />
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory" />
        <filter class="solr.LowerCaseFilterFactory" />
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
      </analyzer>
    </fieldType>
  </types>

  <fields>
    <!-- general -->
    <field name="id" type="string" indexed="true" stored="true" multiValued="false" required="true"/>
    <field name="django_ct" type="string" indexed="true" stored="true" multiValued="false"/>
    <field name="django_id" type="string" indexed="true" stored="true" multiValued="false"/>
    <field name="_version_" type="long" indexed="true" stored ="true"/>

    <dynamicField name="*_i"  type="int"    indexed="true"  stored="true"/>
    <dynamicField name="*_s"  type="string"  indexed="true"  stored="true"/>
    <dynamicField name="*_l"  type="long"   indexed="true"  stored="true"/>
    <dynamicField name="*_t"  type="text_en"    indexed="true"  stored="true"/>
    <dynamicField name="*_b"  type="boolean" indexed="true"  stored="true"/>
    <dynamicField name="*_f"  type="float"  indexed="true"  stored="true"/>
    <dynamicField name="*_d"  type="double" indexed="true"  stored="true"/>
    <dynamicField name="*_dt" type="date" indexed="true" stored="true"/>
    <dynamicField name="*_p" type="location" indexed="true" stored="true"/>
    <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>


    <field name="text" type="text_en" indexed="true" stored="true" multiValued="false" />

    <field name="publish" type="date" indexed="true" stored="true" multiValued="false" />

  </fields>

  <!-- field to use to determine and enforce document uniqueness. -->
  <uniqueKey>id</uniqueKey>

  <!-- field for the QueryParser to use when an explicit fieldname is absent -->
  <defaultSearchField>text</defaultSearchField>

  <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
  <solrQueryParser defaultOperator="AND"/>
</schema>



#Save the following output to 'schema.xml' and place it in your Solr configuration directory.

vi ~/Downloads/solr-4.10.4/example/solr/myblog/conf/schema.xml

SimilarFacedeMacBook-Pro:StudyDjango similarface$ python manage.py rebuild_index
 from pkg_resources import resource_stream
/Library/Python/2.7/site-packages/django/core/management/base.py:265: RemovedInDjango110Warning: OptionParser usage for Django management commands is deprecated, use ArgumentParser instead
  RemovedInDjango110Warning)


WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the `rebuild_index` command.
Are you sure you wish to continue? [y/N] y
Removing all documents from your index because you said so.
Failed to clear Solr index: [Reason: Error 404 Not Found]
All documents removed.
Indexing 8 posts
/Library/Python/2.7/site-packages/haystack/fields.py:137: RemovedInDjango110Warning: render() must be called with a dict, not a Context.
  return t.render(Context({'object': obj}))

Failed to add documents to Solr: [Reason: Error 404 Not Found]