zoukankan      html  css  js  c++  java
  • django-全文检索

    全文检索不同于特定字段的模糊查询,使用全文检索的效率更高,并且能够对于中文进行分词处理

    需要的第三方库:

    • haystack:django的一个包,可以方便地对model里面的内容进行索引、搜索,设计为支持whoosh,solr,Xapian,Elasticsearc四种全文检索引擎后端,属于一种全文检索的框架
    • whoosh:纯Python编写的全文搜索引擎,虽然性能比不上sphinx、xapian、Elasticsearc等,但是无二进制包,程序不会莫名其妙的崩溃,对于小型的站点,whoosh已经足够使用
    • jieba:一款免费的中文分词包

    操作

    首先pip安装包

    pip install django-haystack
    pip install whoosh
    pip install jieba

    设置settings

    添加应用:

    INSTALLED_APPS = (
        ...
        'haystack',
    )

    添加搜索引擎:

    HAYSTACK_CONNECTIONS = {
        'default': {
            'ENGINE': 'haystack.backends.whoosh_cn_backend.WhooshEngine',
            'PATH': os.path.join(BASE_DIR, 'whoosh_index'),
        }
    }
    
    #自动生成索引
    HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
    #每一页显示多少数据
    HAYSTACK_SEARCH_RESULTS_PER_PAGE = 18
     

    添加url:

    urlpatterns = [
        ...
        url(r'^search/', include('haystack.urls')),
    ]

    在应用目录下建立search_indexes.py

    # coding=utf-8
    from haystack import indexes
    from models import GoodsInfo
    
    
    class GoodsInfoIndex(indexes.SearchIndex, indexes.Indexable):
        text = indexes.CharField(document=True, use_template=True)
    
        def get_model(self):
            return GoodsInfo
    
        def index_queryset(self, using=None):
            return self.get_model().objects.all()

    在目录“templates/search/indexes/应用名称/”下创建“模型类名称_text.txt”文件

    #goodsinfo_text.txt,这里列出了要对哪些列的内容进行检索,模型类中的某些字段
    {{ object.gName }}
    {{ object.gSubName }}
    {{ object.gDes }}

    在目录“templates/search/”下建立search.html

    <!DOCTYPE html>
    <html>
    <head>
        <title></title>
    </head>
    <body>
    {% if query %}
        <h3>搜索结果如下:</h3>
        {% for result in page.object_list %}
            <a href="/{{ result.object.id }}/">{{ result.object.gName }}</a><br/>
        {% empty %}
            <p>没找到</p>
        {% endfor %}
    
        {% if page.has_previous or page.has_next %}
            <div>
                {% if page.has_previous %}<a href="?q={{ query }}&amp;page={{ page.previous_page_number }}">{% endif %}&laquo; 上一页{% if page.has_previous %}</a>{% endif %}
            |
                {% if page.has_next %}<a href="?q={{ query }}&amp;page={{ page.next_page_number }}">{% endif %}下一页 &raquo;{% if page.has_next %}</a>{% endif %}
            </div>
        {% endif %}
    {% endif %}
    </body>
    </html>

    建立ChineseAnalyzer.py文件

    保存在haystack的安装文件夹下,路径如“/home/python/.virtualenvs/django_py2/lib/python2.7/site-packages/haystack/backends”

    import jieba
    from whoosh.analysis import Tokenizer, Token
    
    
    class ChineseTokenizer(Tokenizer):
        def __call__(self, value, positions=False, chars=False,
                     keeporiginal=False, removestops=True,
                     start_pos=0, start_char=0, mode='', **kwargs):
            t = Token(positions, chars, removestops=removestops, mode=mode,
                      **kwargs)
            seglist = jieba.cut(value, cut_all=True)
            for w in seglist:
                t.original = t.text = w
                t.boost = 1.0
                if positions:
                    t.pos = start_pos + value.find(w)
                if chars:
                    t.startchar = start_char + value.find(w)
                    t.endchar = start_char + value.find(w) + len(w)
                yield t
    
    
    def ChineseAnalyzer():
        return ChineseTokenizer()

    复制whoosh_backend.py文件,改名为whoosh_cn_backend.py

    from .ChineseAnalyzer import ChineseAnalyzer 
    查找
    analyzer=StemmingAnalyzer()
    改为
    analyzer=ChineseAnalyzer()

    生成索引

    初始化索引:

    python manage.py rebuild_index

    在模板中创建搜索栏

    <form method='get' action="/search/" target="_blank">
        <input type="text" name="q">
        <input type="submit" value="查询">
    </form>

    关于全文索引使用的固定参数一些说明:

    我们打开haystack第三方包中的urls文件

    haystack
        ----urls.py        
    
    
    # encoding: utf-8
    
    from __future__ import absolute_import, division, print_function, unicode_literals
    
    from django.conf.urls import url
    
    from haystack.views import SearchView
    
    urlpatterns = [
        url(r'^$', SearchView(), name='haystack_search'),
    ]

    然后进入haystack.views 文件

    #这里将搜索显示的数据默认为20个
    RESULTS_PER_PAGE = getattr(settings, 'HAYSTACK_SEARCH_RESULTS_PER_PAGE', 20)
    #在search文件下的search template
    = 'search/search.html' extra_context = {} query = '' results = EmptySearchQuerySet() request = None form = None results_per_page = RESULTS_PER_PAGE

    更多详情请看

    haystack.views.py

    # encoding: utf-8
    
    from __future__ import absolute_import, division, print_function, unicode_literals
    
    from django.conf import settings
    from django.core.paginator import InvalidPage, Paginator
    from django.http import Http404
    from django.shortcuts import render
    
    from haystack.forms import FacetedSearchForm, ModelSearchForm
    from haystack.query import EmptySearchQuerySet
    
    RESULTS_PER_PAGE = getattr(settings, 'HAYSTACK_SEARCH_RESULTS_PER_PAGE', 20)
    
    
    class SearchView(object):
        template = 'search/search.html'
        extra_context = {}
        query = ''
        results = EmptySearchQuerySet()
        request = None
        form = None
        results_per_page = RESULTS_PER_PAGE
    
        def __init__(self, template=None, load_all=True, form_class=None, searchqueryset=None, results_per_page=None):
            self.load_all = load_all
            self.form_class = form_class
            self.searchqueryset = searchqueryset
    
            if form_class is None:
                self.form_class = ModelSearchForm
    
            if not results_per_page is None:
                self.results_per_page = results_per_page
    
            if template:
                self.template = template
    
        def __call__(self, request):
            """
            Generates the actual response to the search.
    
            Relies on internal, overridable methods to construct the response.
            """
            self.request = request
    
            self.form = self.build_form()
            self.query = self.get_query()
            self.results = self.get_results()
    
            return self.create_response()
    
        def build_form(self, form_kwargs=None):
            """
            Instantiates the form the class should use to process the search query.
            """
            data = None
            kwargs = {
                'load_all': self.load_all,
            }
            if form_kwargs:
                kwargs.update(form_kwargs)
    
            if len(self.request.GET):
                data = self.request.GET
    
            if self.searchqueryset is not None:
                kwargs['searchqueryset'] = self.searchqueryset
    
            return self.form_class(data, **kwargs)
    
        def get_query(self):
            """
            Returns the query provided by the user.
    
            Returns an empty string if the query is invalid.
            """
            if self.form.is_valid():
                return self.form.cleaned_data['q']
    
            return ''
    
        def get_results(self):
            """
            Fetches the results via the form.
    
            Returns an empty list if there's no query to search with.
            """
            return self.form.search()
    
        def build_page(self):
            """
            Paginates the results appropriately.
    
            In case someone does not want to use Django's built-in pagination, it
            should be a simple matter to override this method to do what they would
            like.
            """
            try:
                page_no = int(self.request.GET.get('page', 1))
            except (TypeError, ValueError):
                raise Http404("Not a valid number for page.")
    
            if page_no < 1:
                raise Http404("Pages should be 1 or greater.")
    
            start_offset = (page_no - 1) * self.results_per_page
            self.results[start_offset:start_offset + self.results_per_page]
    
            paginator = Paginator(self.results, self.results_per_page)
    
            try:
                page = paginator.page(page_no)
            except InvalidPage:
                raise Http404("No such page!")
    
            return (paginator, page)
    
        def extra_context(self):
            """
            Allows the addition of more context variables as needed.
    
            Must return a dictionary.
            """
            return {}
    
        def get_context(self):
            (paginator, page) = self.build_page()
    
            context = {
                'query': self.query,
                'form': self.form,
                'page': page,
                'paginator': paginator,
                'suggestion': None,
            }
    
            if hasattr(self.results, 'query') and self.results.query.backend.include_spelling:
                context['suggestion'] = self.form.get_suggestion()
    
            context.update(self.extra_context())
    
            return context
    
        def create_response(self):
            """
            Generates the actual HttpResponse to send back to the user.
            """
    
            context = self.get_context()
    
            return render(self.request, self.template, context)
    
    
    def search_view_factory(view_class=SearchView, *args, **kwargs):
        def search_view(request):
            return view_class(*args, **kwargs)(request)
        return search_view
    
    
    class FacetedSearchView(SearchView):
        def __init__(self, *args, **kwargs):
            # Needed to switch out the default form class.
            if kwargs.get('form_class') is None:
                kwargs['form_class'] = FacetedSearchForm
    
            super(FacetedSearchView, self).__init__(*args, **kwargs)
    
        def build_form(self, form_kwargs=None):
            if form_kwargs is None:
                form_kwargs = {}
    
            # This way the form can always receive a list containing zero or more
            # facet expressions:
            form_kwargs['selected_facets'] = self.request.GET.getlist("selected_facets")
    
            return super(FacetedSearchView, self).build_form(form_kwargs)
    
        def extra_context(self):
            extra = super(FacetedSearchView, self).extra_context()
            extra['request'] = self.request
            extra['facets'] = self.results.facet_counts()
            return extra
    
    
    def basic_search(request, template='search/search.html', load_all=True, form_class=ModelSearchForm, searchqueryset=None, extra_context=None, results_per_page=None):
        """
        A more traditional view that also demonstrate an alternative
        way to use Haystack.
    
        Useful as an example of for basing heavily custom views off of.
    
        Also has the benefit of thread-safety, which the ``SearchView`` class may
        not be.
    
        Template:: ``search/search.html``
        Context::
            * form
              An instance of the ``form_class``. (default: ``ModelSearchForm``)
            * page
              The current page of search results.
            * paginator
              A paginator instance for the results.
            * query
              The query received by the form.
        """
        query = ''
        results = EmptySearchQuerySet()
    
        if request.GET.get('q'):
            form = form_class(request.GET, searchqueryset=searchqueryset, load_all=load_all)
    
            if form.is_valid():
                query = form.cleaned_data['q']
                results = form.search()
        else:
            form = form_class(searchqueryset=searchqueryset, load_all=load_all)
    
        paginator = Paginator(results, results_per_page or RESULTS_PER_PAGE)
    
        try:
            page = paginator.page(int(request.GET.get('page', 1)))
        except InvalidPage:
            raise Http404("No such page of results!")
    
        context = {
            'form': form,
            'page': page,
            'paginator': paginator,
            'query': query,
            'suggestion': None,
        }
    
        if results.query.backend.include_spelling:
            context['suggestion'] = form.get_suggestion()
    
        if extra_context:
            context.update(extra_context)
    
        return render(request, template, context)
    View Code
  • 相关阅读:
    Windows快捷方式的形式关机方法
    [Andriod官方训练教程]创建你的第一个App之创建一个简单的用户界面
    使用GDAL对HDF数据进行geoloc校正
    破解sqlyog 30天试用限制
    [Andriod官方训练教程]创建你的第一个App之运行你的App
    [Andriod官方训练教程]创建你的第一个App之开始另一个Activity
    GDAL对空间数据的管理
    win linux 下 部署war包到Tomcat根目录
    Spring session
    使用GDAL对HDF数据进行校正
  • 原文地址:https://www.cnblogs.com/welan/p/9231430.html
Copyright © 2011-2022 走看看