python相似性检测的安装包

最新推荐文章于 2024-08-15 08:45:00 发布

Johline

最新推荐文章于 2024-08-15 08:45:00 发布

阅读量6.2k

点赞数

分类专栏： python相似度

python相似度专栏收录该内容

7 篇文章 0 订阅

订阅专栏

安装python-Levenshtein模块

pip install python-Levenshtein

使用python-Levenshtein模块

import Levenshtein

算法说明

1). Levenshtein.hamming(str1, str2)
计算汉明距离。要求str1和str2必须长度一致。是描述两个等长字串之间对应 位置上不同字符的个数。

2). Levenshtein.distance(str1, str2)
计算编辑距离（也称为 Levenshtein距离）。是描述由一个字串转化成另一个字串最少的操作次数，在其中的操作包括插入、删除、替换。
算法实现参考动态规划整理。

3). Levenshtein.ratio(str1, str2)
计算莱文斯坦比。计算公式r = (sum - ldist) / sum, 其中sum是指str1 和 str2 字串的长度总和，ldist是 类编辑距离
注意 ：这里的类编辑距离不是2中所说的编辑距离，2中三种操作中每个操作+1，而在此处，删除、插入依然+1，但是替换+2
这样设计的目的：ratio('a', 'c')，sum=2, 按2中计算为（2-1）/2 = 0.5,’a','c'没有重合，显然不合算，但是替换操作+2，就可以解决这个问题。

4). Levenshtein.jaro(s1 , s2 )
计算jaro距离，

其中的 m 为s1 , s2的匹配长度，当某位置的认为匹配当该位置字符相同，或者在不超过

t是调换次数的一半

5.) Levenshtein.jaro_winkler(s 1 , s 2 )
计算 Jaro–Winkler距离：

import Levenshtein 报错：ImportError: No module named Levenshtein
于是去： python-Levenshtein 下载源码进行安装（在 http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-levenshtein其实也有编译好的exe），第一次安装的时候报错：error: Unable to find vcvarsall.bat ，但其实我是装了VS2010的，所以执行如下步骤正常安装：
1.设置环境变量，执行：
SET VS90COMNTOOLS=%VS100COMNTOOLS%
2.再去安装：
setup.py install
就可以正常，编译，安装了。
 
 
  
  $ python
>>> import Levenshtein
>>> help(Levenshtein.ratio)
ratio(...)
    Compute similarity of two strings.

    ratio(string1, string2)

    The similarity is a number between 0 and 1, it's usually equal or
    somewhat higher than difflib.SequenceMatcher.ratio(), becuase it's
    based on real minimal edit distance.

    Examples:
    >>> ratio('Hello world!', 'Holly grail!')
    0.58333333333333337
    >>> ratio('Brian', 'Jesus')
    0.0

>>> help(Levenshtein.distance)
distance(...)
    Compute absolute Levenshtein distance of two strings.

    distance(string1, string2)

    Examples (it's hard to spell Levenshtein correctly):
    >>> distance('Levenshtein', 'Lenvinsten')
    4
    >>> distance('Levenshtein', 'Levensthein')
    2
    >>> distance('Levenshtein', 'Levenshten')
    1
    >>> distance('Levenshtein', 'Levenshtein')
    0

 
 difflib 库
>>> import difflib

>>> difflib.SequenceMatcher(None, 'abcde', 'abcde').ratio()
1.0

>>> difflib.SequenceMatcher(None, 'abcde', 'zbcde').ratio()
0.80000000000000004

>>> difflib.SequenceMatcher(None, 'abcde', 'zyzzy').ratio()
0.0
 
FuzzyWuzzy
   
   
    
    git clone git://github.com/seatgeek/fuzzywuzzy.git fuzzywuzzy
cd fuzzywuzzy
python setup.py install

>>> from fuzzywuzzy import fuzz
>>> from fuzzywuzzy import process

Simple Ratio
>>> fuzz.ratio("this is a test", "this is a test!")
    96

Partial Ratio
>>> fuzz.partial_ratio("this is a test", "this is a test!")
    100

Token Sort Ratio
>>> fuzz.ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")
    90
>>> fuzz.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")
    100

Token Set Ratio
>>> fuzz.token_sort_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")
    84
>>> fuzz.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")
    100
    
    gitclone git://github.com/seatgeek/fuzzywuzzy.git fuzzywuzzy
cdfuzzywuzzy
pythonsetup.pyinstall
 
>>> fromfuzzywuzzyimportfuzz
>>> fromfuzzywuzzyimportprocess
 
SimpleRatio
>>> fuzz.ratio("this is a test", "this is a test!")
    96
 
PartialRatio
>>> fuzz.partial_ratio("this is a test", "this is a test!")
    100
 
TokenSortRatio
>>> fuzz.ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")
    90
>>> fuzz.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")
    100
 
TokenSetRatio
>>> fuzz.token_sort_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")
    84
>>> fuzz.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")
    100

   
   
google-diff-match-patch
   
   
    
    
     
     import diff match patch textA = "the cat in the red hat" textB = "the feline in the blue hat"
     
     dmp = diff match patch.diff match patch()  #create a diff match patch object diffs = dmp.diff main(textA, textB)   # All 'diff' jobs start with invoking diff main()
     
     d value = dmp.diff levenshtein(diffs) print d_value
     
     maxLenth = max(len(textA), len(textB)) print float(d_value)/float(maxLenth)
     
     similarity = (1 - float(d_value)/float(maxLenth)) * 100 print similarity
    
    
    
    importdiff_match_patch
textA = "the cat in the red hat"
textB = "the feline in the blue hat"
 
dmp = diff_match_patch.diff_match_patch()  #create a diff_match_patch object
diffs = dmp.diff_main(textA, textB)   # All 'diff' jobs start with invoking diff_main()
 
d_value = dmp.diff_levenshtein(diffs)
printd_value
 
maxLenth = max(len(textA), len(textB))
printfloat(d_value)/float(maxLenth)
 
similarity = (1 - float(d_value)/float(maxLenth)) * 100
printsimilarity
   
   
title2
 
 
  
  第二种方法安装
 
 
 
 
  
  参考博客：http://blog.csdn.net/TH_NUM/article/details/77095177
 
 
 
 
  
  安装pip install  
  
  python-Levenshtein,出现错误：Microsoft Visual C++ 14.0 is required
 
 
 
 
  
  出现错误主要是因为直接使用 pip install 【第三方库名】 安装自己需要的第三方库。
 
 
 
 
  
  解决办法：
 
 
 
 
  
  一定要安装和自己windows版本和python版本对应的第三方库

 
 
 
 
  
  在这里下载需要的第三方库：http://www.lfd.uci.edu/~gohlke/pythonlibs

 
 
 
 
  
  安装步骤：
 
 
 
 
  
  首先：在网站上下载对于版本的 python_Levenshtein-0.12.0-cp36-cp36m-win_amd64.whl
 
 
 
 
  
  然后：然后在控制台上切换到放置位置后输入pip install python_Levenshtein-0.12.0-cp36-cp36m-win_amd64.whl
 
 
 
 
  
  就完成了！！！