matlab源代码 语义相似度计算,如何计算两个句子之间的相似度（句法和语义）...

我应该每次取两个句子并计算它们是否相似。我的意思是，在语法和语义上都是这样。如何计算两个句子之间的相似度(句法和语义)

INPUT1：奥巴马签署法律。奥巴马签署了一项新法律。

INPUT2： 总线停在这里。 车辆停在这里。

INPUT3：纽约的火灾。 纽约被烧毁。

INPUT4：在纽约的火灾。在纽约大火中死亡50人。

我不想用本体树作为灵魂。我写了一个代码来计算句子之间Levenshtein distance(LD)，然后决定是否第二个句子：

可以忽略不计(INPUT1和2)，

应更换的第一句话(INPUT 3)，或

与第一句(INPUT4)一起存储。

我对代码不满意，因为LD只计算语法级别(还有其他什么方法？)。语义如何融入(比如公交车就像是一辆车？)。

的代码放在这里：

%# As the difference is computed, a decision is made on the new event

%# (string 2) to be ignored, to replace existing event (string 1) or to be

%# stored separately. The higher the LD metric, the higher the difference

%# between two strings. Of course, lower difference indices either identical

%# or similar events. However, the higher difference indicates the new event

%# as a fresh event.

%#.........................................................................

%# Calculating the LD between two strings of events.

%#.........................................................................

L1=length(str1)+1;

L2=length(str2)+1;

L=zeros(L1,L2); %# Initializing the new length.

g=+1; %# just constant

m=+0; %# match is cheaper, we seek to minimize

d=+1; %# not-a-match is more costly.

% do BC's

L(:,1)=([0:L1-1]*g)';

L(1,:)=[0:L2-1]*g;

m4=0; %# loop invariant

%# Calculating required edits.

for idx=2:L1;

for idy=2:L2

if(str1(idx-1)==str2(idy-1))

score=m;

else

score=d;

end

m1=L(idx-1,idy-1) + score;

m2=L(idx-1,idy) + g;

m3=L(idx,idy-1) + g;

L(idx,idy)=min(m1,min(m2,m3)); % only minimum edits allowed.

end

end

%# The LD between two strings.

D=L(L1,L2);

%#....................................................................

%# Making decision on what to do with the new event (string 2).

%#...................................................................

if (D<=4) %# Distance is so less that string 2 seems identical to string 1.

store=str1; %# Hence string 2 is ignored. String 1 remains stored.

elseif (D>=5 && D<=15) %# Distance is larger to be identical but not enough to

%# make string 2 an individual event.

store= str2; %# String 2 is somewhat similar to string 1.

%# So, string 1 is replaced with string 2 and stored.

else

%# For all other distances, string 2 is stored along with string 1.

store={str1; str2};

end

任何帮助表示赞赏。

2010-09-07

Tinglin

+0

“语义上”。没有简单的文本书算法。自然语言(特别是英语)是一个非常复杂而反复无常的野兽。 –

2010-09-07 22:16:49

+0

@Amro：“'＃'”使它们变灰，因为这里的注释是SO？ –

2010-09-14 08:41:33

+0

@Lazer：是的，它的眼睛更容易..我希望StackOverflow引入了包含代码块的功能，如：'...'，以便为该特定语言正确突出显示 –

2010-09-14 15:54:46