简单字符串匹配问题-——用哈希解决

Pattern matching is the most fundamental algorithmic operation on text strings.This algorithm implements the find command available in any web browser or text

editor:

Problem: Substring Pattern Matching
Input: A text string t and a pattern string p.

Output: Does t contain the pattern p as a substring, and if so where?

      

基本解法：

int findmatch(char *p, char *t)
{
      int i,j;                                                                                      
      int m, n;                                                                      
      m = strlen(p);
      n = strlen(t);
      for (i=0; i<=(n-m); i=i+1) {
      j=0;
      while ((j<m) && (t[i+j]==p[j]))
            j = j+1;
            if (j == m) return(i);
      }
      return(-1);
}

Hashing and Strings

Hash tables are a very practical way to maintain a dictionary. They exploit the fact
that looking an item up in an array takes constant time once you have its index. A
hash function is a mathematical function that maps keys to integers. We will use
the value of our hash function as an index into an array, and store our item at that
position.

The first step of the hash function is usually to map each key to a big integer.
Let α be the size of the alphabet on which a given string S is written. Let char(c)
be a function that maps each symbol of the alphabet to a unique integer from 0 to
α − 1. The function

maps each string to a unique (but large) integer by treating the characters of the
string as “digits” in a base-α number system.

Efficient String Matching via Hashing

Strings are sequences of characters where the order of the characters matters, since
ALGORITHM is different than LOGARITHM. Text strings are fundamental to a
host of computing applications, from programming language parsing/compilation,
to web search engines, to biological sequence analysis.

The primary data structure for representing strings is an array of characters.
This allows us constant-time access to the ith character of the string. Some auxiliary
information must be maintained to mark the end of the string—either a special
end-of-string character or (perhaps more usefully) a count of the n characters in
the string.

The most fundamental operation on text strings is substring search, namely:

Problem: Substring Pattern Matching
Input: A text string t and a pattern string p.
Output: Does t contain the pattern p as a substring, and if so where?

The simplest algorithm to search for the presence of pattern string p in text t
overlays the pattern string at every position in the text, and checks whether every
pattern character matches the corresponding text character. As demonstrated in
Section 2.5.3 (page 43), this runs in O(nm) time, where n = |t| and m = |p|.

This quadratic bound is worst-case. More complicated, worst-case linear-time
search algorithms do exist: see Section 18.3 (page 628) for a complete discussion.
But here we give a linear expected-time algorithm for string matching, called the
Rabin-Karp algorithm. It is based on hashing. Suppose we compute a given hash
function on both the pattern string p and the m-character substring starting from
the ith position of t. If these two strings are identical, clearly the resulting hash
values must be the same. If the two strings are different, the hash values will
almost certainly be different. These false positives should be so rare that we can
easily spend the O(m) time it takes to explicitly check the identity of two strings
whenever the hash values agree.

This reduces string matching to n−m+2 hash value computations (the n−m+1
windows of t, plus one hash of p), plus what should be a very small number of O(m)
time verification steps. The catch is that it takes O(m) time to compute a hash
function on an m-character string, and O(n) such computations seems to leave us
with an O(mn) algorithm again.

But let’s look more closely at our previously defined hash function, applied to
the m characters starting from the jth position of string S:

What changes if we now try to compute H(S, j + 1)—the hash of the next
window of m characters? Note that m−1 characters are the same in both windows,
although this differs by one in the number of times they are multiplied by α. A
little algebra reveals that

This means that once we know the hash value from the j position, we can find
the hash value from the (j + 1)st position for the cost of two multiplications, one
addition, and one subtraction. This can be done in constant time (the value of
αm−1 can be computed once and used for all hash value computations). This math
works even if we compute H(S, j) mod M, where M is a reasonably large prime
number, thus keeping the size of our hash values small (at most M) even when the
pattern string is long.

Rabin-Karp is a good example of a randomized algorithm (if we pick M in some
random way).We get no guarantee the algorithm runs in O(n+m) time, because we
may get unlucky and have the hash values regularly collide with spurious matches.
Still, the odds are heavily in our favor—if the hash function returns values uniformly
from 0 to M − 1, the probability of a false collision should be 1/M. This is quite
reasonable: if M ≈ n, there should only be one false collision per string, and if
M ≈ nk for k ≥ 2, the odds are great we will never see any false collisions.

Duplicate Detection Via Hashing

The key idea of hashing is to represent a large object (be it a key, a string, or a
substring) using a single number. The goal is a representation of the large object
by an entity that can be manipulated in constant time, such that it is relatively
unlikely that two different large objects map to the same value.

Hashing has a variety of clever applications beyond just speeding up search. I
once heard Udi Manber—then Chief Scientist at Yahoo—talk about the algorithms
employed at his company. The three most important algorithms at Yahoo, he said,
were hashing, hashing, and hashing.

Consider the following problems with nice hashing solutions:

• Is a given document different from all the rest in a large corpus? – A search
engine with a huge database of n documents spiders yet another webpage.
How can it tell whether this adds something new to add to the database, or
is just a duplicate page that exists elsewhere on the Web?

Explicitly comparing the new document D to all n documents is hopelessly
inefficient for a large corpus. But we can hash D to an integer, and compare
it to the hash codes of the rest of the corpus. Only when there is a collision
is D a possible duplicate. Since we expect few spurious collisions, we can
explicitly compare the few documents sharing the exact hash code with little
effort.

•Is part of this document plagiarized from a document in a large corpus? – A
lazy student copies a portion of a Web document into their term paper. “The
Web is a big place,” he smirks. “How will anyone ever find which one?”

This is a more difficult problem than the previous application. Adding, deleting,
or changing even one character from a document will completely change
its hash code. Thus the hash codes produced in the previous application
cannot help for this more general problem.

However, we could build a hash table of all overlapping windows (substrings)
of length w in all the documents in the corpus. Whenever there is a match of
hash codes, there is likely a common substring of length w between the two
documents, which can then be further investigated. We should choose w to
be long enough so such a co-occurrence is very unlikely to happen by chance.

The biggest downside of this scheme is that the size of the hash table becomes
as large as the documents themselves. Retaining a small but well-chosen
subset of these hash codes (say those which are exact multiples of 100) for
each document leaves us likely to detect sufficiently long duplicate strings.

• How can I convince you that a file isn’t changed? – In a closed-bid auction,
each party submits their bid in secret before the announced deadline. If you
knew what the other parties were bidding, you could arrange to bid $1 more
than the highest opponent and walk off with the prize as cheaply as possible.
Thus the “right” auction strategy is to hack into the computer containing
the bids just prior to the deadline, read the bids, and then magically emerge
the winner.

How can this be prevented? What if everyone submits a hash code of their
actual bid prior to the deadline, and then submits the full bid after the deadline?
The auctioneer will pick the largest full bid, but checks to make sure the
hash code matches that submitted prior to the deadline. Such cryptographic
hashing methods provide a way to ensure that the file you give me today is
the same as original, because any changes to the file will result in changing
the hash code.

Although the worst-case bounds on anything involving hashing are dismal, with
a proper hash function we can confidently expect good behavior. Hashing is a fundamental
idea in randomized algorithms, yielding linear expected-time algorithms
for problems otherwise Θ(n log n), or Θ(n2) in the worst case.