一般要对文件做校验,比较多的使用的时md5sum ,sha1sum来做,列如:
gavingeng@gavingeng:~/tmp$ md5sum access.log b8a5a5440b2291d5c90e550dba23454f access.log gavingeng@gavingeng:~/tmp$ sha1sum access.log c230f166ad459b2c5bff77147606e45a59163b49 access.log gavingeng@gavingeng:~/tmp$ md5sum access.log |awk '{print $1}' b8a5a5440b2291d5c90e550dba23454f
可以跟redis/memcached等nosql服务来一起使用,key为md5值,value为文件名以及文件全路径