使用clonedigger来检查python中的重复代码

安装Clonedigger

$ sudo pip install clonedigger
$ clonedigger -help
Usage: To run Clone Digger type:
python clonedigger.py [OPTION]... [SOURCE FILE OR DIRECTORY]...

The typical usage is:
python clonedigger.py source_file_1 source_file_2 ...
  or
python clonedigger.py path_to_source_tree
Don't forget to remove automatically generated sources, tests and third party libraries from the source tree.

Notice:
The semantics of threshold options is discussed in the paper "Duplicate code detection using anti-unification", which can be downloaded from the site http://clonedigger.sourceforge.net . All arguments are optional. Supported options are:


Options:
  -h, --help            show this help message and exit
  -l LANGUAGE, --language=LANGUAGE
                        the programming language
  --no-recursion        do not traverse directions recursively
  -o OUTPUT, --output=OUTPUT
                        the name of the output file ("output.html" by default)
  --clustering-threshold=CLUSTERING_THRESHOLD
                        read the paper for semantics
  --distance-threshold=DISTANCE_THRESHOLD
                        the maximum amount of differences between pair of
                        sequences in clone pair (5 by default). Larger value
                        leads to larger amount of false positives
  --hashing-depth=HASHING_DEPTH
                        default value if 1, read the paper for semantics.
                        Computation can be speeded up by increasing this value
                        (but some clones can be missed)
  --size-threshold=SIZE_THRESHOLD
                        the minimum clone size. The clone size for its turn is
                        equal to the count of lines of code in its the largest
                        fragment
  --clusterize-using-dcup
                        mark each statement with its D-cup value instead of
                        the most similar pattern. This option together with
                        --hashing-depth=0 make it possible to catch all
                        considered clones (but it is slow and applicable only
                        to small programs)
  --dont-print-time     do not print time
  -f, --force
  --force-diff          force highlighting of differences based on the diff
                        algorithm
  --fast                find only clones, which differ in variable and
                        function names and constants
  --ignore-dir=IGNORE_DIRS
                        exclude directories from parsing
  --eclipse-output=ECLIPSE_OUTPUT
                        for internal usage only
  --cpd-output          output as PMDs CPDs XML format. If output file not
                        defined, output.xml is generated
  --report-unifiers
  --func-prefixes=F_PREFIXES
                        skip functions/methods with these prefixes (provide a
                        CSV string as argument)
  --file-list=FILE_LIST
                        a file that contains a list of file names that must be
                        processed by Clone Digger

使用Clonedigger

clonedigger -l python --ignore-dir=/path_to/exclude_file -o ./output.html -f --size-threshold=10 /path_to/src/

集成到Jenkins，使用violations插件生成trend图

$ clonedigger  --cpd-output -o ./cpdout.xml  /path_to/src/