edu.umn.cs.spatialHadoop.operations.Main 中关于bin/shadoop generate命令代码如下:
pgd.addClass("generate", RandomSpatialGenerator.class,
"Generates a random file containing spatial data");
可知,该命令实现类为edu.umn.cs.spatialHadoop.RandomSpatialGenerator类。
在RandomSpatialGenerator类的main函数中,首先解析命令行参数,例如官网中的示例:
bin/shadoop generate test mbr:0,0,1000000,1000000 size:1.gb shape:rect
具体代码如下:
//spatialhadoop 实现的自定义类,用于解析命令行参数
CommandLineArguments cla = new CommandLineArguments(args);
//获得随机生成区域的最小包围矩形,其中提示信息有错误,提示信息提示输入参数为mbr:<x,y,w,h> - (*) The MBR of the generated data. Originated at (x,y) with dimensions (w,h)
//实际应为<x1,y1,x2,y2>,代表左下角和右上角坐标
Rectangle mbr = cla.getRectangle();
if (mbr == null) {
printUsage();
throw new RuntimeException("Set MBR of the generated file using rect:<x,y,w,h>");
}
//获得输出文件路径
Path outputFile = cla.getPath();
//获得随机文件生成的图形,可以为点、矩形和多边形
Shape stockShape = cla.getShape(false);
//随机文件块大小
long blocksize = cla.getBlockSize();
//在生成随机矩形时使用,例如random.nextInt(rectSize)
int rectSize = cla.getRectSize();
//使用特定种子生成随机文件
long seed = cla.getSeed();
//默认生成矩形
if (stockShape == null)
stockShape = new Rectangle();
//生成文件大小
long totalSize = cla.getSize();
//可以为空或grid
String sindex = cla.get("sindex");
//是否覆盖原有文件
boolean overwrite = cla.isOverwrite();
//生成的图形为点 时使用
DistributionType type = DistributionType.UNIFORM;
String strType = cla.get("type");
if (strType != null) {
strType = strType.toLowerCase();
if (strType.startsWith("uni"))
type = DistributionType.UNIFORM;
else if (strType.startsWith("gaus"))
type = DistributionType.GAUSSIAN;
else if (strType.startsWith("cor"))
type = DistributionType.CORRELATED;
else if (strType.startsWith("anti"))
type = DistributionType.ANTI_CORRELATED;
else if (strType.startsWith("circle"))
type = DistributionType.CIRCLE;
else {
System.err.println("Unknown distribution type: "+cla.get("type"));
printUsage();
return;
}
}
接下来有两种生成随机文件的方法:分别是本地生成和mapreduce方法生成。具体代码如下
if (totalSize < 100*1024*1024)
generateFileLocal(outputFile, stockShape, sindex, totalSize, mbr, type, rectSize, seed, blocksize, overwrite);
else
generateMapReduce(outputFile, mbr, totalSize, blocksize, stockShape,
sindex, seed, rectSize, type, overwrite);