Hdfs/代码总结

1.计算hdfs上文件大小

对每个文件，filestatus.getLen()。

2.获取当前正在运行job的数量

Configuration conf=new Configuration();

conf.set("mapred.job.tracker","URL");

JobClient client=new JobClient(new JobConf(conf));

client.getQueues()[0].getQueueName();

client.getJobsFromQueue(client.getQueues()[0].getQueueName()).length;

3.删除hdfs上的文件

  Configuration conf = new Configuration();
  FileSystem hdfs = FileSystem.get(URI.create(path),conf);
  hdfs.delete(new Path(path),false);

4.在hdfs上创建文件

Configuration  conf=new Configuration();

FileSystem  hdfs=FileSystem.get(conf);

byte[]  buff="hello ".getBytes();

Path dfs=new Path("xxx");

FSDataOutputStream  outputstream=hdfs.create(dfs);

outputStream.write(buff,0,buff,length);

5.在hdfs上创建目录

Configuration conf=new Configuration();

FileSystem hdfs=FileSystem.get(conf);

Path dfs=new Path("xxx");

hdfs.mkdirs(dfs);

6.重命名hdfs上的文件

Configuration conf=new Configuration();

FileSystem hdfs=FileSystem.get(conf);

boolean  isRename=hdfs.rename(srcpath,dstpath);

7.查找文件在hdfs集群的位置

Configuration  conf=new Configuration();

FileSystem  hdfs=FileSystem.get(conf);

Path  srcpath=new Path("xxx");

FileStatus  filestatus=hdfs.getFileStatus(srcpath);

BlockLocation[]  blklocations=hdfs.getFileBlockLocations(filestatus,0,filestatus.getLen());

int  blockLen=blkLocations.length;

for(int i=0; i< blockLen;i++)

{

String[]  hosts=blkLocations[i].getHosts();

System.out.println("block_"+i+"_location:"+hosts[0]);

}