Records在Yarn中原代码中主要用于RPC通訊,如以下语句生成一个新应用程序的請求,GetNewApplicationRequest request =
Records.newRecord(GetNewApplicationRequest.class);
Records⾥的代码非常简单,调用newRecord时调用factory.newRecordInstance()方法,源代码如下:
<span style="font-size:18px;">public class Records {
// The default record factory
private static final RecordFactory factory =
RecordFactoryProvider.getRecordFactory(null);
public static <T> T newRecord(Class<T> cls) {
return factory.newRecordInstance(cls);
}
}</span>
fractory是RecordFactory类型的,是一个接口,只有一个方法,代码如下:
public interface RecordFactory {
public <T> T newRecordInstance(Class<T> clazz);
}
这个factory是RecordFactoryProvider.getRecordFactory(null)来初始化的。RecordFactoryProvider的源代码如下:
<span style="font-size:18px;">public class RecordFactoryProvider {
private static Configuration defaultConf;
static {
defaultConf = new Configuration();
}
private RecordFactoryProvider() {
}
public static RecordFactory getRecordFactory(Configuration conf) {
if (conf == null) {
//Assuming the default configuration has the correct factories set.
//Users can specify a particular factory by providing a configuration.
conf = defaultConf;
}
String recordFactoryClassName = conf.get(
YarnConfiguration.IPC_RECORD_FACTORY_CLASS,
YarnConfiguration.DEFAULT_IPC_RECORD_FACTORY_CLASS);
return (RecordFactory) getFactoryClassInstance(recordFactoryClassName);
}
private static Object getFactoryClassInstance(String factoryClassName) {
try {
Class<?> clazz = Class.forName(factoryClassName);
Method method = clazz.getMethod("get", null);
method.setAccessible(true);
return method.invoke(null, null);
} catch (ClassNotFoundException e) {
throw new YarnRuntimeException(e);
} catch (NoSuchMethodException e) {
throw new YarnRuntimeException(e);
} catch (InvocationTargetException e) {
throw new YarnRuntimeException(e);
} catch (IllegalAccessException e) {
throw new YarnRuntimeException(e);
}
}
}</span>
在方法⾥,由于传递的参数是null,所以conf = defaultConf,默认的DEFAULT_IPC_RECORD_FACTORY_CLASS 是 "org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl";调用getFactoryClassInstance("org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl")。
在getFactoryClassInstance方法⾥.,直接调用它的静态get方法,没有参数。RecordFactoryPBImpl的源代码如下:
public class RecordFactoryPBImpl implements RecordFactory {
private static final String PB_IMPL_PACKAGE_SUFFIX = "impl.pb";
private static final String PB_IMPL_CLASS_SUFFIX = "PBImpl";
private static final RecordFactoryPBImpl self = new RecordFactoryPBImpl();
private Configuration localConf = new Configuration();
private ConcurrentMap<Class<?>, Constructor<?>> cache = new ConcurrentHashMap<Class<?>, Constructor<?>>();
private RecordFactoryPBImpl() {
}
public static RecordFactory get() {
return self;
}
@SuppressWarnings("unchecked")
@Override
....
}
RecordFactoryPBImpl是一个单例模式,get方法返回静态的final 变量 self.
在newRecordInstance(Class<T> clazz)方法⾥,首先判断cache是否能直接得到此类的构造器Constructor,如果得到,就直接调用constructor.newInstance();方法初始化一个对象,然后把此对象返回;如果缓冲器⾥没有,先调用getPBImplClassName(clazz)得到PBImpl的类实现,如getPBImplClassName(clazz);传递的是GetNewApplicationRequest,所在包是org.apache.hadoop.yarn.api.protocolrecords,按照规則,实现类是GetNewApplicationRequestPBImpl,所在包是org.apache.hadoop.yarn.api.protocolrecords.impl.pb, getPBImplClassName()方法返回的内容是org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetNewApplicationRequestPBImpl。
接着调用localConf.getClassByName("org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetNewApplicationRequestPBImpl")来返回class对象。然后得到它的constructor对象,放到cache⾥。
public <T> T newRecordInstance(Class<T> clazz) {
Constructor<?> constructor = cache.get(clazz);
if (constructor == null) {
Class<?> pbClazz = null;
try {
pbClazz = localConf.getClassByName(getPBImplClassName(clazz));
} catch (ClassNotFoundException e) {
throw new YarnRuntimeException("Failed to load class: ["
+ getPBImplClassName(clazz) + "]", e);
}
try {
constructor = pbClazz.getConstructor();
constructor.setAccessible(true);
cache.putIfAbsent(clazz, constructor);
} catch (NoSuchMethodException e) {
throw new YarnRuntimeException("Could not find 0 argument constructor", e);
}
}
try {
Object retObject = constructor.newInstance();
return (T)retObject;
} catch (InvocationTargetException e) {
throw new YarnRuntimeException(e);
} catch (IllegalAccessException e) {
throw new YarnRuntimeException(e);
} catch (InstantiationException e) {
throw new YarnRuntimeException(e);
}
}
private String getPBImplClassName(Class<?> clazz) {
String srcPackagePart = getPackageName(clazz);
String srcClassName = getClassName(clazz);
String destPackagePart = srcPackagePart + "." + PB_IMPL_PACKAGE_SUFFIX;
String destClassPart = srcClassName + PB_IMPL_CLASS_SUFFIX;
return destPackagePart + "." + destClassPart;
}
private String getClassName(Class<?> clazz) {
String fqName = clazz.getName();
return (fqName.substring(fqName.lastIndexOf(".") + 1, fqName.length()));
}
private String getPackageName(Class<?> clazz) {
return clazz.getPackage().getName();
}