Moving data in and out of hadoop

Moving data in and out of Hadoop, which I’ll refer to in this chapter as data ingress  and egress, is the process by which data is transported from an external system into an internal system, and vice versa.

ingress, n.进入；[计]入口；准许进入；入境
egress,

• n. 外出；出口
• vi. 外出
• vt. 使外出

vice versa, 反之亦然

input的separator 是 mapred.data.field.separator
mapper的output分隔符是 map.output.key.value.fields.spec
reducer的output分隔符是 reduce.output.key.value.fields.spec

以上貌似都不能用来做数据的风格符，但

  job.getConfiguration().set("mapred.textoutputformat.separator",

或者

<property>
      <name>mapred.textoutputformat.separator</name>
       <value>,</value>
</property>

就可以在oozie上设置逗号分隔的输出了,注，逗号不用冒号括起来。

In addition, we’ll  cover using Flume for moving log data into Hadoop, and in the process we’ll evaluate two competing log collection and aggregation tools,ChukwaandScribe. Finally, we’ll cover how to useSqoopfor database ingress and egress activities, and we’ll look at how to ingress and egress data inHBase.

idempotent 报错

英[,aɪdem'pəʊt(ə)nt; aɪ'dempət(ə)nt] 美[aɪ'dɛmpətənt]

• adj. 幂等的
• n. [数] 幂等