Spark修炼之道（基础篇）——Linux大数据开发基础：第十五节：基础正则表达式（一)

参考书目：鸟哥的LINUX私房菜基础学习篇（第三版） 
Linux Shell Scripting Cookbook

本节主要内容

1. 基础正则表达式

1. 基础正则表达式

(1)^行开始符

^匹配一行的开始，例如’^Spark’ 匹配所有Spark开始的行

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//grep -n表示查找到的结果显示行号
root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '^Spark' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>:Spark is <span class="hljs-operator" style="box-sizing: border-box;">a</span> fast <span class="hljs-operator" style="box-sizing: border-box;">and</span> general cluster computing <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">system</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> Big Data. It provides
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:Spark is built <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">using</span> [Apache Maven](<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">http</span>://maven.apache.org/).
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">53</span>:Spark also comes <span class="hljs-operator" style="box-sizing: border-box;">with</span> several sample programs <span class="hljs-operator" style="box-sizing: border-box;">in</span> <span class="hljs-operator" style="box-sizing: border-box;">the</span> `examples` <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">directory</span>.
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">83</span>:Spark uses <span class="hljs-operator" style="box-sizing: border-box;">the</span> Hadoop core library <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> talk <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> HDFS <span class="hljs-operator" style="box-sizing: border-box;">and</span> other Hadoop-supported
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>

(2)$行结束符

匹配一行的结束，例如′Spark [Math Processing Error]’ 匹配所有以Spark结束的行

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'Spark$' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## Building Spark</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

(3).匹配任意一个字符

例如 Spa.k可以匹配Spark、Spaak等

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'Spa.k' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>:Spark is <span class="hljs-operator" style="box-sizing: border-box;">a</span> fast <span class="hljs-operator" style="box-sizing: border-box;">and</span> general cluster computing <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">system</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> Big Data. It provides
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>:rich <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">set</span> <span class="hljs-operator" style="box-sizing: border-box;">of</span> higher-level tools including Spark SQL <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> SQL <span class="hljs-operator" style="box-sizing: border-box;">and</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">
//其它省略</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

上面没有匹配小写spark，要匹配可以采用

<code class="hljs mel has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//-i选项表示忽略大小写</span>
root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span>:~/ShellLearning# grep -<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'Spa.k'</span> /hadoopLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/README.md </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

(4)[]匹配其中一个

[Ss]park只匹配Spark和spark

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[Ss]park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>,
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:and</span> <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Streaming</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> stream processing.
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/>
<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>其它省略
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>

(5) [^]不匹配[]中的任何一个字符

例如 ‘[^T]he’ ，不匹配The，但可匹配 the、che等

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[^T]he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

(6) [-]匹配固定范围的字符

例如[a-h]he，只匹配ahe、bhe、che…hhe，不匹配ihe、the等

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[a-h]he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>,
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:guide</span>, on the [project web page](<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">http:</span>/<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/spark.apache.org/documentation</span>.html)
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

(7)? 匹配0次或1次

例如t?he只匹配he和the，不匹配tthe

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//?属于特殊符号，需要\进行转义
root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 't\?he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>:rich <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">set</span> <span class="hljs-operator" style="box-sizing: border-box;">of</span> higher-level tools including Spark SQL <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> SQL <span class="hljs-operator" style="box-sizing: border-box;">and</span> DataFrames,
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>:<<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">http</span>://spark.apache.org/>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>:You can find <span class="hljs-operator" style="box-sizing: border-box;">the</span> latest Spark documentation, including <span class="hljs-operator" style="box-sizing: border-box;">a</span> programming
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:guide, <span class="hljs-command" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">on</span> <span class="hljs-title" style="box-sizing: border-box;">the</span> [<span class="hljs-title" style="box-sizing: border-box;">project</span> <span class="hljs-title" style="box-sizing: border-box;">web</span> <span class="hljs-title" style="box-sizing: border-box;">page</span>](<span class="hljs-title" style="box-sizing: border-box;">http</span>://<span class="hljs-title" style="box-sizing: border-box;">spark</span>.<span class="hljs-title" style="box-sizing: border-box;">apache</span>.<span class="hljs-title" style="box-sizing: border-box;">org</span>/<span class="hljs-title" style="box-sizing: border-box;">documentation</span>.<span class="hljs-title" style="box-sizing: border-box;">html</span>)</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">
//其它省略</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>

(8)+ 至少匹配一次

‘S+park’可以匹配Spark、SSpark、SSSpark等

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'S\+park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

(9) * 匹配零次或多少

‘S*park’可匹配park、Spark、SSpark、SSSpark等

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'S*park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>,
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:and</span> <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Streaming</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> stream processing.
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:You</span> can find the latest <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> documentation, including a programming
/<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/其它省略</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>

(10) {n}，匹配n次

例如[a-z]{3}，匹配任意3个小写字母，等同于[a-z][a-z][a-z]

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[a-z]\{3\}' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>

(11) 其它限定次数匹配

{n, }至少匹配n次 
{n, m}至少匹配n次，最多匹配m次

(13) 转义字符\

Ubuntu Linux ?,+,(,), {,}是特殊字符，在使用正则表达式时，如果不加转义符，会匹配将其视为一般字符，如果要设置为正则表达式式符，需要使用\进行转义，前面的例子已经给出示例。

(14) ()匹配一组字符

例如Sp(ar)\?k 匹配Spark和Spk，

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo "Spark Spk Spak" | grep -n 'Sp\(ar\)\?k'</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Spk</span> <span class="hljs-constant" style="box-sizing: border-box;">Spak</span>
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

(15) URL匹配实战

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.\(\([A-Za-z]*\)\.\?\)*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span>

</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>

上面整个例子可以分下列步骤完成： 
(1)匹配http://

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

(2)匹配域名

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.[A-Za-z]*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

(3)处理重复部分

<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.\(\([A-Za-z]*\)\.\?\)*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

转载： http://blog.csdn.net/lovehuangjiaju/article/details/48952457