参考书目:鸟哥的LINUX私房菜基础学习篇(第三版)
Linux Shell Scripting Cookbook
本节主要内容
- 基础正则表达式
1. 基础正则表达式
(1)^行开始符
^匹配一行的开始,例如’^Spark’ 匹配所有Spark开始的行
<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//grep -n表示查找到的结果显示行号 root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '^Spark' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>:Spark is <span class="hljs-operator" style="box-sizing: border-box;">a</span> fast <span class="hljs-operator" style="box-sizing: border-box;">and</span> general cluster computing <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">system</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> Big Data. It provides <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span>:Spark is built <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">using</span> [Apache Maven](<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">http</span>://maven.apache.org/). <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">53</span>:Spark also comes <span class="hljs-operator" style="box-sizing: border-box;">with</span> several sample programs <span class="hljs-operator" style="box-sizing: border-box;">in</span> <span class="hljs-operator" style="box-sizing: border-box;">the</span> `examples` <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">directory</span>. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">83</span>:Spark uses <span class="hljs-operator" style="box-sizing: border-box;">the</span> Hadoop core library <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> talk <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> HDFS <span class="hljs-operator" style="box-sizing: border-box;">and</span> other Hadoop-supported </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
(2)$行结束符
匹配一行的结束,例如′Spark [Math Processing Error]’ 匹配所有以Spark结束的行
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'Spark$' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">20</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">## Building Spark</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
(3).匹配任意一个字符
例如 Spa.k可以匹配Spark、Spaak等
<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'Spa.k' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>:Spark is <span class="hljs-operator" style="box-sizing: border-box;">a</span> fast <span class="hljs-operator" style="box-sizing: border-box;">and</span> general cluster computing <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">system</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> Big Data. It provides <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>:rich <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">set</span> <span class="hljs-operator" style="box-sizing: border-box;">of</span> higher-level tools including Spark SQL <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> SQL <span class="hljs-operator" style="box-sizing: border-box;">and</span> <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"> //其它省略</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
上面没有匹配小写spark,要匹配可以采用
<code class="hljs mel has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//-i选项表示忽略大小写</span> root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span>:~/ShellLearning# grep -<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">in</span> <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'Spa.k'</span> /hadoopLearning/spark-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.5</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span>-bin-hadoop2<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>/README.md </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
(4)[]匹配其中一个
[Ss]park只匹配Spark和spark
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[Ss]park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:and</span> <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Streaming</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> stream processing. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/> <span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">//</span>其它省略 </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>
(5) [^]不匹配[]中的任何一个字符
例如 ‘[^T]he’ ,不匹配The,但可匹配 the、che等
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[^T]he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
(6) [-]匹配固定范围的字符
例如[a-h]he,只匹配ahe、bhe、che…hhe,不匹配ihe、the等
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[a-h]he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:guide</span>, on the [project web page](<span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">http:</span>/<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/spark.apache.org/documentation</span>.html) </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
(7)? 匹配0次或1次
例如t?he只匹配he和the,不匹配tthe
<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">//?属于特殊符号,需要\进行转义 root@sparkslave02:~/ShellLearning<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 't\?he' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>:rich <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">set</span> <span class="hljs-operator" style="box-sizing: border-box;">of</span> higher-level tools including Spark SQL <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> SQL <span class="hljs-operator" style="box-sizing: border-box;">and</span> DataFrames, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>:<<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">http</span>://spark.apache.org/> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span>:You can find <span class="hljs-operator" style="box-sizing: border-box;">the</span> latest Spark documentation, including <span class="hljs-operator" style="box-sizing: border-box;">a</span> programming <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>:guide, <span class="hljs-command" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">on</span> <span class="hljs-title" style="box-sizing: border-box;">the</span> [<span class="hljs-title" style="box-sizing: border-box;">project</span> <span class="hljs-title" style="box-sizing: border-box;">web</span> <span class="hljs-title" style="box-sizing: border-box;">page</span>](<span class="hljs-title" style="box-sizing: border-box;">http</span>://<span class="hljs-title" style="box-sizing: border-box;">spark</span>.<span class="hljs-title" style="box-sizing: border-box;">apache</span>.<span class="hljs-title" style="box-sizing: border-box;">org</span>/<span class="hljs-title" style="box-sizing: border-box;">documentation</span>.<span class="hljs-title" style="box-sizing: border-box;">html</span>)</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"> //其它省略</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li></ul>
(8)+ 至少匹配一次
‘S+park’可以匹配Spark、SSpark、SSSpark等
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'S\+park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>
(9) * 匹配零次或多少
‘S*park’可匹配park、Spark、SSpark、SSSpark等
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n 'S*park' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:rich</span> set of higher-level tools including <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">SQL</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> <span class="hljs-constant" style="box-sizing: border-box;">DataFrames</span>, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:and</span> <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Streaming</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> stream processing. <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:<http</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">://spark</span>.apache.org/> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">15</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:You</span> can find the latest <span class="hljs-constant" style="box-sizing: border-box;">Spark</span> documentation, including a programming /<span class="hljs-regexp" style="color: rgb(0, 136, 0); box-sizing: border-box;">/其它省略</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li></ul>
(10) {n},匹配n次
例如[a-z]{3},匹配任意3个小写字母,等同于[a-z][a-z][a-z]
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[a-z]\{3\}' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># Apache Spark</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> is a fast <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">and</span> general cluster computing system <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> <span class="hljs-constant" style="box-sizing: border-box;">Big</span> <span class="hljs-constant" style="box-sizing: border-box;">Data</span>. <span class="hljs-constant" style="box-sizing: border-box;">It</span> provides </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li></ul>
(11) 其它限定次数匹配
{n, }至少匹配n次
{n, m}至少匹配n次,最多匹配m次
(13) 转义字符\
Ubuntu Linux ?,+,(,), {,}是特殊字符,在使用正则表达式时,如果不加转义符,会匹配将其视为一般字符,如果要设置为正则表达式式符,需要使用\进行转义,前面的例子已经给出示例。
(14) ()匹配一组字符
例如Sp(ar)\?k 匹配Spark和Spk,
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># echo "Spark Spk Spak" | grep -n 'Sp\(ar\)\?k'</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:Spark</span> <span class="hljs-constant" style="box-sizing: border-box;">Spk</span> <span class="hljs-constant" style="box-sizing: border-box;">Spak</span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
(15) URL匹配实战
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.\(\([A-Za-z]*\)\.\?\)*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span> </code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
上面整个例子可以分下列步骤完成:
(1)匹配http://
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
(2)匹配域名
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.[A-Za-z]*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
(3)处理重复部分
<code class="hljs ruby has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">root<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">@sparkslave02</span><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">:~/ShellLearning/Chapter15</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># grep -n '[A-Za-z]*://[A-Za-z]*\.\(\([A-Za-z]*\)\.\?\)*' /hadoopLearning/spark-1.5.0-bin-hadoop2.4/README.md </span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
转载: http://blog.csdn.net/lovehuangjiaju/article/details/48952457