Notes for Formal Languages and Automata Theory

Operations : union , concatenation , Kleene closure, positive closure

Alphabet

Non-empty finite set of symbols
$\Sigma$

Strings

finite sequence of symbols of $\Sigma$
(word)

empty string : $\epsilon$

${\Sigma }^{\ast }$ : the set of all strings over alphabet $\Sigma$ (including $\epsilon$)

${\Sigma }^{\ast }$ is monoid with respect to concatenation

Languages

A language over alphabet $\Sigma$ is a subset (which can be infinite) of ${\Sigma }^{\ast }$
(personally, similar to vocabulary)

Kleene star or Kleene closure of a language $L$
$L^{\ast }={\cup }_{n\ge 0}{L}^n$

which is consistent with the notation of ${\Sigma }^{\ast }$ by considering $\Sigma$ as a language over alphabet $\Sigma$

Positive closure of a language $L$
$L^{+}={\cup }_{n\ge 1}{L}^n$

Some Properties

P1 Concatenation is associative

P2 Concatenation is not commutative

P3 L { ε } = { ε } L = L L\{\varepsilon\}=\{\varepsilon\}L=L L{ε}={ε}L=L

P4 $L\varnothing =\varnothing L=\varnothing$

P5 Distributive properties

1. $(L_1\cup L_2)L_3=L_1{L}_3\cup L_2{L}_3$
2. $L_1(L_2\cup L_3)=L_1{L}_2\cup L_1{L}_3$

P6 if $L_1\subseteq L_2$ and $L_3\subseteq L_4$, then $L_1{L}_3\subseteq L_2{L}_4$

P7 ∅ ∗ = { ε } \varnothing^*=\{\varepsilon\} ∅∗={ε}

P8 { ε } ∗ = { ε } \{\varepsilon\}^*=\{\varepsilon\} {ε}∗={ε}

P9 if $\epsilon \in L$, then $L^{\ast }=L^{+}$

P10 $L{L}^{\ast }=L^{\ast }L=L^{+}$

P11 $(L^{\ast }{)}^{\ast }=L^{\ast }$

P12 $L^{\ast }{L}^{\ast }=L^{\ast }$

P13 $(L_1{L}_2{)}^{\ast }{L}_1=L_1(L_2{L}_1{)}^{\ast }$

P14 $(L_1\cup L_2{)}^{\ast }=(L_1^{\ast }{L}_2^{\ast }{)}^{\ast }$

Regular Expressions

1. $\varnothing$, $\epsilon$, and $a$ for each $a\in \Sigma$ are regular expressions representing the language for $\varnothing$, { ε } \{\varepsilon\} {ε}, and { a } \{a\} {a} respectively
2. a. $(r+s)$ representing the language $R\cup S$
   b. $(rs)$ representing the language $RS$
   c. $(r^{\ast })$ representing the language $R^{\ast }$

(keep minimum possible number of parenthesis in writing)

regular language $L(r)$ represented by regular expression $r$
sometimes just use $r$ instead of $L(r)$ to indicate the regular language $L$

Equivalent
expressions $r_1$ and $r_2$ representing the same language
we write $r_1\approx r_2$ ($r_1=r_2$ is also reasonable)

Some identities

1. $r\epsilon \approx \epsilon r\approx r$
2. $r_1{r}_2\not\approx r_2{r}_1$, in general
3. $(r_1{r}_2)r_3\approx r_1(r_2{r}_3)$
4. $r\varnothing \approx \varnothing r\approx \varnothing$
5. ${\varnothing }^{\ast }\approx \epsilon$
6. ${\epsilon }^{\ast }\approx \epsilon$
7. if { ε } ∈ L ( r ) \{\varepsilon\} \in L(r) {ε}∈L(r) then $r^{\ast }\approx r^{+}$
8. $r{r}^{\ast }\approx r^{\ast }r\approx r^{+}$
9. Distributive identity.
   $(r_1+r_2)r_3\approx r_1{r}_3+r_2{r}_3$
   $r_1(r_2+r_3)\approx r_1{r}_2+r_1{r}_3$
10. $(r^{\ast }{)}^{\ast }\approx r^{\ast }$
11. $(r_1{r}_2{)}^{\ast }{r}_1\approx r_1(r_2{r}_1{)}^{\ast }$
12. $(r_1+r_2{)}^{\ast }\approx (r_1^{\ast }{r}_2^{\ast }{)}^{\ast }$

Grammars

Context-Free Grammars (CFG)

A grammar is a quadruple
$$\mathcal{G}=(V,T,S,P)$$
where

1. $V=N\cup T$, where $N$ is a finite set of non-terminals
2. $T$ is a finite set of terminals
3. $S\in N$ is the start symbol
4. $P$ is a finite set of $N\times V^{\ast }$, called the set of production rules.

Some definitions

• Consider a binary relation ${\Rightarrow }_{\mathcal{G}}$ on $V^{\ast }$, let’s say, one step relation.
• $\alpha {\Rightarrow }_{\mathcal{G}}\beta$ iff. $\alpha ={\alpha }_{1}A{\alpha }_2,\beta ={\alpha }_1\gamma {\alpha }_2$ and $A\to \gamma \in P$.
• if $\alpha {\Rightarrow }_{\mathcal{G}}\beta$, then we call $\alpha$ yields $\beta$ in one step in $\mathcal{G}$.
• The reflexive-transitive closure（自反传递闭包）of ${\Rightarrow }_{\mathcal{G}}$ is denoted by ${\Rightarrow }_{\mathcal{G}}^{\ast }$. That is, for $\alpha ,\beta \in V^{\ast }$, $\alpha {\Rightarrow }_{\mathcal{G}}^{\ast }\beta$ iff. $\exists n\ge 0$ and ${\alpha }_1,{\alpha }_2,...,{\alpha }_n\in V^{\ast }$ such that $\alpha ={\alpha }_0{\Rightarrow }_{\mathcal{G}}{\alpha }_1{\Rightarrow }_{\mathcal{G}}{\alpha }_2{\Rightarrow }_{\mathcal{G}}...{\Rightarrow }_{\mathcal{G}}{\alpha }_{n-1}{\Rightarrow }_{\mathcal{G}}{\alpha }_n=\beta$
• if $\alpha {\Rightarrow }_{\mathcal{G}}^{\ast }\beta$, then we say $\alpha$ derives $\beta$. Further, $\alpha {\Rightarrow }_{\mathcal{G}}^{\ast }\beta$ is called as a derivation in $\mathcal{G}$, and $\beta$ is the yield of the derivation.
• In a given context, if we’re dealing with only one grammar $\mathcal{G}$, then simply write $\Rightarrow$ instead of ${\Rightarrow }_{\mathcal{G}}$
• if $\alpha ={\alpha }_0{\Rightarrow }_{\mathcal{G}}{\alpha }_1{\Rightarrow }_{\mathcal{G}}{\alpha }_2{\Rightarrow }_{\mathcal{G}}...{\Rightarrow }_{\mathcal{G}}{\alpha }_{n-1}{\Rightarrow }_{\mathcal{G}}{\alpha }_n=\beta$ is a derivation, then the length of the derivation is $n$ and it maybe written as $\alpha {\Rightarrow }_{\mathcal{G}}^n\beta$
• A string $\alpha \in V^{\ast }$ is said to be a sentential form in $\mathcal{G}$, if $\alpha$ can be derived from the start symbol $S$ of $\mathcal{G}$, viz. $S{\Rightarrow }^{\ast }\alpha$
• In particular, if $\alpha \in {\Sigma }^{\ast }$, then the sentential form $\alpha$ is known as a sentence. In which case, we say $\alpha$ is generated by $\mathcal{G}$
• The language generated by $\mathcal{G}$, denoted by $L(\mathcal{G})$, is the set of all sentences generated by $\mathcal{G}$. That is, L ( G ) = { x ∈ Σ ∗ ∣ S ⇒ ∗ x } L(\mathcal{G})=\{x\in \Sigma^*|S\Rightarrow^*x\} L(G)={x∈Σ∗∣S⇒∗x}

Production rules $A\to \alpha$ is independent of neighbouring symbols, i.e. context free.

Thus the grammar defined here is known as context free grammar, simply CFG.

Derivation Trees

also known as parse trees
the yield $\alpha$ of derivation $A{\Rightarrow }^{\ast }\alpha$ can be identified in the derivation tree by juxtaposing the lables of leaf nodes from left to right.

Ambiguity

ambiguous grammar/unambiguous grammar
inherently ambiguous language

Regular Grammars

linear grammar : every production rule has at most one nonterminal symbol
left-linear (right-linear) grammar : the nonterminal symbol is at the left end (right end)

The language generated by a right linear grammar is regular.
For every regular language $L$, there exists a right linear grammar that generates $L$

Right linear grammars are also called regular grammars

Things also hold for left linear grammar.

Directed Graph Representation

Directed graph (short for digraph)

Chomsky Normal Form (CNF)

All productions are of the form $A\to BC$ or $A\to a$, where $A,B,C$ are variables, and $a$ is terminal symbol.

Greibach Normal Form (GNF)

All productions are shown as the following form :
$$A\to aX,wherea\in T,X\in V^{\ast }$$

Simplification of CFG

“3R”

1. Reduction (two phases, 1.generating+2.reachable=useful) (Removal of useless production)
2. Removal of unit production
3. Removal of $ϵ$-production

Finite Automata (FA)

Deterministic Finite Automata (DFA)

A (deterministic) finite automaton $\mathcal{A}=(Q,\Sigma ,q_0,\delta ,F)$
$Q$ is a finite set called the set of states
$\Sigma$ is a finite set called the input alphabet
$q_0\in Q$, called the initial/start state
$\delta :Q\times \Sigma \to Q$ is a function called transition function or next-state function (which can be extended, say $\hat{\delta }:Q\times {\Sigma }^{\ast }\to Q$)
$F\subseteq Q$, called the set of final/accept states

Useful graphical representation : Transition Diagram
which can be constructed as follows:

1. Every state in $Q$ is represented by a node
2. If $\delta (p,a)=q$, then there’s an arc from $p$ to $q$ labeled $a$
3. There’s an arrow with no source into the initial state $q_0$
4. Final states are indicated by double circle

“accepted by”

Language of a DFA : $L(\mathcal{A})$

Configurations
$C\in Q\times {\Sigma }^{\ast }$

the computation of $\mathcal{A}$ on the input $x$

Nondeterministic Finite Automata (NFA)

NFA

$\epsilon$-NFA

A nondeterministic finite automaton $\mathcal{N}=(Q,\Sigma ,q_0,\delta ,F)$, where $Q,\Sigma ,q_0,F$ are as in a DFA; whereas, the transition function $\delta$ is as below:
δ : Q × ( Σ ∪ { ε } ) → ℘ ( Q ) \delta : Q \times (\Sigma \cup \{\varepsilon\}) \rightarrow \wp(Q) δ:Q×(Σ∪{ε})→℘(Q)
is a function so that, for a given state an a input symbol (possibly $\epsilon$), $\delta$ assigns a set of next states, possibly empty set.

$\delta$ can also be extended to $\hat{\delta }$

Every DFA can be treated as an NFA.

L ( N ) = { x ∈ Σ ∗ ∣ δ ^ ( q 0 , x ) ∩ F ≠ ∅ } L(\mathscr{N})=\{x \in \Sigma^* | \hat\delta(q_0,x) \cap F \neq \varnothing \} L(N)={x∈Σ∗∣δ^(q0​,x)∩F​=∅}

Equivalence of RE & FA

Pushdown Automata (PDA)

$$P=(Q,\Sigma ,\Gamma ,\delta ,q_0,Z_0,F)$$
$\Gamma$ : A finite stack alphabet.
$Z_0$ : The start symbol in the stack.

Equivalence of CFG & PDA

Turing Machine

e.g. L = { a n b n c n ∣ n ≥ 0 } L = \{ a^nb^nc^n | n\geq 0\} L={anbncn∣n≥0}
not a CFL

Can be recognized by “A modified PDA with 2 stacks” - an variation of TM

TM is a seven-tuple $P=(Q,\Sigma ,\Gamma ,\delta ,q_0,B,F)$

• $Q$ is finite set of states
• $\Sigma$ is finite set of input symbols
• $\Gamma$ is finite set of tape symbols
• $delta$ is transition function : Q × Γ ⇒ Q × Γ × { R , L } Q\times \Gamma \Rightarrow Q\times \Gamma \times \{R,L\} Q×Γ⇒Q×Γ×{R,L}, $\delta (q,X)=(p,Y,\to /\leftarrow )$
• $q_0$ is start state
• $B$ is blank symbol
• $F$ is finite set of final states

Instantaneous

How to describe the configuration of TM

• sequence of symbols in tape
• state of TM
• read/write head of TM

i.e. $X_1...X_{i-1}q{X}_i...X_n$, where $q\in Q$

Language of TM

{ w ∣ q 0 w − > α p β , p ∈ F , α , β ∈ Γ ∗ } \{w|q_0w -> \alpha p\beta , p\in F, \alpha,\beta \in \Gamma^*\} {w∣q0​w−>αpβ,p∈F,α,β∈Γ∗}

The kind of languages accepted by TM is called recursively enumerable(RE) language. （递归可枚举语言）