Top Banner
Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2
29

Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Sep 06, 2018

Download

Documents

hadan
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Automata and Formal Languages - CM0081Introduction

Andrés Sicard-Ramírez

Universidad EAFIT

Semester 2018-2

Page 2: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Administrative Information

Course CoordinatorAndrés Sicard Ramírez

Head of the Department of Mathematical SciencesCarlos Mario Vélez Sánchez

Course web pagehttp://www1.eafit.edu.co/asr/courses/automata-CM0081/

Exams, programming labs, etc.See course web page.

ConventionThe numbers assigned to examples, exercises or theorems correspond to thenumbers in the textbook (Hopcroft, Motwani and Ullman 2007).

Introduction 2/29

Page 3: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Computability (Decidability)

QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved bya computer (informal definition).

Introduction 3/29

Page 4: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Computability (Decidability)

QuestionWhat can a computer do at all?

DefinitionA computable (or decidable) problem is a problem than can be solved bya computer (informal definition).

Introduction 4/29

Page 5: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Computability (Decidability)

Example (The halting problem: An undecidable problem)Given an program P and an input I, to decide if the program will halt orwill run forever.

The halting algorithm

P Yes

I No

Introduction 5/29

Page 6: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

Comparison of several time complexity functions

𝑓(𝑛) 10 50 100log 𝑛 2.3 sec 3.9 sec 4.6 sec

𝑛 10.0 sec 50.0 sec 1.7 min

𝑛2 1.7 min 41.7 min 2.8 h  

2𝑛 17.1 min 3.6 × 105 c 4.0 × 1020  c

3𝑛 16.4 h   2.3 × 1014 c 1.6 × 1038  c

𝑛! 42.0 d   9.7 × 1054 c 3.0 × 10148 c

Introduction 6/29

Page 7: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer al-gorithm that run in polynomial time.

Introduction 7/29

Page 8: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

QuestionWhat can a computer do efficiently?

DefinitionA tractable problem is a problem than can be solved by a computer al-gorithm that run in polynomial time.

Introduction 8/29

Page 9: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

DefinitionA literal is an atomic formula (propositional variable) or the negation of anatomic formula.

DefinitionA (propositional logic) formula 𝐹 is said to be in conjunctive normal form,if and only if,

𝐹 has the form 𝐹1 ∧ ⋯ ∧ 𝐹𝑛, 𝑛 ≥ 1,where each 𝐹1, … , 𝐹𝑛 is a disjunction of literals.

Introduction 9/29

Page 10: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

Example (3-SAT: An intractable problem)To determine the satisfiability of a propositional formula in conjunctivenormal form where each disjunction of literals is limited to at most threeliterals.The problem was proposed in Karp’s 21 NP-complete problems (Karp1972).

Introduction 10/29

Page 11: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

Improvements on 3-SAT deterministic algorithmic complexity†

𝑂(1.3303𝑛) Makino, Tamaki and Yamamoto (2011, 2013)𝑂(1.3334𝑛) Moser and Scheder (2011)𝑂(1.439𝑛) Kutzkov and Scheder (2010)𝑂(1.465𝑛) Scheder (2008)𝑂(1.473𝑛) Brueggemann and Kern (2004)𝑂(1.481𝑛) Dantsin, Goerdt, Hirsch, Kannan, Kleinberg, Papadimitriou,

Raghavan and Schöning (2002)𝑂(1.497𝑛) Schiermeyer (1996)𝑂(1.505𝑛) Kullmann (1999)𝑂(1.6181𝑛) Monien and Speckenmeyer (1979, 1985)𝑂(2𝑛) Brute-force search

†Main sources: Hertli (2011, 2015). Last updated: July 2018.Introduction 11/29

Page 12: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

SupercomputersMachines from: www.top500.org†

Petaflops (PFs): 1015 floating-point operations per second

Date Machine PFs2018-06 Summit 122.302016-06 Sunway TaihuLight 93.012013-06 Tianhe-2 33.862012-06 Blue Gene/Q 16.322011-06 K computer 8.16

†Last updated: TOP500 List - June 2018.Introduction 12/29

Page 13: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

Simulation: Running 3-SAT times on different supercomputers using thefaster deterministic algorithm, i.e., 𝑂(1.3303𝑛)

Machine PFs 150 200 400Summit 122.30 32.1 sec 1.6 y 9.9 × 1022 c

Sunway TaihuLight 93.01 42.1 sec 2.1 y 1.3 × 1023 c

Tianhe-2 33.86 1.9 min 5.8 y 3.6 × 1023 c

Blue Gene/Q 16.32 4.0 min 12.0 y 7.4 × 1023 c

K computer 8.16 8.0 min 24.0 y 1.5 × 1024 c

Introduction 13/29

Page 14: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Algorithmic Complexity (Tractability)

Simulation: Running 3-SAT times for different deterministic algorithmsusing the faster supercomputer, i.e., 122.30 PFs

Complexity 150 200 400𝑂(1.3303𝑛) 32.1 sec 1.6 y 9.9 × 1022 c

𝑂(1.3334𝑛) 45.5 sec 2.6 y 2.5 × 1023 c

𝑂(1.439𝑛) 48.6 sec 1.1 × 105 c 4.4 × 1036 c

𝑂(1.465𝑛) 1.9 y   3.8 × 106 c 5.6 × 1039 c

𝑂(2𝑛) 3.7 y   4.2 × 1033 c 6.7 × 1093 c

Introduction 14/29

Page 15: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Computability and Algorithmic Complexity

Problem(TM-Computability)

⎧{{{{{{{{{⎨{{{{{{{{{⎩

Computable(decidable)problem(AlgorithmComplexity)

⎧{{{{⎨{{{{⎩

Tractableproblem

Intractableproblem

Non-computable(undecidable)problem(Hypercomputation)

Introduction 15/29

Page 16: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Course Outline

Language Machine Other modelsRegular DFA • Regular expressions

• NFA• 𝜀-NFA

Context-free Pushdownautomata

Recursive Halting TMs • 𝜆-calculus• Total recursive functions

Recursivelyenumerable

TMs • 𝜆-calculus• Partial recursive functions

DFA: Deterministic finite automataNFA: Non-deterministic finite automata𝜀-NFA: Non-deterministic finite automata with 𝜀-transitionsTM: Turing machine

Introduction 16/29

Page 17: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Paradigms of Programming

Some paradigmsImperative: Describe computation in terms of state-transformingoperations such as assignment. Programming is done with statements.

Logic: Predicate calculus as a programming language. Programming isdone with sentences.

Functional: Describe computation in terms of (mathematical)functions. Programming is done with expressions.

Examples

Imperative⎧{⎨{⎩

CC++Java

Logic {CLP(R)Prolog

Functional⎧{⎨{⎩

Standard MLErlangHaskell

Introduction 17/29

Page 18: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Side effects“A side effect introduces a dependency between the global state of thesystem and the behaviour of a function... Side effects are essentially invisibleinputs to, or outputs from, functions.” (O’Sullivan, Goerzen and Stewart2008, p. 27)

Introduction 18/29

Page 19: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 19/29

Page 20: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Pure functions“A function may be described as pure if both these statements aboutthe function hold:”†

1. “The function always evaluates the same result value given thesame argument value(s). The function result value cannotdepend on any... state that may change as program executionproceeds or between different executions of the program, nor canit depend on any external input from I/O devices.”

2. “Evaluation of the result does not cause any semanticallyobservable side effect or output, such as mutation of mutableobjects or output to I/O devices.”

†Wikipedia: Pure function (July 28, 2014).Introduction 20/29

Page 21: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Pure functions“...take all their input as explicit arguments, and produce all theiroutput as explicit results.” (Hutton 2016, § 10.1)

Introduction 21/29

Page 22: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Referential transparencyEquals can be replaced by equals

“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 22/29

Page 23: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Referential transparencyEquals can be replaced by equals“By definition, a function in Haskell defines a fixed relation betweeninputs and output: whenever a function 𝑓 is applied to the argumentvalue 𝑎𝑟𝑔 it will produce the same output no matter what the overallstate of the computation is. Haskell, like any other pure functionallanguage, is said to be ‘referentially transparent’ or ‘side-effect free’.This property does not hold for imperative languages...” (Grune, Bal,Jacobs and Langendoen 2003, pp. 544–545)

Introduction 23/29

Page 24: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Pure Functional Programming

Reasoning about (pure) functional programsEquational reasoning + induction + co-induction + …

Introduction 24/29

Page 25: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

Reading

HomeworkTo read from the textbook the following sections:§ 1.1. Why Study Automata Theory?§ 1.2. Introduction to Formal Proofs§ 1.3. Additional Forms of Proofs

Introduction 25/29

Page 26: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

References

Brueggemann, T. and Kern, W. (2004). ‘An Improved Deterministic LocalSearch Algorithm for 3-SAT’. In: Theoretical Computer Science 329.1–3,pp. 303–313. doi: 10.1016/j.tcs.2004.08.002 (cit. on p. 11).Dantsin, E. et al. (2002). ‘A Deterministic (2 − 2/(𝑘 + 1))𝑛 Algorithm for𝑘-SAT Based on Local Search’. In: Theoretical Computer Science 289.1,pp. 69–83. doi: 10.1016/S0304-3975(01)00174-8 (cit. on p. 11).Grune, D., Bal, H. E., Jacobs, C. J. H. and Langendoen, K. G. (2003).Modern Compiler Desing. John Wiley & Sons (cit. on pp. 22, 23).Hertli, T. (2011). ‘3-SAT Faster and Simpler - Unique-SAT Bounds forPPSZ Hold in General’. In: Proceedings of the 52nd Annual Symposium onFoundations of Computer Science (FOCS 2011). IEEE, pp. 277–284. doi:10.1109/FOCS.2011.22 (cit. on p. 11).— (2015). ‘Improved Exponential Algorithms for SAT and ClSP’.PhD thesis. ETH Zurich. doi: 10.3929/ethz-a-010512781 (cit. on p. 11).

Introduction 26/29

Page 27: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

References

Hopcroft, J. E., Motwani, R. and Ullman, J. D. (2007). Introduction toAutomata theory, Languages, and Computation. 3rd ed. Pearson Education(cit. on p. 2).Hutton, G. (2016). Programming in Haskell. 2nd ed. Cambridge UniversityPress (cit. on p. 21).Karp, R. M. (1972). ‘Reducibility Among Combinatorial Problems’. In:Complexity of Computer Computations. Ed. by Miller, R. E. andThatcher, J. W. Plenum Press, pp. 85–103. doi:10.1007/978-1-4684-2001-2_9 (cit. on p. 10).Kullmann, O. (1999). ‘New Methods for 3-SAT Decision and Worst-CaseAnalysis’. In: Theoretical Computer Science 223.1–2, pp. 1–72. doi:10.1016/S0304-3975(98)00017-6 (cit. on p. 11).Kutzkov, K. and Scheder, D. (2010). ‘Using CSP to Improve Deterministic3-SAT’. CoRR abs/1007.1166. url: https://arxiv.org/abs/1007.1166(cit. on p. 11).

Introduction 27/29

Page 28: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

References

Makino, K., Tamaki, S. and Yamamoto, M. (2011). ‘Derandomizing HSSWAlgorithm for 3-SAT’. In: Computing and Combinatorics (COCOON 2011).Ed. by Fu, B. and Du, D.-Z. Vol. 6842. Lecture Notes in Computer Science.Springer, pp. 1–12. doi: 10.1007/978-3-642-22685-4_1 (cit. on p. 11).— (2013). ‘Derandomizing HSSW Algorithm for 3-SAT’. In: Algorithmica67.2, pp. 112–124. doi: 10.1007/s00453-012-9741-4 (cit. on p. 11).Monien, B. and Speckenmeyer, E. (1979). 3-Satisfiability is Testable in𝑂(1.62𝑟) Steps. Tech. rep. 3/1979. Reihe Theoretische Informatik,Universität-Gesamthochschule-Paderborn (cit. on p. 11).— (1985). ‘Solving Satisfiability in less than 2𝑛 Steps’. In: DiscreteApplied Mathematics 10.3, pp. 287–295. doi:10.1016/0166-218X(85)90050-2 (cit. on p. 11).Moser, R. A. and Scheder, D. (2011). ‘A Full Derandomization ofSchöning’s 𝑘-SAT Algorithm’. In: Proceedings of the Forty-third AnnualACM Symposium on Theory of Computing (STOC 2011), pp. 245–252.doi: 10.1145/1993636.1993670 (cit. on p. 11).

Introduction 28/29

Page 29: Automata and Formal Languages - CM0081 Introduction · Automata and Formal Languages - CM0081 Introduction Andrés Sicard-Ramírez Universidad EAFIT Semester 2018-2

References

O’Sullivan, B., Goerzen, J. and Stewart, D. (2008). Real World Haskell.O’REILLY (cit. on p. 18).Scheder, D. (2008). ‘Guided Search and a Faster Deterministic Algorithmfor 3-SAT’. In: Proc. of the 8th Latin American Symposium on TheoreticalInformatic (LATIN 2008). Ed. by Laber, E. S., Bornstein, C.,Nogueira, T. L. and Faria, L. Vol. 4957. Lecture Notes in Computer Science.Springer, pp. 60–71. doi: 10.1007/978-3-540-78773-0_6 (cit. on p. 11).Schiermeyer, I. (1996). ‘Pure Literal Look Ahead: An 𝑂(1.497𝑛)3-Satisfability Algorithm (Extended Abstract)’. Workshop on theSatisfability Problem, Siena 1996. url:http://gauss.ececs.uc.edu/franco_files/SAT96/sat-workshop-abstracts.html (cit. on p. 11).

Introduction 29/29