Algorithms and Data Structures - Course Introduction

Example

A sequence of numbers

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .

c© 2006 Antonio Carzaniga

Example

A sequence of numbers

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .

The well-known Fibonacci sequence

Leonardo da Pisa (ca. 1170–ca. 1250)son of Guglielmo “Bonaccio”a.k.a. Leonardo Fibonacci


The Fibonacci Sequence

Mathematical definition of the Fibonacci sequence

Fn =

0 if n = 0

1 if n = 1

Fn−1 + Fn−2 if n > 1


The Fibonacci Sequence

Mathematical definition of the Fibonacci sequence

Fn =

0 if n = 0

1 if n = 1

Fn−1 + Fn−2 if n > 1

In “pseudo-code”

FIBONACCI(n)

1 if n = 02 then return 03 elseif n = 14 then return 15 else return FIBONACCI(n− 1) + FIBONACCI(n− 2)


Questions on Our First Algorithm

FIBONACCI(n)




FIBONACCI(n)


1. Is the algorithm correct?◮ for every valid input, does it terminate?◮ if so, does it do the right thing?



FIBONACCI(n)



2. How much time does it take to complete?



FIBONACCI(n)



2. How much time does it take to complete?

3. Can we do better?c© 2006 Antonio Carzaniga

Correctness


Correctness

FIBONACCI(n)


Fn =

0 if n = 0

1 if n = 1

Fn−1 + Fn−2 if n > 1


Correctness

FIBONACCI(n)


Fn =

0 if n = 0

1 if n = 1

Fn−1 + Fn−2 if n > 1

The algorithm is clearly correct◮ assuming n ≥ 0


Performance

How long does it take?


Performance

How long does it take?

Let’s try it. . .


Results

0

5

10

15

20

25

30

20 25 30 35 40 45

Tim

e (s

econ

ds)

n

"Python""Scheme"

"C""C-wiz""Java"

"C-gcc"


Comments


Comments

Different implementations perform differently

◮ it is better to let the compiler do the optimization

◮ simple language tricks don’t seem to pay off


Comments




However, the differences do not seem to be substantial

◮ all implementations, sooner or later, seem to hit a wall. . .


Comments




However, the differences do not seem to be substantial

◮ all implementations, sooner or later, seem to hit a wall. . .

Conclusion: the problem lies with the algorithm


Complexity of Our First Algorithm

We need a mathematical characterization of the performance ofthe algorithm

We’ll call it the algorithm’s computational complexity





Formally, but a bit superficially for now, let T (n) be the number ofbasic steps needed to compute FIBONACCI(n)






FIBONACCI(n)







FIBONACCI(n)


T (0) = 2; T (1) = 3






FIBONACCI(n)


T (0) = 2; T (1) = 3

T (n) = T (n− 1) + T (n− 2) + 3






FIBONACCI(n)


T (0) = 2; T (1) = 3

T (n) = T (n− 1) + T (n− 2) + 3 ⇒ T (n) ≥ Fn


Complexity of Our First Algorithm (2)

So, let’s try to understand how Fn grows with n

T (n) ≥ Fn = Fn−1 + Fn−2




T (n) ≥ Fn = Fn−1 + Fn−2

Now, since Fn ≥ Fn−1 ≥ Fn−2 ≥ Fn−3 ≥ . . .

Fn ≥ 2Fn−2




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4)




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6))




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . .




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2

This means that

T (n) ≥ (√

2)n ≈ (1.4)n




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2

This means that

T (n) ≥ (√

2)n ≈ (1.4)n

T (n) grows exponentially with n




T (n) ≥ Fn = Fn−1 + Fn−2


Fn ≥ 2Fn−2 ≥ 2(2Fn−4) ≥ 2(2(2Fn−6)) ≥ . . . ≥ 2n2

This means that

T (n) ≥ (√

2)n ≈ (1.4)n

T (n) grows exponentially with n

Can we do better?


A Better Algorithm

Again, the sequence is 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . .


A Better Algorithm


Idea: we can build Fn from the ground up!


A Better Algorithm


Idea: we can build Fn from the ground up!

SMARTFIBONACCI(n)

1 if n = 02 then return 03 elseif n = 14 then return 15 else pprev ← 06 prev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev

10 prev ← f11 return f


Results

0

5

10

15

20

25

30

0 20 40 60 80 100

Tim

e (s

econ

ds)

n

"PythonSmart""Scheme"

"C""C-wiz""Java"

"C-gcc"


Complexity of SmartFibonacci

SMARTFIBONACCI(n)

1 if n = 02 then return 03 elseif n = 14 then return 15 else prev ← 06 pprev ← 17 for i ← 2 to n8 do f ← prev + pprev9 pprev ← prev




SMARTFIBONACCI(n)



T (n) =



SMARTFIBONACCI(n)



T (n) = 6 + 6(n− 1)



SMARTFIBONACCI(n)



T (n) = 6 + 6(n− 1) = 6n



SMARTFIBONACCI(n)



T (n) = 6 + 6(n− 1) = 6n

The complexity of SMARTFIBONACCI(n) is linear in n



SMARTFIBONACCI(n)



T (n) = 6 + 6(n− 1) = 6n

The complexity of SMARTFIBONACCI(n) is linear in n . . . almost. . .


Algorithms and Data Structures - Course Introduction

Documents

Transcript of Algorithms and Data Structures - Course Introduction