Download Presentation
## Algorithms

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Algorithms**Richard Anderson University of Washington IUCEE: Algorithms**Today’s topics**• Teaching Algorithms • Active Learning in Algorithms • Big Ideas: Solving Problems in Practice • Mysore / Theory Discussion IUCEE: Algorithms**Text books**IUCEE: Algorithms**University of Washington Course**• Algorithm Design, by Jon Kleinberg and Eva Tardos, 2005. • Ten week term • 3 lectures per week (50 minutes) • Midterm, Final CSE 421 Introduction to Algorithms (3) Techniques for design of efficient algorithms. Methods for showing lower bounds on computational complexity. Particular algorithms for sorting, searching, set manipulation, arithmetic, graph problems, pattern matching. Prerequisite: CSE 322; CSE 326. IUCEE: Algorithms**Course overview**• Stable Marriage (2) • Basic Graph Algorithms (3) • Greedy Algorithms (2) • Graph Algorithms (4) • Divide and Conquer and Recurrences (5) • Dynamic Programming (5) • Network Flow and Applications (5) • NP Completeness (3) IUCEE: Algorithms**Analyzing the course and content**• What is the purpose of each unit? • Long term impact on students • What are the learning goals of each unit? • How are they evaluated • What strategies can be used to make material relevant and interesting? • How does the context impact the content IUCEE: Algorithms**Overall course context**• Senior level elective • Students are not required to take this class • Approximately half the students take this course • Theory course: no expectation of programming • Data structures is a pre-requisite • Little coordination with data structures course • Some overlap in material • Generally different instructors • Text book highly regarded by faculty • Course is “algorithms by techniques” IUCEE: Algorithms**Stable Marriage**• Very interesting choice for start of the course • Stable Marriage is a non-standard topic for the class • Advanced algorithm to start the class with new ideas • Show a series of different algorithmic techniques IUCEE: Algorithms**All of Computer Science is the Study of Algorithms**IUCEE: Algorithms**How to study algorithms**• Zoology • Mine is faster than yours is • Algorithmic ideas • Where algorithms apply • What makes an algorithm work • Algorithmic thinking**Introductory Problem:Stable Matching**• Setting: • Assign TAs to Instructors • Avoid having TAs and Instructors wanting changes • E.g., Prof A. would rather have student X than her current TA, and student X would rather work for Prof A. than his current instructor. IUCEE: Algorithms**Formal notions**• Perfect matching • Ranked preference lists • Stability m1 w1 m2 w2 IUCEE: Algorithms**m1: w1 w2**m2: w2 w1 w1: m1 m2 w2: m2 m1 Example (1 of 3) m1 w1 m2 w2**Example (2 of 3)**• m1: w1 w2 • m2: w1 w2 • w1: m1 m2 • w2: m1 m2 m1 w1 m2 w2 Find a stable matching**m1: w1 w2**m2: w2 w1 w1: m2 m1 w2: m1 m2 Example (3 of 3) m1 w1 m2 w2**A closer look**• Stable matchings are not necessarily fair m1 w1 m1: w1 w2 w3 m2: w2 w3 w1 m3: w3 w1 w2 w1: m2 m3 m1 w2: m3 m1 m2 w3: m1 m2 m3 m2 w2 m3 w3 How many stable matchings can you find?**Example**m1 w1 m1: w1 w2 w3 m2: w1 w3 w2 m3: w1 w2 w3 w1: m2 m3 m1 w2: m3 m1 m2 w3: m3 m1 m2 m2 w2 m3 w3**Intuitive Idea for an Algorithm**• m proposes to w • If w is unmatched, w accepts • If w is matched to m2 • If w prefers m to m2, w accepts • If w prefers m2 to m, w rejects • Unmatched m proposes to highest w on its preference list that m has not already proposed to**Algorithm**Initially all m in M and w in W are free While there is a free m w highest on m’s list that m has not proposed to if w is free, then match (m, w) else suppose (m2, w) is matched if w prefers m to m2 unmatch (m2, w) match (m, w)**Does this work?**• Does it terminate? • Is the result a stable matching? • Begin by identifying invariants and measures of progress • m’s proposals get worse • Once w is matched, w stays matched • w’s partners get better (have lower w-rank)**Claim: The algorithm stops in at most n2 steps**• Why? Each m asks each w at most once**When the algorithms halts, every w is matched**• Why? • Hence, the algorithm finds a perfect matching**The resulting matching is stable**• Suppose • m1 prefers w2 to w1 • How could this happen? m1 w1 m2 w2 m1 proposed to w2 before w1 w2 rejected m1 for m3 w2 prefers m3 to m1 w2 prefers m2 to m3**Result**• Simple, O(n2) algorithm to compute a stable matching • Corollary • A stable matching always exists**Basic Graph Algorithms**• This material is necessary review • Terminology varies so cover it again • Formal setting for the course revisited • Big Oh notation again • Debatable on how much depth to go into formal proofs on simple algorithms IUCEE: Algorithms**Polynomial time efficiency**• An algorithm is efficient if it has a polynomial run time • Run time as a function of problem size • Run time: count number of instructions executed on an underlying model of computation • T(n): maximum run time for all problems of size at most n • Why Polynomial Time? • Generally, polynomial time seems to capture the algorithms which are efficient in practice • The class of polynomial time algorithms has many good, mathematical properties IUCEE: Algorithms**Ignoring constant factors**• Express run time as O(f(n)) • Emphasize algorithms with slower growth rates • Fundamental idea in the study of algorithms • Basis of Tarjan/Hopcroft Turing Award IUCEE: Algorithms**Formalizing growth rates**• T(n) is O(f(n)) [T : Z+ R+] • If n is sufficiently large, T(n) is bounded by a constant multiple of f(n) • Exist c, n0, such that for n > n0, T(n) < c f(n) • T(n) is O(f(n)) will be written as: T(n) = O(f(n)) • Be careful with this notation IUCEE: Algorithms**Explain that there will be some review from 326**Graph Theory • G = (V, E) • V – vertices • E – edges • Undirected graphs • Edges sets of two vertices {u, v} • Directed graphs • Edges ordered pairs (u, v) • Many other flavors • Edge / vertices weights • Parallel edges • Self loops By default |V| = n and |E| = m IUCEE: Algorithms**Breadth first search**• Explore vertices in layers • s in layer 1 • Neighbors of s in layer 2 • Neighbors of layer 2 in layer 3 . . . s IUCEE: Algorithms**Testing Bipartiteness**• If a graph contains an odd cycle, it is not bipartite IUCEE: Algorithms**Directed Graphs**• A Strongly Connected Component is a subset of the vertices with paths between every pair of vertices. IUCEE: Algorithms**Topological Sort**• Given a set of tasks with precedence constraints, find a linear order of the tasks 321 322 401 142 143 341 326 421 370 431 378 IUCEE: Algorithms**Greedy Algorithms**• Introduce an algorithmic paradigm • Its hard to give a formal definition of greedy algorithms • Proof techniques are important • Need to formally prove that these things work • New material to students IUCEE: Algorithms**Greedy Algorithms**• Solve problems with the simplest possible algorithm • The hard part: showing that something simple actually works • Pseudo-definition • An algorithm is Greedy if it builds its solution by adding elements one at a time using a simple rule IUCEE: Algorithms**Greedy solution based on earliest finishing time**Example 1 Example 2 Example 3 IUCEE: Algorithms**Scheduling all intervals**• Minimize number of processors to schedule all intervals IUCEE: Algorithms**Algorithm**• Sort by start times • Suppose maximum depth is d, create d slots • Schedule items in increasing order, assign each item to an open slot • Correctness proof: When we reach an item, we always have an open slot IUCEE: Algorithms**Scheduling tasks**• Each task has a length ti and a deadline di • All tasks are available at the start • One task may be worked on at a time • All tasks must be completed • Goal minimize maximum lateness • Lateness = fi – di if fi >= di IUCEE: Algorithms**Example**Time Deadline 2 2 3 4 2 3 Lateness 1 3 2 Lateness 3 IUCEE: Algorithms**Determine the minimum lateness**Show the schedule 2, 3, 4, 5 first and compute lateness Time Deadline 6 2 4 3 4 5 5 12 IUCEE: Algorithms**Homework Scheduling**• Tasks to perform • Deadlines on the tasks • Freedom to schedule tasks in any order IUCEE: Algorithms**Greedy Algorithm**• Earliest deadline first • Order jobs by deadline • This algorithm is optimal This result may be surprising, since it ignores the job lengths IUCEE: Algorithms**Analysis**• Suppose the jobs are ordered by deadlines, d1 <= d2 <= . . . <= dn • A schedule has an inversion if job j is scheduled before i where j > i • The schedule A computed by the greedy algorithm has no inversions. • Let O be the optimal schedule, we want to show that A has the same maximum lateness as O IUCEE: Algorithms**Shortest Paths and MST**• These graph algorithms are presented in the framework of greedy algorithms • Students will have seen the algorithms previously • Attempt is made to have students really understand the proofs • Classical results IUCEE: Algorithms**Assume all edges have non-negative cost**Dijkstra’s Algorithm S = {}; d[s] = 0; d[v] = infinity for v != s While S != V Choose v in V-S with minimum d[v] Add v to S For each w in the neighborhood of v d[w] = min(d[w], d[v] + c(v, w)) 4 3 y 1 u 1 1 0 1 4 s x 2 2 2 2 v 2 3 5 IUCEE: Algorithms z**Proof**• Let v be a vertex in V-S with minimum d[v] • Let Pv be a path of length d[v], with an edge (u,v) • Let P be some other path to v. Suppose P first leaves S on the edge (x, y) • P = Psx + c(x,y) + Pyv • Len(Psx) + c(x,y) >= d[y] • Len(Pyv) >= 0 • Len(P) >= d[y] + 0 >= d[v] y x s u v IUCEE: Algorithms**http://www.cs.utexas.edu/users/EWD/**• Edsger Wybe Dijkstra was one of the most influential members of computing science's founding generation. Among the domains in which his scientific contributions are fundamental are • algorithm design • programming languages • program design • operating systems • distributed processing • formal specification and verification • design of mathematical arguments IUCEE: Algorithms**Greedy Algorithm 1Prim’s Algorithm**• Extend a tree by including the cheapest out going edge 15 t a 6 14 3 9 4 e 10 13 c s 11 20 5 Construct the MST with Prim’s algorithm starting from vertex a Label the edges in order of insertion 17 2 7 g b 8 f 22 u 12 1 16 v IUCEE: Algorithms**Application: Clustering**• Given a collection of points in an r-dimensional space, and an integer K, divide the points into K sets that are closest together IUCEE: Algorithms