About This Book

Why We Wrote This Book

Algorithms is intended for an upper-level undergraduate or graduate course in algorithms, based on our combined 25 years of experience in teaching this course. Our major goals in writing this book were:

Faced with a new computational problem, a designer will often be able to solve it by using one of the algorithms in the book, perhaps after modifying or adapting it slightly. However, some problems cannot be solved by any of the algorithms in this book. For this reason, we present a repertoire of design techniques that can be used to solve the problem and help the reader to develop intuition about which techniques are likely to succeed. The chapters on NP-completeness and how to deal with it also tell how to recognize problems that are hard to solve and which techniques are available in that case.

Working with algorithms should be fun and exciting. The design of algorithms is a creative task requiring the solution of new problems and old problems in disguise. To be successful, we believe that it is important to enjoy the challenge that a new problem poses. To this end, we have included more examples and exercises of a combinatorial and recreational nature than is typical for a book of this type. All too often the challenge of an unsolved problem is experienced as a threat rather than as an opportunity, and we hope that these examples and exercises help to remove the threat.

Examples of real-word applications of algorithms in this book include data compression and the Boyer-Moore-Horspool algorithm, which is used as part of the implementation of agrep. Most sections of the book introduce a motivating example in the first paragraph. As examples, the closest-pair problem begins with a pattern recognition example, and the longest common subsequence problem begins with a discussion of the analysis of proteins.

Algorithm design and analysis are best learned by experience. For this reason, we provide large numbers of worked examples and exercises. Worked examples show how to deal with algorithms, and exercises let the reader practice the techniques. There are over 300 worked examples throughout the book. These examples clarify and show how to develop algorithms, demonstrate applications of the theory, elucidate proofs, and help motivate the material. The book contains over 1450 exercises, from routine to challenging, which were carefully developed through classroom testing. Close attention was paid to clarity and precision. Because some instant feedback is essential for students, we provide answers to about one-third of the end-of-section exercises (marked with "S") in the back of the book. Solutions to the remaining end-of-section exercises are reserved for instructors.

Prerequisites

The principal computer science prerequisite is a data structures course that covers stacks, queues, linked lists, trees, and graphs. A course in discrete mathematics that covers logic, asymptotic notation (e.g., "big oh" notation), and recurrence relations and their solution by iteration is the main mathematics prerequisite. We do not use advanced methods such as generating functions. In one or two places, we use some basic concepts from calculus. The mathematics topics and data structures used in this book are summarized in Chapters 2 and 3. Some or all of these chapters can be used for reference or review, or incorporated into an algorithms course as needed.

Contents

Following the first three chapters (containing an introduction, mathematics topics, and data structures), the book presents five chapters that emphasize design techniques.

Chapter 4 features searching techniques, including novel applications such as region finding in digital pictures.

The divide-and-conquer technique is introduced in Chapter 5. Among the problems considered are a tiling problem, finding the closest pair of points in the plane, and Strassen's matrix product algorithm. Chapter 6 deals with sorting and selection. Divide-and-conquer is used to develop many of the algorithms in this chapter.

Chapter 7 shows how to use the greedy method to develop algorithms. After showing how to use the greedy method in a simple setting (coin changing), we present Kruskal's algorithm, Prim's algorithm, Dijkstra's algorithm, Huffman's algorithm, and a solution of the continuous knapsack problem.

Chapter 8 covers the technique of dynamic programming. As in Chapter 7, we first show how dynamic programming operates in a simple setting (computing Fibonacci numbers). We next revisit the coin-changing problem (from Chapter 7 on the greedy method) and contrast dynamic programming with the greedy method. We then discuss optimal grouping of matrices, the longest common subsequence problem, and the algorithms of Floyd and Warshall.

Chapter 9 discusses text searching techniques, including the Knuth-Morris-Pratt and Boyer-Moore-Horspool algorithms, and algorithms for nonexact searching.

In Chapter 10, we investigate NP-completeness—a theoretical approach to recognizing and understanding the limitations of algorithms. We include many examples from different areas such as cellular phone networks, games, and biological computing to illustrate the ubiquity and universality of this concept.

It is widely believed that NP-complete problems cannot be solved efficiently by algorithms. Nevertheless, these problems arise in applications and have to be solved in practice. Chapter 11, Coping with NP-Completeness, presents a collection of techniques originating in practice and theory to deal with NP-complete problems. Among the approaches discussed are approximation, parameterization, and use of heuristics.

Chapter 12 presents fundamental algorithms for parallel architectures, including algorithms for the PRAM and sorting networks, and offers an introduction to computation in distributed environments.

Pedagogy

Each section concludes with Section Exercises. The book contains over 1100 Section Exercises. Some of these exercises check for basic understanding of the material (e.g., some ask for a trace of an algorithm), while others check for a deeper understanding of the material (e.g., some investigate alternative algorithms). Exercises felt to be more challenging than average are indicated with a star.

Each chapter ends with a Notes section, which is followed by Chapter Exercises. Notes sections contain suggestions for further reading and pointers to references. Chapter Exercises, some of which have hints, integrate the material of the chapter. The book contains over 350 Chapter Exercises. They are, on the whole, more challenging than the Section Exercises. We have included some very challenging Chapter Exercises marked with two stars. These will probably require instructor guidance, and some are appropriate for a small project.

Lower bounds for problems are integrated into the chapters that discuss those problems rather than being segregated into separate chapters. For example, after presenting several sorting algorithms, we discuss a lower bound for comparison-based sorting.

We present and discuss many recent results, for example, parameterized complexity, a recent area of research.

Algorithms are written in pseudocode that is close to the syntax of the familiar C, C++, and Java family of languages. Data types, semicolons, obscure features of the languages, and so on, are not used because we have found that specifying algorithms by writing actual code obscures the algorithm description and makes it difficult for someone not familiar with the language to understand the algorithm. The pseudocode used is completely described in the book.

Figures illustrate concepts, show how algorithms work, elucidate proofs, and motivate the material. Several figures illustrate proofs of theorems. The captions of these figures provide additional explanation and insight into the proofs.

Attention has been given to finding the most direct and easiest to understand proofs of correctness. We present several examples and arguments to show that our time bounds for algorithms are sharp. For example, we show that the upper bound for the worst-case time of Prim's algorithm using a binary heap is sharp and that the upper bound for the worst-case time of Huffman's algorithm is sharp.

Instructor Supplement

An Instructor's Guide is available at no cost from the publisher to instructors who adopt or sample this book. It should be requested from your local Prentice Hall representative. The Instructor's Guide contains solutions to the end-of-section exercises not included in the book and tips for teaching the course.