Sorting

1. Introduction to Sorting Problem

1.1 Importance of Sorting

1.2 Benefits of Studying Sorting Algorithms

Learning Design Strategies

Algorithm Analysis

1.3 Approach to Learning Sorting Algorithms

1.4 Summary


2. Design Strategy: Brute Force - Selection Sort

2.1 Selection Sort Algorithm

def selection_sort(A):
    n = len(A)
    for i in range(n):
        min_index = i
        for j in range(i + 1, n):
            if A[j] < A[min_index]:
                min_index = j
        # Swap the found minimum element with the first element
        A[i], A[min_index] = A[min_index], A[i]
    return A

2.2 Correctness Argument

2.3 Performance Analysis

Time Complexity

Space Complexity

2.4 Importance

2.5 General Notes on Algorithm Analysis

2.6 Summary


3.Basics of Asymptotic Analysis

3.1 Quantifying Algorithm Running Time

3.2 Defining Running Time as a Function of Input Size

3.3 Basic Operations

3.4 Analyzing Selection Sort

Algorithm Overview

Counting Operations

Total Running Time

3.5 Asymptotic Analysis

Dominant and Lower-Order Terms

Simplifying Running Time

Implications of $ \Theta(n^2) $

3.6 Importance in Algorithm Analysis

3.7 Practical Application in Interviews


4. Brute Force: Bubble Sort

Introduction

Bubble Sort Algorithm

Algorithm Steps

  1. For each index $i$ from $0$ to $n - 1$:
  2. For each position of the j pointer:
  3. Result:

Pseudocode

def bubble_sort(A):
    n = len(A)
    for i in range(n):
        for j in range(n - 1, i, -1):
            if A[j - 1] > A[j]:
                # Swap adjacent elements
                A[j - 1], A[j] = A[j], A[j - 1]
    return A

Correctness Argument

Time Complexity Analysis

Comparison with Selection Sort

Practical Considerations

Notable Remarks

Summary


5. Decrease and Conquer Strategy and Insertion Sort

Algorithm Design Strategies

# Design Strategy Description Pros/Cons Examples
1. Brute Force - Directly based on the problem statement. ✅ Straightforward
❌ May vary between individuals due to subjectivity.
Selection Sort, Bubble Sort
2. Decrease and Conquer - Decrease the problem size ($n$) to size $n-1$. ✅: Systematic approach to problem-solving.
✅: Facilitates recursive thinking.
2.1 TOP-DOWN - Do upfront work to nibble away one (or few) element(s): $n \rightarrow n-1$

MAIN IDEA:
1. I pretend to be a Lazy Manager.
2. I nibble away at the problem to make it smaller.
3. I do upfront work of adding the nibble to the solution.
4. I hand over my solution and the remaining problem to my subordinate. (They do the same.)

CODING TIP: Write code from the perspective of an arbitrary lazy manager to sort the array:
..BEFORE: The managers above me would have ensured that all previous elements are sorted. (because they are lazy managers too!)
..MY JOB: I just extend the sorted region by one more element.
..AFTER: I hand over the remaining work to my subordinate. (They do the same.)
Selection Sort, Bubble Sort
2.2 BOTTOM-UP - Work at the end to extend the $(n-1)^{st}$ solution: $n-1 \rightarrow n$

MAIN IDEA:
1. I pretend to be a Lazy Manager.
2. I nibble away at the problem to make it smaller. I keep the nibble for work to be done later.
3. I hand over the remaining problem to my subordinate. (They do the same.)
4. The subordinate returns their solution to me.
5. I do the (later) work of adding my nibble to the solution.
6. I hand over my solution to my manager.
Insertion Sort

Coding Insertion Sort:

def insertion_sort(nums):
  pass
def insertion_sort(nums):
  
    temp = nums[i]
def insertion_sort(nums):

    temp = nums[i]
    j = i - 1             # j is initialized to the index of the last element in the subordinate's subarray
def insertion_sort(nums):

    temp = nums[i]
    j = i - 1

    while __________ nums[j] > temp:    # As long as `nums[j]` is bigger than `temp`, 
      nums[j + 1] = nums[j]             # We keep right shifting `nums[j]` to the right.
      j -= 1                            # We also keep moving the `j` index one step to the left.
def insertion_sort(nums):

    temp = nums[i]
    j = i - 1

    while __________ nums[j] > temp:
      nums[j + 1] = nums[j]
      j -= 1
    
    nums[j + 1] = temp                  # We insert the `temp` value into the position to `j`'s right.
def insertion_sort(nums):
  
  for i in range(1, len(nums)):          # Manager index `i` varies from 1 to n-1.
    
    # All the following arbitrary manager work is repeated by each manager in a bottom up order.
    temp = nums[i]
    j = i - 1
    
    while __________ nums[j] > temp:
      nums[j + 1] = nums[j]
      j -= 1
    
    nums[j + 1] = temp

  return nums                             # Return the sorted version of the original array.
def insertion_sort(nums):
  
  for i in range(1, len(nums)):
    temp = nums[i]
    j = i - 1
    
    while j >= 0 and nums[j] > temp:      # j must always be greater than or equal to 0.
      nums[j + 1] = nums[j]
      j -= 1
    
    nums[j + 1] = temp

  return nums

DESIGN STRATEGY #1: Brute Force

DESIGN STRATEGY #2: Decrease and Conquer

Decrease and Conquer Applied to Sorting

Top-Down Approach (Forward Decrease)

Bottom-Up Approach (Backward Decrease)

Insertion Sort Algorithm

Concept

Efficient Implementation

Example

Time Complexity Analysis

Key Points for Interviews

Summary