Python Advanced Concepts: Iterators, Generators, Decorators & Comprehensions

Introduction

Python provides several advanced features that help developers write cleaner, reusable, and efficient code. Iterators, generators, decorators, and comprehensions are especially useful when working with collections, processing data, and extending function behavior.

In this tutorial, you will learn:

• Iterables and iterators

• The iter() and next() functions

• Creating custom iterators

• Generators and the yield keyword

• Generator expressions

• Decorators and wrapper functions

• Decorators with arguments

• List, dictionary, and set comprehensions

• Practical examples and interview questions

1. What Is an Iterable in Python?

An iterable is an object that can return its elements one at a time. Lists, tuples, strings, sets, and dictionaries are common examples.

Example

fruits = ["Apple", "Banana", "Mango"]

for fruit in fruits:
    print(fruit)

Output

Apple
Banana
Mango

The for loop retrieves elements from the iterable one by one.

2. What Is an Iterator?

An iterator is an object that produces values one at a time. It maintains its current position and supports the iterator protocol through __iter__() and __next__().

Python provides the built-in iter() function to obtain an iterator and next() to retrieve the next value.

Example

numbers = [10, 20, 30]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

Output

10
20
30

Calling next() after all elements have been consumed raises StopIteration.

Handling the End of an Iterator

numbers = [10, 20]

iterator = iter(numbers)

while True:
    try:
        print(next(iterator))
    except StopIteration:
        break

Output:

10
20

A for loop normally handles StopIteration automatically, so it is usually simpler for everyday iteration.

3. Creating a Custom Iterator

You can create your own iterator by defining a class with __iter__() and __next__() methods.

Example

class CountUp:
    def __init__(self, limit):
        self.current = 1
        self.limit = limit

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= self.limit:
            value = self.current
            self.current += 1
            return value

        raise StopIteration

counter = CountUp(3)

for number in counter:
    print(number)

Output

1
2
3

The iterator returns values until the specified limit is reached.

4. What Is a Generator in Python?

A generator is a convenient way to create an iterator. Generator functions use the yield keyword to return a value while preserving their execution state for the next request.

Unlike a typical function that returns and finishes, a generator can pause and resume.

Example

def count_numbers():
    yield 1
    yield 2
    yield 3

for number in count_numbers():
    print(number)

Output

1
2
3

Each call to yield produces one value. Execution resumes from the same point when the next value is requested.

5. The Difference Between return and yield

The return statement exits a function and provides its result. The yield statement produces a value from a generator and suspends execution so it can continue later.

Example Using return

def get_number():
    return 10

print(get_number())

Output:

10

Example Using yield

def get_numbers():
    yield 10
    yield 20
    yield 30

print(list(get_numbers()))

Output:

[10, 20, 30]

Generators are useful when values can be processed one at a time instead of building an entire collection in memory.

6. Generator Expressions

A generator expression creates a generator using a compact syntax similar to a list comprehension, but with parentheses instead of square brackets.

Example

squares = (number ** 2 for number in range(1, 6))

for square in squares:
    print(square)

Output

1
4
9
16
25

Compare the two forms:

# List comprehension
squares_list = [number ** 2 for number in range(1, 6)]

# Generator expression
squares_generator = (number ** 2 for number in range(1, 6))

A list comprehension builds a list immediately. A generator expression produces values as they are requested, which can reduce memory usage for large sequences.

7. What Is a Decorator in Python?

A decorator is a callable that takes another function or callable and returns a modified or wrapped version of it. Decorators are commonly used for logging, timing, authentication, caching, and validation.

The @decorator_name syntax applies a decorator to a function.

Example

def my_decorator(func):
    def wrapper():
        print("Before the function")
        func()
        print("After the function")

    return wrapper

@my_decorator
def greet():
    print("Hello, Python!")

greet()

Output

Before the function
Hello, Python!
After the function

The @my_decorator line is equivalent to assigning the decorated result back to greet.

8. Creating a Reusable Decorator

A reusable decorator should generally support different positional and keyword arguments. The functools.wraps decorator helps preserve useful metadata from the original function.

Example

from functools import wraps

def log_function(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}")
        result = func(*args, **kwargs)
        print("Function completed")
        return result

    return wrapper

@log_function
def add(a, b):
    return a + b

print(add(10, 20))

Output

Calling add
Function completed
30

Here:

• *args accepts positional arguments.

• **kwargs accepts keyword arguments.

• result stores the original function's return value.

• return result ensures the decorated function preserves that return value.

• @wraps(func) helps preserve the original function's metadata.

9. Decorators with Arguments

A decorator can also be configured using arguments. This requires an outer function that accepts the configuration and returns the actual decorator.

Example

from functools import wraps

def repeat(times):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for _ in range(times):
                func(*args, **kwargs)

        return wrapper
    return decorator

@repeat(3)
def greet():
    print("Hello!")

greet()

Output

Hello!
Hello!
Hello!

The repeat(3) configuration tells the decorator to call greet() three times.

10. What Is a List Comprehension?

A list comprehension provides a concise way to create a new list from an iterable, optionally applying a condition.

Syntax

[expression for item in iterable if condition]

Example

numbers = [1, 2, 3, 4, 5, 6]

squares = [number ** 2 for number in numbers]

print(squares)

Output

[1, 4, 9, 16, 25, 36]

List Comprehension with a Condition

numbers = [1, 2, 3, 4, 5, 6]

even_numbers = [
    number for number in numbers
    if number % 2 == 0
]

print(even_numbers)

Output:

[2, 4, 6]

List comprehensions are useful for transforming or filtering collections concisely.

11. Dictionary Comprehension

Dictionary comprehension creates a dictionary using a compact expression.

Example

numbers = [1, 2, 3, 4, 5]

squares = {
    number: number ** 2
    for number in numbers
}

print(squares)

Output

{1: 1, 2: 4, 3: 9, 4: 16, 5: 25}

Dictionary Comprehension with a Condition

numbers = [1, 2, 3, 4, 5, 6]

even_squares = {
    number: number ** 2
    for number in numbers
    if number % 2 == 0
}

print(even_squares)

Output:

{2: 4, 4: 16, 6: 36}

12. Set Comprehension

Set comprehension creates a set, which stores unique elements.

Example

numbers = [1, 2, 2, 3, 3, 4, 5]

unique_squares = {number ** 2 for number in numbers}

print(sorted(unique_squares))

Output

[1, 4, 9, 16, 25]

The set removes duplicate values. sorted() is used to display the result in a predictable order.

13. Practical Example: Processing Sales Data

The following example combines a list comprehension and a generator expression to process sales values.

sales = [1200, 2500, 800, 3200, 1500]

high_sales = [
    amount for amount in sales
    if amount >= 1500
]

total_sales = sum(
    amount for amount in sales
)

print("High Sales:", high_sales)
print("Total Sales:", total_sales)

Output

High Sales: [2500, 3200, 1500]
Total Sales: 9200

The list comprehension selects sales of at least 1500, while the generator expression supplies values to sum().

14. When Should You Use These Features?

Iterators: Use them when values need to be retrieved sequentially through the iterator protocol.

Generators: Use them when values can be produced on demand, especially for large sequences or data-processing pipelines.

Decorators: Use them when you need to add reusable behavior around functions, such as logging or timing.

Comprehensions: Use them to create lists, dictionaries, or sets with concise transformation and filtering logic.

Choose readability over cleverness. A regular loop or function can be better when the logic becomes complicated.

15. Common Mistakes to Avoid

1. Calling next() after an iterator is exhausted without handling StopIteration.

2. Expecting a generator to restart automatically after it has been consumed.

3. Forgetting to return a value from a decorator wrapper when the original function returns a result.

4. Writing decorators that cannot accept the wrapped function's arguments.

5. Using overly complicated nested comprehensions that make code difficult to understand.

6. Assuming a generator expression immediately creates a complete list.

7. Forgetting that a set does not preserve duplicate values.

16. Python Advanced Concepts Practice Questions

1. What is the difference between an iterable and an iterator?

2. Create an iterator that returns numbers from 1 to 5.

3. Write a generator that produces the first ten natural numbers.

4. Explain the difference between return and yield.

5. Create a generator expression that produces even numbers.

6. Write a decorator that displays a message before calling a function.

7. Create a decorator that measures the execution time of a function.

8. Use list comprehension to create squares from 1 to 10.

9. Use dictionary comprehension to map numbers to their cubes.

10. Use set comprehension to remove duplicate squared values.

17. Python Advanced Concepts Interview Questions and Answers

Q1. What is an iterable?

An iterable is an object that can provide its elements one at a time, such as a list, tuple, or string.

Q2. What is an iterator?

An iterator is an object that produces the next value when requested and signals exhaustion with StopIteration.

Q3. What is a generator?

A generator is a convenient way to create an iterator, usually with a function containing yield or a generator expression.

Q4. What is the difference between a list and a generator?

A list stores its elements, while a generator produces values as they are requested.

Q5. What does yield do?

It produces a value from a generator and suspends execution so the generator can resume later.

Q6. What is a decorator?

A decorator takes a function or another callable and returns a wrapped or modified callable.

Q7. What is the purpose of functools.wraps?

It helps preserve metadata such as the original function's name and documentation when creating a wrapper.

Q8. What is list comprehension?

List comprehension is a concise way to create a list by transforming or filtering items from an iterable.

Q9. What is dictionary comprehension?

Dictionary comprehension is a concise way to create dictionaries from an iterable or other data source.

Q10. Why are generators useful?

They produce values on demand, which can reduce memory usage when processing large sequences.

Conclusion

Python's advanced features help developers write efficient, reusable, and readable programs. Iterators and generators support sequential data processing, decorators add reusable behavior to functions, and comprehensions simplify collection creation.

Practice these concepts with small programs and gradually combine them in larger projects to improve your Python programming skills.