python type annotation

Python is well known as a dynamic typing language. This means that every value is checked at runtime without the need to explicitly specify the type of a variable. However, as large projects become more complex and multiple developers collaborate, understanding and maintaining the code can become difficult. To address this, Type Annotation was introduced in Python 3.5. Using type annotations helps improve code readability, prevent bugs, and enhance autocompletion features.

1. Basics of Type Annotation

Type annotation is the syntax that allows explicitly specifying the types of variables or functions. Here are the basic ways to annotate the types of variables and functions:

Variable Annotation:
x: int = 10
y: float = 10.5
name: str = "Alice"

Function Annotation:
def greeting(name: str) -> str:
    return "Hello " + name

1.1 Type Annotation of Variables

By specifying the type of a variable, the code writer can clearly indicate which type is expected. This allows for early error detection through static analysis in tooling and IDEs.

1.2 Annotations for Function Parameters and Return Values

Function annotations can explicitly specify the input and output types of a function, helping to anticipate what type of data the function will receive. This is very helpful during code reviews.

2. Built-in Data Types

Python supports various built-in data types, and these types can be used in annotations.

  • Basic types like int, float, str, bool, None, etc.
  • Container types like List, Dict, Set, Tuple can be further refined using the typing module.
from typing import List, Dict, Tuple

names: List[str] = ["Alice", "Bob", "Charlie"]
scores: Dict[str, int] = {"Alice": 95, "Bob": 85}
position: Tuple[int, int] = (10, 20)

3. Union and Optional

When multiple types are allowed, it is common to use Union, and when None is allowed, Optional is used.

from typing import Union, Optional

value: Union[int, float] = 5.5

def get_user(id: int) -> Optional[Dict[str, str]]:
    if id == 1:
        return {"name": "Alice", "role": "admin"}
    return None

4. User-defined Types

When you need to define complex types, using Type or NewType allows you to write clearer code.

from typing import NewType

UserID = NewType('UserID', int)
admin_user_id: UserID = UserID(524313)

4.1 Type Alias

Using type aliases allows you to express complex type structures with concise names.

Vector = List[float]

def normalize(vec: Vector) -> Vector:
    magnitude = sum(x**2 for x in vec) ** 0.5
    return [x / magnitude for x in vec]

5. Generic Types

Using generic types allows a single function or class to work with multiple types. You can define generic types using the typing.Generic class.

from typing import TypeVar, Generic

T = TypeVar('T')

class Box(Generic[T]):
    def __init__(self, content: T) -> None:
        self.content = content

int_box = Box(123)
str_box = Box("hello")

6. Advanced Example

Here is a slightly more complex example utilizing type annotations.

from typing import List, Dict, Union, Callable

def process_data(data: List[Union[int, float, str]]) -> Dict[str, Union[int, float]]:
    result: Dict[str, Union[int, float]] = {'total': 0, 'numeric_count': 0}

    def is_number(val: Union[int, float, str]) -> bool:
        return isinstance(val, (int, float))

    for item in data:
        if is_number(item):
            result['total'] += item  # Prevents type warnings.
            result['numeric_count'] += 1

    return result

mixed_data: List[Union[int, float, str]] = [10, '20', 30.5, 'forty', '60', 70.2]
output = process_data(mixed_data)
print(output)
# {'total': 110.7, 'numeric_count': 3}

7. Static Type Checking Tools

Type annotations are most useful when used with static type checking tools. In Python, tools like mypy, Pyright, and Pylance are widely used.

For example, mypy is used as follows:

mypy script.py

These tools are very effective in checking the type consistency of the code and preventing unexpected type errors.

8. Conclusion

Type annotation is a powerful feature of Python that greatly helps improve code readability, ease maintenance, and prevent errors early. Additionally, when combined with static analysis tools, it provides greater stability for large projects. Through this tutorial, I hope you will be able to effectively utilize type annotations and write more robust Python code.

Python Iterators and Generators

In programming, iterable objects and their usage are essential for large-scale data processing. Python provides two powerful tools for performing these tasks: iterators and generators. In this article, we will delve deeply into the concepts of iterators and generators, their differences, and how to use them.

Iterator

An iterator is a protocol that represents an object that can be iterated upon, providing an interface to traverse the elements of the object. In Python, an iterator is created by implementing the __iter__() method and the __next__() method. These are automatically called when iterating in a loop and are generally useful when handling large amounts of data.

How iterators work

To understand how an iterator works, we need to look more deeply at the two methods.

  • __iter__()Returns the iterable object, i.e., it returns the object itself. This method is called when the iteration starts. The iterable object is used to obtain an iterator from the starting point.
  • __next__()Returns the next value of the data through iteration. If no more data is available, it should raise a StopIteration exception. This method is called to fetch the next item from the iterable that has the items grouped for iteration.

Simple iterator example

Below is a simple example code of a counter iterator:


class Counter:
    def __init__(self, low, high):
        self.current = low
        self.high = high

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.high:
            raise StopIteration
        else:
            self.current += 1
            return self.current - 1

counter = Counter(1, 5)
for number in counter:
    print(number)
    

In the above example, the Counter class follows the iterator protocol by implementing the __iter__() and __next__() methods. Objects of this class can be used in loops (for loop).

Generator

A generator is a special function that helps to create an iterator more simply, using the yield keyword to return values one at a time. When called, a generator returns a generator object, which is run when the generator function is used to iterate over values and can pause and resume from where it left off when called again.

How generators work

Generators internally automatically implement the __iter__() and __next__() methods, hiding these implementations from the user. Therefore, when a generator function is called, a generator object is returned, which can be used like an iterator.

Generator example

Below is a simple example code of a generator function:


def simple_generator():
    yield 1
    yield 2
    yield 3

for value in simple_generator():
    print(value)
    

In the above example, the simple_generator() function returns values one at a time using the yield keyword every time it is called. This generator can be used in a for loop like other iterators.

Differences between iterators and generators

Iterators and generators have many similarities, but there are a few important differences:

  • Simplicity of implementation: Generators can be implemented more intuitively and simply using the yield keyword. This eliminates the complexity of writing iterators manually.
  • State preservation: Generators automatically preserve their state. When a generator is paused, it remembers all current states, so calling yield continually keeps that state intact.
  • Memory usage: Generators do not generate results immediately and create values one at a time as needed, making them memory efficient. Compared to iterators, they are more useful for processing large-scale data.

Advanced usage example

Generators can be combined with complex logic to write highly efficient code. Below is an example of generating the Fibonacci sequence using a generator:


def fibonacci_generator():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib_gen = fibonacci_generator()
for _ in range(10):
    print(next(fib_gen))
    

In this example, the fibonacci_generator generates an infinite Fibonacci sequence, and you can output as many values as needed using a for loop or the next() function.

Practical applications

Iterators and generators are often used in situations where it is necessary to process large streams of data or to generate values one at a time without the need to store the entire list of results in memory, optimizing memory usage.

File reading: Each line of a file can be read as a generator to handle larger files in a memory-efficient manner.


def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line.strip()

for line in read_large_file("large_file.txt"):
    print(line)
    

Conclusion

Iterators and generators are very powerful features of Python, and using them can help perform complex and large-scale data processing efficiently and with better readability. By understanding and appropriately utilizing these two concepts, you will be able to write more efficient and scalable code.

I hope this tutorial has helped deepen your understanding of Python iterators and generators. Consider applying this content in your future Python programming journey.

Python Closures and Decorators Closures are a way to remember the environment in which a function was created. They allow you to retain access to variables from an outer function even after that function has finished executing. Decorators, on the other hand, are a way to modify or enhance functions or methods without changing their actual code. In Python, closures are created when a nested function references variables from its enclosing function. Decorators are typically defined as functions that return another function, allowing you to add functionality to existing functions.

In Python programming, closures and decorators are advanced topics that can confuse many beginners and intermediate developers. In this course, we will thoroughly explain the concepts of closures and decorators and how they can be utilized in Python code.

What is a Closure?

A closure is a concept that is created when using nested functions (inner functions). The inner function can reference the local variables of the outer function and has the characteristic of remembering these variables even after the outer function has finished executing. This allows the inner function to ‘capture’ the context of the outer function.

Basic Structure of Closures

To understand the structure of a closure, let’s look at a simple example:

def outer_function(message):
    def inner_function():
        print(message)
    return inner_function

closure = outer_function("Hello, Closure!")
closure()

In the above code, outer_function returns inner_function. closure references the inner function inner_function and is able to access the local variable message of the outer_function. At this time, the message variable can still be accessed in the inner_function even after the outer function has concluded.

Application of Closures: State Retention

Closures provide flexibility in function usage by allowing a function to create instances and are useful when you want to retain state.

def counter():
    count = 0
    
    def increment():
        nonlocal count
        count += 1
        return count
    
    return increment

counter_instance = counter()
print(counter_instance())  # Output: 1
print(counter_instance())  # Output: 2

In this example, the increment function retains the state of the count variable. The nonlocal keyword enables the inner function to reassign the variable of the outer function.

What is a Decorator?

A decorator is a powerful tool that adds additional functionality to existing functions. A decorator is another function that takes a function as an argument, allowing you to dynamically change or extend that function.

Basic Structure of Decorators

Decorators work by taking a function as an argument and returning a new function:

def simple_decorator(func):
    def wrapper():
        print("Before doing something")
        func()
        print("After doing something")
    return wrapper

def basic_function():
    print("I am a basic function.")

decorated_function = simple_decorator(basic_function)
decorated_function()

This code wraps basic_function to add pre-processing and post-processing.

Python’s Decorator Syntax @

Python provides a simple syntax for directly applying decorators to functions. You can wrap functions with a decorator using the @ symbol:

@simple_decorator
def another_function():
    print("I am another function.")

another_function()

Real Example of a Decorator: Measuring Function Execution Time

Here is a practical example of a decorator that measures the execution time of a function:

import time

def timing_decorator(func):
    def wrapper(*args, **kwargs):
        start_time = time.time()
        result = func(*args, **kwargs)
        end_time = time.time()
        print(f"Execution time of {func.__name__} function: {end_time - start_time:.4f} seconds")
        return result
    return wrapper

@timing_decorator
def slow_function():
    time.sleep(1)

slow_function()

In the above example, the timing_decorator measures and prints the execution time of the slow_function. This allows for the extension of function behavior without directly affecting the code.

Combining Closures and Decorators

Closures and decorators are often used together to create robust and flexible program structures. Closures allow decorators to retain state or continuously access certain data.

Conclusion

In this course, we learned about closures and decorators in Python. Closures provide the ability for functions to capture and reference variables from the outer scope, while decorators offer a way to wrap and extend functions in code. A good understanding of these two topics will enable you to write more efficient and powerful Python code.

06 Chapter Python Programming, How Should I Start?

Python is one of the most popular programming languages in the modern programming world, and its range of use and applications is expanding day by day. This language, which is easily accessible to both beginners and professionals, is particularly appealing among developers who value simplicity and productivity. In this article, we will provide the basic information and step-by-step guidelines necessary to get started with Python programming.

1. What is Python?

Python is a high-level programming language that was first released by Guido van Rossum in 1991, designed with a priority on code readability and simplicity. One of Python’s main philosophies is that “the more readable the code, the better it is.” For this reason, Python has the following characteristics:

  • Simple and clear syntax: Python’s syntax is similar to English, making it easy for beginners to learn.
  • Extensive standard library: Provides libraries that can be used in various fields by default.
  • Platform independence: Can run on various operating systems such as Windows, macOS, and Linux.
  • Loose data types: Variables can be handled according to their characteristics without needing to explicitly specify the data type.

2. Installing Python

The first step in starting with Python is to install the Python interpreter. The installation process may vary slightly depending on your operating system. Follow the steps below to install Python.

Installing Python on Windows

  1. Go to the official Python website and download the latest version.
  2. Run the downloaded installer, and make sure to select the “Add Python to PATH” option during the installation process. This setting allows you to use Python commands in the command prompt.
  3. Once the installation is complete, open the “cmd” window and type python --version to verify that the installation was successful.

Installing Python on macOS

  1. Python is installed by default on macOS. However, it is recommended to download the latest installer from the official website to use the most recent version.
  2. Run the downloaded installer and follow the instructions to install.
  3. Open the terminal and enter python3 --version to check the installation.

Installing Python on Linux

  1. Most Linux distributions come with Python pre-installed. To install or update to the latest version, open the terminal and use the commands sudo apt-get update and sudo apt-get install python3.
  2. After installation is complete, check the version with python3 --version.

3. Setting Up the Development Environment

After installing Python, you need to set up an environment to write and execute code. There are several ways to do this, but here are some representative tools:

Python IDLE

Python IDLE is a basic integrated development environment (IDE) provided with Python, offering a lightweight coding and execution environment. It is useful for writing and executing simple scripts.

Visual Studio Code

Visual Studio Code is an IDE developed by Microsoft that offers extensive plugin support and powerful debugging tools. It supports various programming languages and is widely used for Python development. Start programming in Python by installing the Python extension plugin.

Jupyter Notebook

Jupyter Notebook is a widely used tool for data science and machine learning projects that allows you to combine code, explanations, and visualizations in a single document. To install, enter pip install jupyter, and then run it with the jupyter notebook command.

Searching Subdirectories with Python

Dealing with files and directories is very common in programming. In particular, searching subdirectories is used for various purposes such as organizing system files and analyzing log files. In this tutorial, we will learn in detail how to effectively search subdirectories using Python.

Python provides several libraries for manipulating files and directories. In this article, we will explain how to search subdirectories in Python using the os, glob, and pathlib modules. We will compare the pros and cons of each method and present the best usage examples.

Using the os Module

The os module provides various functionalities to interact with the operating system. By using this module, you can create, delete, and modify files and directories, as well as manipulate file paths, attributes, and permissions. In particular, the os.walk() function is useful for traversing the directory tree to search for subdirectories and files.

os.walk() Function

The os.walk() function starts from the root directory and traverses all subdirectories, obtaining the path, directory names, and file names. The return value of the function is a tuple, where each element includes (directory path, list of all subfolders in the directory, list of all files in the directory).


import os

def search_subdirectories(root_dir):
    for dirpath, dirnames, filenames in os.walk(root_dir):
        print(f'Current directory path: {dirpath}')
        print(f'Subdirectories: {dirnames}')
        print(f'Files: {filenames}')
        print('---------------------')
        
search_subdirectories('/mnt/data')  # Specify the path of the root directory to search.

The above code uses os.walk() to output all subdirectories and files of the given directory. It prints the current directory, subdirectories, and files to the console for each path.

Implementing Filtering

If you only need specific file extensions or name patterns from a large directory, you can search for the desired information through filtering. The following code demonstrates filtering so that the user can search for only ‘.txt’ files.


def search_text_files(root_dir):
    for dirpath, dirnames, filenames in os.walk(root_dir):
        text_files = [f for f in filenames if f.endswith('.txt')]
        if text_files:
            print(f'Text files found in {dirpath}: {text_files}')
        
search_text_files('/mnt/data')  # Specify the directory to search.

In the above example, it searches and outputs only files with the ‘.txt’ extension. You can use different patterns or conditions for filtering as needed.

Using the glob Module

The glob module allows you to search for files using Unix style pathname patterns. It easily searches for files that match specific extensions or name patterns through simple pattern matching.

glob.glob() Function

The glob.glob() function returns all path names matching the specified pattern. In Python 3.5 and above, you can recursively search for subdirectories using the ** pattern.


import glob

def search_with_glob(pattern):
    files = glob.glob(pattern, recursive=True)
    for file in files:
        print(file)
        
search_with_glob('/mnt/data/**/*.txt')  # Search for all .txt files including subdirectories

The above code uses the ** pattern to search for all text files in the target directory and its subdirectories. The glob module provides easy-to-use yet very powerful file search functionality.

Conclusion

In this tutorial, we learned how to search subdirectories using Python. The os.walk() function allows for depth-first traversal of the directory tree, while the glob module provides simple and powerful file searching through pattern matching. Each method can be chosen and applied according to the purpose of use.

Another file-related module in Python is pathlib, which provides an object-oriented approach to make file path manipulation more intuitive. We will cover pathlib in a separate tutorial.

I hope this tutorial provides you with the basics and tips you need to perform file exploration tasks. If you have any further questions or would like to discuss specific use cases, please leave a comment in the section below.