Skip to main content

Handling Timeouts

Introduction​

In the world of web scraping or API calls, it's inevitable to encounter issues like server latency or network instability. These problems might make your application wait indefinitely for a response. To avoid such situations, it's crucial to know how to handle timeouts in Python using the requests library.

Understanding Timeouts​

A timeout is the maximum amount of time a program or operation will wait for a process to complete. If the process doesn't finish within the specified time, an error is raised. In requests, the timeout parameter determines how long the client will wait for the server to respond before giving up.

Setting Timeouts​

In requests, you can set a timeout by passing the timeout parameter in seconds to your request. Below is an example of a GET request with a timeout:

import requests

try:
    response = requests.get('https://www.example.com', timeout=5)
except requests.exceptions.Timeout:
    print("The request timed out")
except requests.exceptions.RequestException as err:
    print ("Something else went wrong", err)
else:
    print("Success!")

In this example, if the server does not respond within 5 seconds, a requests.exceptions.Timeout exception is raised.

Timeout Types​

There are two types of timeouts: Connect Timeout and Read Timeout.

  • Connect Timeout: The time it takes to establish a connection with the server.
  • Read Timeout: The time it takes to read the response from the server.

You can set these timeouts individually in requests by passing a tuple to the timeout parameter with the first element being the connect timeout and the second being the read timeout. 

response = requests.get('https://www.example.com', timeout=(2, 5))

In this example, the connection to the server will timeout if it takes longer than 2 seconds, and the server's response will timeout if it takes longer than 5 seconds to read.

Handling Timeouts​

When a timeout exception is raised, you can handle it just like any other exception in Python. This generally involves using a try/except block. In the except block, you can choose what should happen if a timeout occurs. 

import requests

try:
    response = requests.get('https://www.example.com', timeout=5)
except requests.exceptions.Timeout:
    print("The request timed out, retrying...")
    # Here you can add a retry logic
except requests.exceptions.RequestException as err:
    print ("Something else went wrong", err)

In this example, the program will print an error message if a timeout occurs and can be set to retry the request.

Conclusion​

Timeouts are a crucial part of error and exception handling when working with the requests library in Python. Knowing how to properly set and handle timeouts can help make your applications more robust and reliable.