Skip to main content

Consuming APIs with requests

Introduction​

APIs (Application Programming Interfaces) are how modern software talks to other software — a weather app calling a forecast service, a script pulling records from a company's internal system, or a mobile app saving data to the cloud. The requests library is Python's de facto standard for calling APIs as a client: sending HTTP requests and reading responses, with a much friendlier interface than Python's built-in urllib.

This chapter uses a free public test API, https://jsonplaceholder.typicode.com, which simulates a blog's posts, comments, and users — no API key required, so you can run every example as-is.

By the end of this chapter you will be able to:

  • Send GET and POST requests and parse JSON responses.
  • Pass query parameters, headers, and authentication tokens correctly.
  • Handle HTTP errors and timeouts defensively instead of assuming every request succeeds.

Setup:

pip install requests
import requests
print(requests.__version__)   # e.g. 2.31.x

Concepts​

The request/response cycle​

  • A GET request retrieves data; it should never change server state.
  • A POST request sends data, typically to create something new.
  • PUT and DELETE update and remove a resource respectively.
  • Every response carries a status_code (200s = success, 400s = client error, 500s = server error), headers, and a body — response.json() parses a JSON body into Python dicts/lists.

Query parameters and headers​

  • Pass query parameters via the params dict, not by hand-building the URL string — requests handles URL-encoding for you.
  • Pass custom headers via the headers dict — commonly used for content negotiation (Accept) and identifying the client (User-Agent).

Sending data​

  • Use the json argument (not data) to send a dictionary as a JSON body — requests serializes it and sets Content-Type: application/json automatically.

Defensive error handling​

  • response.raise_for_status() raises requests.exceptions.HTTPError for 4xx/5xx responses instead of leaving you to check status_code manually.
  • Always set a timeout — a request with no timeout can hang forever if the server never responds.
  • requests.exceptions.RequestException is the base class for every error requests can raise (HTTPError, Timeout, ConnectionError, ...), so catching it last acts as a catch-all.

Sessions and authentication​

  • A requests.Session() reuses the underlying TCP connection and lets you set headers or auth once for many requests, rather than repeating them on every call.
  • auth=(user, pass) sends HTTP Basic authentication.

Examples​

GET and response basics:

import requests

response = requests.get('https://jsonplaceholder.typicode.com/posts/1')

print(response.status_code)              # 200
print(response.headers['Content-Type'])  # application/json; charset=utf-8
print(response.json())                   # {'userId': 1, 'id': 1, 'title': '...', 'body': '...'}

Query parameters and headers:

params = {'userId': 1}
headers = {'Accept': 'application/json', 'User-Agent': 'python-lab/1.0'}

response = requests.get(
    'https://jsonplaceholder.typicode.com/posts',
    params=params,
    headers=headers,
)

posts = response.json()
print(response.url)     # .../posts?userId=1
print(len(posts))       # 10

POST with a JSON payload:

new_post = {
    'title': 'Learning requests',
    'body': 'This post was created from a Python lab.',
    'userId': 1,
}

response = requests.post('https://jsonplaceholder.typicode.com/posts', json=new_post)

print(response.status_code)  # 201
print(response.json())       # {'title': '...', 'body': '...', 'userId': 1, 'id': 101}

Defensive error handling:

try:
    response = requests.get('https://jsonplaceholder.typicode.com/posts/9999', timeout=5)
    response.raise_for_status()
    print(response.json())
except requests.exceptions.HTTPError as e:
    print(f'HTTP error: {e}')
except requests.exceptions.Timeout:
    print('The request timed out.')
except requests.exceptions.RequestException as e:
    print(f'Request failed: {e}')

Sessions:

session = requests.Session()
session.headers.update({'Authorization': 'Bearer YOUR_TOKEN_HERE'})

response = session.get('https://jsonplaceholder.typicode.com/posts/1')
print(response.status_code)

Challenges​

1. Fetch a single post and user​

  • GET /posts/5 and print just the title field from the JSON response.
  • GET /users/3 and print the user's name and email.
💡 Show Answer
import requests

post = requests.get('https://jsonplaceholder.typicode.com/posts/5').json()
print(post['title'])

user = requests.get('https://jsonplaceholder.typicode.com/users/3').json()
print(user['name'], user['email'])

2. Query parameters and headers​

  • Fetch all comments for post ID 1 using the /comments endpoint with a postId query parameter.
  • Add a custom header X-Lab-Student with your name and confirm it was sent by inspecting response.request.headers.
💡 Show Answer
import requests

response = requests.get(
    'https://jsonplaceholder.typicode.com/comments',
    params={'postId': 1},
    headers={'X-Lab-Student': 'Alex Chen'},
)
comments = response.json()
print(f'{len(comments)} comments returned')
print(response.request.headers['X-Lab-Student'])  # Alex Chen

3. POST, PUT, and DELETE​

  • Create a new post with your own title and body, and print the id the API assigned.
  • Use requests.put() to update post 1's title, then requests.delete() to delete post 1.
  • Compare the status codes returned by each.
💡 Show Answer
import requests

BASE = 'https://jsonplaceholder.typicode.com/posts'

created = requests.post(BASE, json={'title': 'My new post', 'body': 'Hello', 'userId': 1})
print('POST:', created.status_code, created.json()['id'])  # 201

updated = requests.put(f'{BASE}/1', json={'title': 'Updated title', 'body': '...', 'userId': 1})
print('PUT:', updated.status_code)   # 200

deleted = requests.delete(f'{BASE}/1')
print('DELETE:', deleted.status_code)  # 200

POST returns 201 Created (a new resource exists); PUT and DELETE both return 200 OK on this test API, though a real API commonly returns 204 No Content for a successful DELETE.

4. A safe_get helper​

  • Write a function safe_get(url) that returns the parsed JSON on success and None on any request exception.
  • Deliberately mistype a domain name and observe which exception type is raised.
  • Set timeout=0.001 on a real request and confirm a Timeout exception is raised.
💡 Show Answer
import requests

def safe_get(url, **kwargs):
    try:
        response = requests.get(url, timeout=kwargs.pop('timeout', 5), **kwargs)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f'safe_get failed: {e}')
        return None

safe_get('https://jsonplaceholder.typicode.com/posts/1')   # works, returns dict
safe_get('https://this-domain-does-not-exist.example')     # ConnectionError -> None
safe_get('https://jsonplaceholder.typicode.com/posts/1', timeout=0.001)  # Timeout -> None

5. Sessions and authentication​

  • Create a Session, set a custom header on it once, and make three different GET requests through it.
  • Try https://httpbin.org/basic-auth/user/pass with the wrong password and observe the status code.
  • In a comment, explain why storing an API token directly in source code is a bad practice, and name one alternative.
💡 Show Answer
import requests

session = requests.Session()
session.headers.update({'X-Lab-Student': 'Alex Chen'})

for post_id in (1, 2, 3):
    r = session.get(f'https://jsonplaceholder.typicode.com/posts/{post_id}')
    print(post_id, r.status_code)

wrong_auth = requests.get('https://httpbin.org/basic-auth/user/pass', auth=('user', 'wrong'))
print(wrong_auth.status_code)  # 401

# A token committed to source code ends up in git history forever, and anyone
# with read access to the repo (or a leaked clone) can use it. Load it instead
# from an environment variable (os.environ['API_TOKEN']) or a secrets manager,
# so it can be rotated without a code change and never appears in version control.