Data Structures

Introduction

A very common use of Python is reading data from a file, processing it, and generating a report. This chapter works through that pattern using FIX (Financial Information eXchange) data — a text-based standard used to exchange financial information electronically between systems. A line of FIX data is a series of key=value tokens separated by a delimiter, for example 8=FIX.4.4|9=211|35=D. Each key is a predefined FIX tag number with a specific meaning (8 is BeginString, 9 is BodyLength, 35 is MsgType, and so on).

The exercises here focus on Python's core data structures — primarily dict, but also list, tuple, and set — along with two idiomatic techniques: dictionary comprehensions (building a dictionary in a single expression) and jump tables (a dictionary that maps a key directly to a function, avoiding long if/elif chains).

---

Concepts

Dictionaries

fix_data = {}
fix_data["8"] = "FIX.4.4"
fix_data["9"] = "211"
fix_data["35"] = "D"

print(fix_data["35"])       # D
print(fix_data.keys())      # dict_keys(['8', '9', '35'])
print(fix_data.items())     # dict_items([...])

Splitting and tokenizing strings

line = "8=FIX.4.4|9=211|35=D"
tokens = line.split("|")

result = {}
for token in tokens:
    key, value = token.split("=")
    result[key] = value

print(result)

Dictionary comprehensions

tokens = ["8=FIX.4.4", "9=211", "35=D"]
result = {t.split("=")[0]: t.split("=")[1] for t in tokens}
print(result)

Jump tables (dictionary of functions)

def proc8(value):
    print(f"BeginString: {value}")

def proc9(value):
    print(f"BodyLength: {value}")

def proc_other(value):
    print(f"Unhandled value: {value}")

handlers = {
    "8": proc8,
    "9": proc9,
}

key, value = "8", "FIX.4.4"
handler = handlers.get(key, proc_other)
handler(value)

Other collection types

data = b"hello"           # bytes - immutable binary sequence
mutable_data = bytearray(b"hello")  # mutable binary sequence

from collections import Counter, defaultdict
counts = Counter(["a", "b", "a", "c", "a"])
print(counts)   # Counter({'a': 3, 'b': 1, 'c': 1})

---

Examples

TOKEN_SEPARATOR = "|"
KV_SEPARATOR = "="

def tokenize_line(line):
    tokens = line.strip().split(TOKEN_SEPARATOR)
    result = {}
    for token in tokens:
        if KV_SEPARATOR in token:
            key, value = token.split(KV_SEPARATOR, 1)
            result[key] = value
    return result

def proc8(value):
    print(f"BeginString: {value}")

def proc9(value):
    print(f"BodyLength: {value}")

def proc35(value):
    print(f"MsgType: {value}")

def proc_other(value):
    print(f"Unhandled tag with value: {value}")

HANDLERS = {"8": proc8, "9": proc9, "35": proc35}

def process_data(data):
    for key, value in data.items():
        handler = HANDLERS.get(key, proc_other)
        handler(value)

line = "8=FIX.4.4|9=211|35=D|49=SENDER|56=TARGET"
data = tokenize_line(line)
print(data)
process_data(data)

---

Challenges

1. Handler functions

• Write handler functions proc8(), proc9(), and proc35(), each of which prints the value it receives with a label.
• Write a catch-all function proc_other() for any other tag.

💡 Show Answer

def proc8(value):
    print(f"In proc8 with value {value}")

def proc9(value):
    print(f"In proc9 with value {value}")

def proc35(value):
    print(f"In proc35 with value {value}")

def proc_other(value):
    print(f"In proc_other with value {value}")

---

2. Tokenizing a line of FIX data

• Write a function tokenize_line(line) that splits a line of FIX data into tokens, then splits each token into a key/value pair, and returns a dictionary of all the pairs.
• Read a line from fixdata.txt, tokenize it, and print the resulting dictionary.

💡 Show Answer

TOKEN_SEPARATOR = "|"
KV_SEPARATOR = "="

def tokenize_line(line):
    tokens = line.strip().split(TOKEN_SEPARATOR)
    result = {}
    for token in tokens:
        key, value = token.split(KV_SEPARATOR, 1)
        result[key] = value
    return result

with open("fixdata.txt") as f:
    for line in f:
        data = tokenize_line(line)
        print(data)

---

3. Processing FIX tokens

• Write a function process_data(data) that loops through a dictionary of FIX key/value pairs.
• For each key, call the appropriate handler function (proc8, proc9, proc35, etc.), falling back to proc_other() for anything else.

💡 Show Answer

def process_data(data):
    for key, value in data.items():
        if key == "8":
            proc8(value)
        elif key == "9":
            proc9(value)
        elif key == "35":
            proc35(value)
        else:
            proc_other(value)

---

4. Optimizing with a comprehension and a jump table

• Rewrite tokenize_line() to build the dictionary using a single dictionary comprehension instead of a manual loop.
• Rewrite process_data() to use a jump table — a dictionary that maps each key directly to its handler function — instead of an if/elif chain.

💡 Show Answer

def tokenize_line(line):
    tokens = line.strip().split(TOKEN_SEPARATOR)
    return {t.split(KV_SEPARATOR, 1)[0]: t.split(KV_SEPARATOR, 1)[1] for t in tokens}

HANDLERS = {
    "8": proc8,
    "9": proc9,
    "35": proc35,
}

def process_data(data):
    for key, value in data.items():
        handler = HANDLERS.get(key, proc_other)
        handler(value)

---

5. Exploring other data structures

• Investigate the binary sequence types bytes, bytearray, and memoryview.
• Investigate the array module for storing numbers efficiently.
• Investigate the collections module (e.g. Counter, defaultdict, OrderedDict) as alternatives to the built-in containers.

💡 Show Answer

# bytes / bytearray
b = b"hello"
ba = bytearray(b)
ba[0] = ord("H")
print(bytes(ba))     # b'Hello'

# array module
from array import array
numbers = array("i", [1, 2, 3, 4])
print(numbers)

# collections module
from collections import Counter, defaultdict

counts = Counter(["a", "b", "a", "c", "a"])
print(counts.most_common(1))    # [('a', 3)]

grouped = defaultdict(list)
grouped["fruits"].append("apple")
print(grouped)