PYTHON / FILE HANDLING
Binary files and bytes
Read and write raw bytes with 'rb'/'wb', convert between bytes and str, and inspect or patch individual byte positions.
What you will learn
- Open files with 'rb'/'wb' to get bytes objects with no decoding applied
- Know that b[0] is an int 0-255 while b[0:1] is a one-byte bytes object
- Bridge str and bytes with .encode(enc) and .decode(enc) at the boundary
- Read large binary files in fixed chunks, stopping when read(n) returns b''
Understanding Binary files and bytes
Every file on disk is a sequence of bytes. Text mode ('r', 'w') stacks two conveniences on top of that: it decodes bytes into str using an encoding, and it translates line endings. Binary mode removes both layers, so open('f', 'rb').read() hands you a bytes object containing exactly the byte values stored in the file, and a binary write puts exactly the byte values you supply. That is what you want for images, zip archives, compiled files, or anything whose bytes are not text in a known encoding.
The mental model for bytes is an immutable sequence of integers from 0 to 255. Its repr is misleading at first because Python prints printable ASCII bytes as characters: bytes([0x89, 0x50, 0x4E, 0x47]) shows up as b'\x89PNG', but nothing was decoded. Because the elements are integers, data[0] gives 137 rather than b'\x89', while data[0:1] gives the one-byte bytes object. When you need to change bytes in memory, use bytearray, the mutable sibling that supports buf[8] = 51 and .append().
Positions behave differently in binary mode too. f.seek(n) in a binary file means byte n from the start, and f.tell() returns a real byte count, so you can jump straight to a header field or a known offset. In text mode those numbers are opaque cookies because one character may occupy several bytes. Combine 'r+b' with seek and write to patch a file in place without rewriting it, and use f.read(n) in a loop when the file is too big to hold in memory.
data = bytes([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]) # PNG signature
with open("sample.bin", "wb") as f:
n = f.write(data)
print("bytes written:", n)
with open("sample.bin", "rb") as f:
head = f.read(4)
rest = f.read()
print("head:", head)
print("head[0]:", head[0], type(head[0]).__name__)
print("head[0:1]:", head[0:1])
print("hex:", head.hex(" "))
print("is PNG:", head == b"\x89PNG")
print("rest length:", len(rest))Binary mode transfers raw byte values with no encoding or newline translation, and a bytes object is an immutable sequence of integers 0-255.
Worked examples
Bytes are not text until you decode them
Shows that a binary read returns the encoded byte count, and that decoding only works with the right codec.
text = "caf\u00e9"
raw = text.encode("utf-8")
print(raw, len(raw), len(text))
with open("note.bin", "wb") as f:
f.write(raw)
with open("note.bin", "rb") as f:
back = f.read()
print(back == raw)
print(back.decode("utf-8"))
try:
back.decode("ascii")
except UnicodeDecodeError as e:
print("decode failed:", e.reason)Example explained
Line 1The 4-character string becomes 5 bytes because é is two bytes in UTF-8.
Line 2Writing raw in 'wb' mode and reading it back in 'rb' mode gives an identical bytes object, byte for byte.
Line 3decode("utf-8") reassembles the two-byte sequence into one character.
Line 4decode("ascii") raises UnicodeDecodeError because byte 0xc3 has no ASCII meaning; the bytes themselves are fine, the codec is wrong.
Chunked reading
Reads a binary file in fixed 4-byte pieces instead of loading it whole.
with open("blob.bin", "wb") as f:
f.write(bytes(range(10)))
total = 0
with open("blob.bin", "rb") as f:
while chunk := f.read(4):
print(len(chunk), chunk.hex())
total += len(chunk)
print("total:", total)Example explained
Line 1bytes(range(10)) builds the byte values 0 through 9 without any text involved.
Line 2f.read(4) returns at most 4 bytes, so the last chunk is short rather than padded.
Line 3At end of file read(4) returns the empty bytes object b'', which is falsy, so the walrus loop stops.
Line 4Memory use stays at one chunk regardless of file size.
Patching a byte in place
Uses 'r+b' with seek to overwrite a single byte, and bytearray to do the same in memory.
with open("patch.bin", "wb") as f:
f.write(b"VERSION 1 DATA")
with open("patch.bin", "r+b") as f:
f.seek(8)
print("byte at 8:", f.read(1))
print("tell:", f.tell())
f.seek(8)
f.write(b"2")
with open("patch.bin", "rb") as f:
print(f.read())
buf = bytearray(b"VERSION 1")
buf[8] = ord("3")
print(bytes(buf))Example explained
Line 1'r+b' opens for reading and writing without truncating, unlike 'wb' which would erase the file.
Line 2seek(8) moves to byte offset 8, the digit, because offsets in binary mode are plain byte counts.
Line 3Reading advances the position, so the second seek(8) is needed before writing over that same byte.
Line 4bytearray allows item assignment; buf[8] = ord("3") stores the integer 51 at that index.
Important notes
Binary mode has no encoding or newline layer, so open('f.bin', 'rb', encoding='utf-8') raises ValueError: binary mode doesn't take an encoding argument.
f.read() with no size argument copies the entire file into memory; for multi-gigabyte files use chunks, or memoryview over a chunk to slice it without copying.
Common mistakes
Calling f.write("abc") on a file opened with 'wb'; Python raises TypeError: a bytes-like object is required, not 'str' because binary mode has no encoder attached.
Comparing data[0] == b'\x00' to test the first byte; this is always False since data[0] is the int 0, so header checks silently never match - use data[0] == 0 or data[0:1] == b'\x00'.
Opening an image or zip with 'r' instead of 'rb'; on Windows the \r\n to \n translation quietly corrupts the data, and on any platform the decode step usually raises UnicodeDecodeError.
Try it yourself
Change, predict, then run
Write the bytes b'\x1f\x8b\x08\x00hello' to a file in binary mode, then reopen it and print whether the first two bytes match the gzip signature b'\x1f\x8b', plus the hex of the whole file.
Open the Python workspaceCheck your understanding
A file's first byte on disk has the value 0xFF. After data = open('f.bin', 'rb').read(), which expression evaluates to True?
- data[0] == 255
- data[0] == b'\xff'
- data[0] == '\xff'
- data[0:1] == 255
Show answer
Indexing a bytes object returns the integer value of that single byte, so data[0] is 255. data[0] == b'\xff' is tempting because slicing does produce bytes, but comparing an int to a bytes object is simply False; the bytes form would be data[0:1] == b'\xff'. Comparing to the str '\xff' fails too, since no decoding happened.