Definition
These modules connect Python to your computer. pathlib handles file paths. os reads environment variables and system information. sys gives access to command-line arguments and the interpreter. shutil copies and moves files.
In simple words
They're how your program looks around the computer: where am I, which files are here, what did the user type, and please copy this file over there.
Why it matters
Data pipelines read folders of files, and API keys come from environment variables. Scripts that work on one machine and not another usually have a path problem.
How it works
In code
import os
import shutil
import sys
import tempfile
from pathlib import Path
folder = Path(tempfile.mkdtemp())
data = folder / "reports" / "sales.csv"
data.parent.mkdir(parents=True)
data.write_text("day,amount\nMon,100\n", encoding="utf-8")
print("Name:", data.name, " Suffix:", data.suffix)
print("Exists:", data.exists())
print("CSV files:", [p.name for p in folder.rglob("*.csv")])
shutil.copy(data, folder / "copy.csv")
print("Now in folder:", sorted(p.name for p in folder.iterdir()))
print("API key set?", os.environ.get("MY_API_KEY") is not None)
print("Python version:", sys.version_info.major)
shutil.rmtree(folder)Output
Name: sales.csv Suffix: .csv Exists: True CSV files: ['sales.csv'] Now in folder: ['copy.csv', 'reports'] API key set? False Python version: 3
Common mistakes
- Joining paths with + and \ by hand, which breaks across systems. Use pathlib's /.
- Calling shutil.rmtree on a path you haven't checked. It deletes with no recycle bin.
Check yourself
Summary
- pathlib for paths, os for environment, sys for the interpreter, shutil for copying.
- Keep secrets out of code.
Finished reading?
Sources: Python docs: pathlib