Back to journal

Data · · 1 min read

When the data grows, the algorithm matters

An introduction to Big O, and a small experiment for the next time your Python code slows down.

When the data grows, the algorithm matters

Expanded from my Telegram note. The English edition is a translation and editorial adaptation.

A Python script can feel fast on a small sample and become uncomfortable on the full dataset. Before changing hardware, it helps to ask how the work grows with the input.

That is why I shared Mosh Hamedani’s introduction to data structures and algorithms. It connects Big O, memory use and basic structures to a question we meet in data work: what happens when the input gets much larger?

Count the work

Imagine checking whether each customer identifier appears in another list. Scanning that list again for every customer repeats work. Building a set once can change the cost of the later membership checks, while using additional memory. The actual result depends on the operation, implementation and data.

Make the difference visible

Try the same operation on increasing input sizes. Record elapsed time and memory, keeping the environment consistent. Repeat each measurement rather than trusting one run.

A nested loop is a useful place to investigate, but it is not automatically a bug. Sometimes the input is small enough that readability matters more. Sometimes an optimized library already does the expensive part efficiently.

A next step

Take one slow function from a practice project. Write down what its loops repeat, predict how doubling the input changes the work, then measure. The point is to connect a complexity argument to behavior you can observe.

Ali Rashedi