It’s one of those wonders of technology!
There are essentially three parts to the technology:
One, to check account balance and decide whether to dispense the requested amount at all. This involves the ATM switch software doing a realtime check with the core banking system of the bank whose ATM / debit card you’ve inserted into the ATM slot.
Two, if the ATM switch approves the dispensing of the requested amount, then to calculate the number of notes of each denomination to dispense. This depends primarily on the requested amount and the level of stock of notes of different denominations inside the ATM machine. This is relatively simple arithmetic although some degree of optimization may be carried out.
Third, to actually dispense the currency notes as computed in the second step. Most of the wonders of technology happen in this part. As some of the other answers have explained, ATM has separate cassettes for different denominations of currency notes and uses sophisticated optical technology to “count” how many notes it’s dispensing.
UPDATE DATED 6 APRIL 2020:
Made a few edits and added this addendum basis recent learnings.
In border towns of certain countries in Asia (e.g. Cambodia) and Europe (e.g. Switzerland), there’s relatively free movement of people from one country to another. A coworker in Basel in Switzerland used to cross over to Germany side of Basel everyday on the way to work to downtown Basel in Switzerland.
A few ATMs in such places store and dispense currency notes of the home country and the neighboring country across the border.