Gringotts is the wizarding bank, run by goblins. It is housed in a huge, white building, which towers over the other shops.
Witches and wizards can place their money and valuables in one of the many underground vaults. These vaults are accessible through very fast cart rides down to your personal vaults where you can reclaim your possessions, as long as you have your key.
What do you do here?
Here you can store possessions and money in one of the many vaults. You can also exchange Muggle money for the wizarding currency of Knuts, Sickles, and Galleons (at an exchange rate of 1 Galleon to 5 pounds, 17 Sickles per Galleon and 29 Knuts to a Sickle).
What does this place have to do with Harry?
Book 1: Harry and Hagrid visit Gringotts to get money to buy school supplies. Here Harry finds out that his parents have left him with quite a bit of money. Also, Hagrid retrieves the Philosopher's/Sorcerer's stone from vault 713 for Headmaster Dumbledore.
Book 2: Hermione and the Weasleys catch up with Harry after his sojourn into Knockturn Alley on the steps to Gringotts. Here Hermione exchanges some Muggle money to buy school supplies and we meet the Grangers for the first time. We also find out how poor the Weasleys are, as their vault only contained one Galleon and a few Sickles.
Book 3: Once Harry gets to Diagon Alley, he withdraws quite a bit of money from his vault. Also, Sirius Black uses money from his Gringotts vault to get the Firebolt for Harry.
Book 4: Mrs. Weasley takes some money from Harry's vault to purchase his school supplies for him.