On Hampstead Hill, the ridge where the Pagoda now stands, Baltimoreans rallied on September 12, 1814 to protect the city from the threat of a British invasion. By water, British troops entered the Patapsco River and bombarded Fort McHenry. By land, they amassed forces at North Point. As they marched on to Baltimore and looked up to Hampstead Hill they saw Rodger’s Bastion – including 100 cannons and 20,000 troops. This sight led the British to return to their ships and leave the Port of Baltimore.
This blog has a nice map on the post entitled "The British Route from Washington to Baltimore in 1814," including Hampstead Hill.