The Arawak people, the original inhabitants of Pigeon island, were driven out by the Caribs around 1000 A.D. The Caribs lived in caves along the shore of the island, and hunted for fish, shellfish, small animals and birds. They also foraged for plant food. In the 16th century, a French pirate, François le Clerc, used Pigeon Island as his base. He forged an agreement with the Caribs, so that they would not attack his ship.
Between 1779 and 1782, Admiral George Rodney took over Pigeon island, and built Fort Rodney. To establish clear viewpoints, Rodney ordered all trees on Pigeon Island to be cut down. From the higher peak, Signal Hill, Rodney was able to observe the French naval base of Fort Royal on Martinique. In 1782, Admiral Rodney sailed from Pigeon Island to confront the French fleet, which he defeated in the Battle of Saints.
Over the next several years, the British built several other structures on Pigeon Island, including two barracks, a mess hall, and a lime kiln.