First European settlers to the area arrived in 1633, with more settlers arriving in 1639. The history of the village is closely linked to sugar farming, and Charlotteville's deep-water harbour was also of importance for the town's development. In more recent times fishing has been important for Charlotteville's economy.
Charlotteville has one of the few remaining tamboo bamboo bands in which rhythms are produced by banging bamboo on the ground. This originated in slavery times when slaves were not allowed to play musical instruments. Today a saxophone player is in the band.
There are a few small restaurants. There are no large hotels; there are only a few small guesthouses and apartments to rent. There is a beach in the village and Pirate's Bay is twenty minutes' walk along a track.
Hidden at the other side of the village is Pirates Bay, also known as "Little Italy" for its popularity in playing football.