Towns on the bay include Swansea and Coles Bay.
At the north of the bay the floodplains of the Aspley and Swan rivers has created Moulting Lagoon, an important Ramsar-listed wetland, much of which lies in the Moulting Lagoon Game Reserve. Nearby is the Friendly Beaches Reserve owned and managed by Bush Heritage Australia.
Prior to European settlement the area was occupied by Tasmanian Aborigines. From autumn through winter various bands of the Oyster Bay Tribe congregated around the coastal areas to harvest shellfish and marine vegetables until the end of July, when swans and ducks arrived in the lagoons and riverine areas to lay their eggs and raise their young. In August most of the bands moved up the Little Swanport and Prosser Rivers to the Eastern Marshes to hunt birds, kangaroos and wallabies. The Linetemairrener people lived at the lagoon year round.
Activities include sea kayaking in the sheltered waters on the east side of the bay from Coles Bay exploring the small and isolated beaches on the Freycinet Peninsula. Recreational fishing is also an activity in the bay with flathead, Australian salmon, trevally, trumpeter and squid sometimes caught. Dolphins and Australian fur seals can sometimes be seen and whales often visit the bay in winter.
Aquaculture has been practiced in the area since the early 1970s. Species cultivated commercially include Pacific oysters, native oysters, mussels, scallops, abalone and sea urchins in the Great Oyster Bay and Mercury Passage. An initial marine farming lease set up in 1978 in Pelican Bay, with additional leases granted for shellfish culture since then in the northern section of Great Oyster Bay and the Swan River.
Infestations of rice grass has been reported by oyster farmers from Little Swanport in 2008 with action being taken to eliminate this ecological threat in the area.
Access to most of the Friendly Beaches Reserve is restricted due to the threat of contamination by the soil pathogen cinnamon fungus.