Its French variant Grande-Condore was well-known during the times of French Indochina. Marco Polo mentioned the island in the description of his 1292 voyage from China to India under the name Sondur and Condur. In Ptolemy's Geography, they are referred to as the Isles of the Satyrs.
In 1702, the English East India Company founded a settlement on this island (the English called it 'Pulo Condore') off the south coast of southern Vietnam, and in 1705 the garrison and settlement were destroyed.
In 1787, through the Treaty of Versailles, Nguyễn Ánh (the future Emperor Gia Long) promised to cede Poulo Condor to the French. In exchange Louis XVI promised to help Nguyễn Ánh to regain the throne, by supplying 1,650 troops (1,200 Kaffir troops, 200 artillery men and 250 black soldiers) on four frigates.
In 1861, the French colonial government established Côn Đảo Prison on the island to house political prisoners. In 1954, it was turned over to the South Vietnamese government, who continued to use it for the same purpose. Notable prisoners held at Côn Sơn in the 1930s included Phạm Văn Đồng and Lê Đức Thọ. Not far from the prison is Hàng Dương Cemetery, where some of the prisoners were buried.
During the Vietnam War, prisoners who had been held at the prison in the 1960s were abused and tortured. In July 1970, two U.S. Congressional representatives, Augustus Hawkins and William Anderson, visited the prison. They were accompanied by Tom Harkin (then an aide), translator Don Luce, and USAID Office of Public Safety Director Frank Walton. When the delegation arrived at the prison, they departed from the planned tour, guided by a map drawn by a former detainee. The map led to the door of a building, which was opened from the inside by a guard when he heard the people outside the door talking. Inside they found prisoners were being shackled within cramped “tiger cages”. Prisoners began crying out for water when the delegation walked in. They had sores and bruises, and some were mutilated. Harkin took photos of the scene. The photos were published in Life magazine on July 17, 1970. Recreations of tiger cages can be seen today at the War Remnants Museum in Ho Chi Minh City. In response, Phil Crane, a Republican from Illinois, visited Côn Sơn and stated that the visit and photos were "distortions of truth." The tiger cages, he said, were "cleaner than the average Vietnamese home."
The prison on Côn Sơn Island was closed in 1975 after the Fall of Saigon. The facilities were reopened some years later however, to temporarily incarcerate boat people captured by local coast guards until the late 1980s.
At the request of Secretary of Defense Robert McNamara, the U.S. Coast Guard started pre-construction plans for a chain of Loran-C radio stations to serve Southeast Asia 15 January 1966 in support of Operation Tight Reign during the Vietnam War. The actual construction of Station Con Son began during April with the delivery of construction materials by USCGC Nettle (WAK-169) and award of construction contracts to Morrison-Knudsen Corp. and Brown and Root Company. Station Con Son was one of five stations in the Southeast Asia chain and was designated SH-3 Yankee. It consisted of a 625 foot (191 m) tower, transmitter equipment buildings, fuel tanks, generators and barracks for personnel located on the north end of Con Son Island. The personnel complement for the station was two officers and 23 enlisted men. After commissioning on 2 September 1966 the station began the testing phase of operations and the five station chain was fully operational by 0400 on 28 October, just nine months after the initial request from the Department of Defense. The station provided, along with its sister stations in the chain, signals that allowed aircraft and ships to receive accurate all-weather positioning data for navigation purposes. During January 1973 the operation of the station was turned over to civilian contractors who were responsible to the United States Coast Guard for all functions of the station. The Coast Guard continued to supply logistical and technical support on an as needed basis. When the fall of the South Vietnamese government was imminent, Station Con Son was directed to stay on the air until the last possible minute to provide navigation signals to aircraft and ships fleeing South Vietnam. Station Con Son stayed on the air until 1246 local time on 29 April 1975 after the crew oversped the generators and damaged critical pieces of electronic gear.