Before the cistern, the location housed an overground structure that included a basilica and a library. These original buildings were destroyed in a fire in 476 and were rebuilt by presumably a certain Illus - hence the name "(court of) the Basilica of Illus" in the sources. The basilica and octagon were destroyed by the Nika riots in 532. Longinus the city prefect restored the courtyard and buildings; a project that was completed around 542. It is during this same restoration that Justinian built the Basilica cistern below the courtyard (Müller-Wiener, 1977, p. 283-284).
Sources
According to Procopius' Buildings (6th century), the reason for the construction of the Basilica Cistern were the seasonal droughts. The cistern was built under the courtyard of the Imperial Portico. As Crow et al., 2008, p. 17 notes: the construction of this cistern reveals the challenge of acquiring a large public space in the crowded eastern end of the city.
Malalas (6th century) simply states that Justinian built the Basilica Cistern to bring the "Water of Handrian's aqueduct into it" (18.17). Malalas is the first to credit the prefect Longinus for the building activity (and not Justinian) (18.91).
The 7th century Chronicon Paschale largely repeats Malalas.
The 8th-century Parastaseis Syntomoi Chronikai, followed by the 10th-century Patria add an interesting detail: they claim the Basilica cistern was built by Constantine the Great, preceding its actual construction under Justinian by two centuries.
Nonetheless, the 11th-century historian Cedrenus correctly attributes the construction to Justinian again.