The first documents about the palace date back to 1284, when Francesco de’ Frescobaldi, the Captain of the Guelph Mass, bought part of the Pipini towered building to establish it as the seat of the foreign courts run by the Magistrates, People’s Captain, Royal Vicar and Commissioner brought from outside the city,  and of the municipal court and the prison. This original core dates back to the thirteenth-century, brick expansion of the Pipini house, facing the current Piazzetta del Bacchino. Craftsmen from Florence expanded the palace between 1334 and 1338. In place of an old portion of the original structure, the current southern wing of the building was built in white limestone. This elegant crenelated structure, with its main, white façade facing east, is characterized by large mullioned windows and beamed, painted halls on the inside.

Later alterations partly modified the outer, medieval look. Some lancet windows - then reinstated - date back to the sixteenth century, as well as the sail of the clock, built in 1534 and designed by Nanni Unghero. The external staircase on the eastern aspect preserves a sixteenth century gallery of arches, while the ramps were renewed in the eighteenth century and restored ‘in style’ in the early twentieth century. In the fourteenth century a stone statue of Robert of Anjou (awarded with the city lordship) was placed in a niche above the access to the first floor. It was then destroyed at the end of the eighteenth century. From 1587 to the beginning of the eighteenth century, part of the ground floor was used as the seat of the Monte di Pietà (who have been present in Prato since 1476).

From the mid-eighteenth century, the palace housed government offices of the Grand Duchy of Tuscany, which were decentralized into larger and more modern structures during the nineteenth century. This led to the neglect of the palace that, by the end of the century, was in a poor condition. After being seriously considered for demolition, between 1890 and the early 1920’s  a number of restoration projects were carried out. In conjunction with these works, the Municipal Gallery, formerly housed in the Palazzo del Comune, was established in 1912. A final, overall restoration was started in 1998 and ended in 2013, September, when the exhibition ‘From Donatello to Lippi. Officina Pratese’ celebrated the return of the museum’s artworks to Palazzo Pretorio.