The buildings were built with the permission of the duchess Johanna Emerencia Gallas. They should have been used most probably as a manor, but soon they became manufactory for production of drapery and cotton textiles. The Refuge is actually a technical monument; it is one of the oldest manufacturies in Bohemia. The production probably did not live up to the expectations and a mangle was set up in No 66 already in 1767. The potter’s workshop for production of stonewear bottles for the healing water from Spa Libverda was opened here in 1812. However, this production did not last long. Subsequently, the interiors of the buildings were turned into flats for office workers, and it was finished at the latest in 1850. In 1896, a refuge for children (Kinderasyl) was established here. The name which is used up until now comes from that time. The Refuge was not only used by orphans but also by children of the workers from Hrádek’s factories. After the independent Czechoslovak Republic was established, there were three classes of Czech school in the building No 67 in 1919–1925, and they were closed down when the new Czech school building was built (nowadays the Primary School of T. G. Masaryk). After the war, the classes returned to the Refuge, and later on the buildings were used as a school canteen and after-school club until 1982. The depository from Hrádek’s museum was moved to the loft in the 1960s, the museum was closed down shortly afterwards. The state company Potraviny (Groceries) rented the premises since 1982. The delicatessen production was here until 1991. Furthermore, not successful privatisation took place when the buildings were used only as a deposit to a bank and the buildings were deteriorating. The council managed to get the buildings at the end of the 90s, but only the north wing where the flats are now was saved. The west wing was demolished to the ground level in 2006 after the roof had collapsed.
Both of the buildings of the Refuge were one-storey buildings with cellars. There were five dormer windows in the mansard roofs; when the buildings were reconstructed the number of windows increased to 10 on each of the longer sides and two dormer windows were added facing the street. The stone portal was left in the facade of the house No 67 to show where the original entrance was. On the wall along the street, the Gallas stone coat of arms is a reminder of the original owners.
The place where the Refuge is would be suitable for the fortification due to its position. Therefore, it is possible that the Hrádek’s castle or the palace used to stand here. That might also be the possible source of the name of the town (Hrádek means ‘Little Castle’). However, there is no proof for such an assertion in terms of archaeological findings yet.
During the archaeological diggings there were found remains (cellars) which were part of an older building. During the dig out of the telephone cable west of the palace at the end of the 20th century, fragments dating back to Renaissance period were found there by surface survey. More complex survey was not held then.