Gregory St silos to come down