Ballarat could become out of reach as more people from Melbourne move in