Premier plays down SA Libs MP speculation