Harper's no longer master of his own house