Lord Hayward said the Tories were forecast to lose many more council seats in May and there is “no evidence” polls are any more accurate six weeks later