Prime minister’s gaffe was a defining moment in his campaign symbolising a party that does not know what it stands for and has nothing to offer