Polster suggests timescale of a potential vote depends on ‘uncertain’ politics and how a potential incoming Labour government approaches relations with the EU