Call comes after Bank’s governor Andrew Bailey said the UK should rebuild relations with the European Union as he warned of the economic consequences of leaving the bloc