The businessman has been banned from Britain on security grounds but is understood to have visited Buckingham Palace and met two former Prime Ministers