‘We thought there was going to be a small majority. And the good news about that is that folks like myself ... knew that we’re going to potentially have an outsize influence,’ Scott Perry says