The cabinet minister said infighting was “part and parcel of politics” and dismissed the row over Tory donor Frank Hester’s ‘racist’ comments as ‘trivial’