A former minister says up to 80 Conservative MPs are prepared to vote against the PM’s Chequers deal.