Farms have been reporting a slowdown in EU labour since the UK voted to leave.