A Downing Street spokesman says the PM apologised but some families reject his remarks.