Health secretary Matt Hancock says it is “unfair” that some people have continued to socialise.