A union representing job centre staff says three in five feel unsafe interacting with the public.