Only three out of about 50 homes in Cwm-yr-Eglwys have permanent residents living there.