The quarry was an extremely dangerous place. All the blasting work meant that unstable rock could fall without warning, burying workmen alive or sweeping them over precipices hundreds of metres high. Between 1822 and 1969 three hundred and sixty-two men were killed altogether, although no more than three men were ever killed at one time. Most of these casualties were overwhelmed in rockfalls.

After a custom-built hospital was built on the shore of Padarn Lake in 1860, men who had had accidents would be taken there immediately to receive medical attention from the hospital doctor. There were a number of such doctors over the years. Perhaps the most famous of these was Dr Mills Roberts, an extremely able man by all accounts. He was also multi-talented: as well as playing football for Wales he was an excellent surgeon who could respond swiftly to the special circumstances of the quarrymen's workplace. Dr Mills Roberts and the workshop smith once created two fully-functioning metal arms for a quarryman who had lost his own arms in an accident.

Long before the coming of the Welfare State, a shilling (5p) — the 'hospital shilling' — would be kept back from the men's pay so that the quarryman, or any member of his family who happened to fall ill, could be treated in hospital for free. It was also quite common to hold charity concerts, where members of the local community would give their talents and labour for free to raise money for injured quarrymen and their families.

‘If there was an accident, quarrymen would go home. Stop work that day. Out of respect, you know’.

There were also more insidious threats to the quarrymen's health — especially tuberculosis and silicosis. Dr Mills Roberts gave the men and their families practical hints on healthy living: their houses should be well aired, they should wear flannel next to the skin and wash frequently. The quarryman should 'live for something beyond tea and bread-and butter, and vary his diet with foodstuffs of all kind, particularly garden vegetables'. His injunction to 'Remember there is No Nourishment in Tea' is revealing, as for many years doctors had been blaming silicosis on the men's habit of drinking tea, strong as tar from being on the brew all day.