pa11y-ci to run our accesibility testing on all GNU social
pages. The process of setting this up was a bit of a ride, but the
results are quite useful:
We also take screenshots of all pages and compare them with the previous, allowing us to spot (possibly) unintended changes in the UI.
The way we do this is by using a
docker image derived from
pa11y-ci installed, which doesn't have an entrypoint. This then
allows us to spin up a webserver, PHP and Redis containers, mimiking a
pa11y-ci uses a JSON config file to tell it
what to do, and handles visiting all pages with the given resolution,
logging in and registering any accesibility violations found.
Note that most of these errors are actually the same.
The saved screenshots are then used to detect changes, with
compare, to generate a report on visual changes.
We were already striving for making GNU social accesible, which is clear from the really low number of violations found, but there's always room for improvement :)