Ongoing Monitoring of Websites

While websites are required to undergo a web accessibility review before launching or significant redesign, it is important to remember that web content always changes. Some departments have multiple editors who contribute to a website and this increases the risk of inadvertent accessibility barriers. It is therefore very important for website managers to monitor the health of their websites at specific intervals.

The SF State ATI team assists with this ongoing monitoring by providing the campus with an automated assessment tool. This automated assessment tool, Compliance Sheriff, can be a valuable resource in ensuring accessibility of your websites. You may set up a scan to run at an interval of your choosing and have a report emailed to you once complete.

The ATI team is available to help you create an account in Compliance Sheriff, set up scan groups, scans, notifications and prepare website accessibility reports. Please contact us for further information.

Getting Started with Compliance Sheriff

Compliance Sheriff is a web application that will help you perform automated accessibility assessments on your website. It can crawl through all the pages on your website and generates reports on the accessibility issues associated with each page. Keep in mind that this tool is not intended to replace a manual assessment, but rather as a way to identify the most frequent accessibility errors. This tool also helps you validate broken links on your website.

If you do not already have an account, please request an account. You can then log in to the Compliance Sheriff website with your username and password.

Screenshot of Compliance Sheriff Dashboard

Dashboard

When you first log in, you will see the Compliance Sheriff Dashboard. From the Dashboard, you may perform a quick scan for your website, view the results and save the scan for future reuse. If you would like to perform a quick scan, click the Scan button and wait until the scan results appear on the right. You may click the Save button to pin the scan result to Dashboard. Note: We do not recommend the use of quick scans because they are limited to 20 pages, which does not give you the flexibility needed for scanning your entire website.

Scans

The Scans navigation consists of two sub-navigations. The “Scans” sub navigation tab allows you to create, modify and view scans. The “Groups” sub navigation will let you create, modify and view Scan Groups.

Monitors

A monitor is a collection of checkpoints that will be executed on specific pages of a website. This is an advanced feature for administrators and does not need to be used.

Checkpoints

This page displays the available checkpoints and checkpoint groups. This is also an advanced feature for administrators and does not need to be used.

Reports

On this page you will find Trend, Summary and Custom Views.

  • Trends show you the changes in accessibility over time for specific Scan Groups. For more information on this feature, contact the ATI team. 
  • Summaries are results of scans that have been completed. Clicking the health result for a scan on the Scan or Scan Groups tab will display a summary of the scan or scan group on this tab. If you select the Summary tab without clicking a scan result, a summary of the last-run scan group is displayed.These provide graphical and tabulated summaries of scan results.
  • Custom views allow you to customize views of summaries. For more information on this feature, contact the ATI team. 

Alerts

An alert is an e-mail message that is automatically sent at the end of a scan or monitor or at scheduled intervals. You may use this in combination with a scheduled scan to notify you that a scan is complete.

1. Select the Scans tab at the top of the page and click on the New button. Note that if you are already in a scan group (indicated on the top), the new scan will be created in that scan group

Screenshot of Creating Scan 2

2. On the next page, click "Show Advanced Options." This will allow you to enter the specific information of the scan. You will need to choose a unique display name, website address (URL), and the checkpoints to test against. The SF State Web Standards use the CSU ATI Accessibility Requirements checkpoint group, so please use this for scans. 

Screenshot of Creating Scan 2

3. Expanding the advanced options will give you further control over the scan. We recommend you choose no more than 3 levels. You may also set a page limit under the Options if your website is large. The standard page limit is 200. 

Screenshot of Creating Scan 3

4. Once configured, click the Save button.

5. To run the scan, click on the Run button. Compliance Sheriff does not automatically run a created scan. 

Screenshot of Creating Scan 4

Once a scan is complete, you may click on the Health % link to view the results of the scan.

The scan results page has six quadrants and below is a brief explanation of each

  • Scan statistics: Displays the number of pages scanned, how long the scan took to run and the total occurrences returned by the scan. Also displays the most important issues detected by the scan (priority 1 checkpoints)
  • Top 10 checkpoints: Displays the 10 checkpoints with the highest occurrence counts in that category.
  • Top 10 checkpoints changed via results revision: Displays the results associated with checkpoints you've modified in the Results Revision Wizard.
  • Top 10 issues: Displays a list of the ten results that Compliance Sheriff recommends for immediate action. More information below. 
  • Top 10 pages: Displays the ten pages containing the highest number of checkpoint violations.
  • Trend: Displays detailed statistics of a scan.

Clicking on the links under these quadrants will give you a closer look at the error and where it exists on a web page. For further information, please review the Help link in compliance sheriff or contact the ATI Team. 

Top 10 issues and viewing errors

When viewing the results of a scan, click “Top 10 Issues” tab. Top 10 issues displays a list of the ten results that Cryptzone recommends for immediate action. While you should focus on the failures list, it is also important to review the warnings to ensure they are not accessibility barriers.

Screenshot of Scan Results 1

You can click on a result item to view a rendered version of the first page that has this specific accessibility issue.

  • The first item on the top left (1/6 in the screenshot below) shows the number of failed pages with this accessibility issue. In this case, it is 6 total pages and you can toggle between pages that have this accessibility issue using the arrows provided.
  • On the right of this selector is a section shows a Summary of the accessibility issue.
  • To the right of the summary, is an Occurrences drop-down list that shows you all occurrences of the issue on that specific page. If there is more than one occurrence, you can use the drop-down to cycle between them. As you change this drop-down, the accessibility issue is usually highlighted on the rendered version of the page below. 
  • The How to Fix button takes you to a website that further explains this accessibility issue and provides guidance on how to fix it.
  • The Code source button lets you toggle between the code view and the rendered view of the page. This is helpful if you cannot identify the error in the rendered view of the page. 
Screenshot of Scan Results 2

To re-run the accessibility assessment scan, navigate to the scans sub-navigation and click on the run button next to the saved scan.

Cyxtera has provided a training video on how to use Compliance Sheriff. Please refer to the table of contents below the video if you would like to skip to that topic in the video. If you would like access to a captioned version of this video, please send us an email at access@sfsu.edu

List of topics in training video
Topic Time (minutes)
Compliance Sheriff Overview  1:22
Checkpoint Groups and Modules 4:37
Checkpoints  9:05
Regular Expressions and Us...  17:29
Scans  21:47
Scan Groups  47:47
Views  57:54
Notifications  86:16
Scorecard and Scan Summary  87:33
Monitors  98:09
Result Revision Wizard  104:04
User Administration  110:11
Conclusion 119:46