SiteMorse Q & A
Home / Site Building & Testing / SiteMorse Q & A
This thread has been split from
http://www.accessifyforum.com/...
This thread is intended to be a one-on-one discussion. Please keep comments until the end of the Q and A session. [edit made on Friday 5 May 2006 at 11:20]
Jon R from SiteMorse has agreed to answer some specific questions about SiteMorse and their testing Methodology. This set of questions is designed to interrogate the validity of SiteMorse test results and the accuracy of the league tables.
Here's the first just to get some context:
Q1. What is your full name and What is your relationship to SiteMorse?
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Last edited by Grant Broome on 05 May 2006 10:26 am; edited 2 times in total
Share: 
| Grant Broome wrote: |
| Q1. What is your full name and What is your relationship to SiteMorse? |
Share:
Thanks Jon,
I guess I should really introduce myself too.
I'm Grant Broome, Web Accessibility Manager for CDSM. CDSM is partnered with the Shaw Trust and co-developed the Shaw Trust Web Accreditation.
Q.2 The SiteMorse League tables measure websites against each other based on WCAG 1.0 checkpoints. How many Priority 1 checkpoints can be comprehensively measured by SiteMorse alone and require no manual verification?
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Share:
| Grant Broome wrote: |
| Q.2 The SiteMorse League tables measure websites against each other based on WCAG 1.0 checkpoints. |
OK, I'll address this bit first, because it seems to be a common misconception. No, we do not rank websites against each other based on accessibility. The league tables (which are a completely separate topic to the SiteMorse product itself) rank sites based upon a combination of many factors, only one of which is based on the WCAG 1.0 checkpoints.
| Grant Broome wrote: |
| How many Priority 1 checkpoints can be comprehensively measured by SiteMorse alone and require no manual verification? |
Now that depends on your definitions (and also prompts the query "are you asking the right question?"). Some priority 1 failures, such as missing titles on frames or missing alt attributes on images, can easily be positively identified by automated software. Of course, this does not mean that if a title or an alternative text is provided that its content is appropriate. That's why SiteMorse will never tell you that you have "passed" accessibility, only whether or not it has identified failures. However, any failures that SiteMorse does identify do not need "manual verification".
Of course, I suspect the answer you are fishing for is "none", if by "comprehensive" you mean that all possible failures (whatever that means, given the ambiguous nature of many of the guidelines) will be positively identified and none will be missed. Since your post was basically a statement masquerading as a question, I'll ask you a question: why is your question relevant?
Share:
| Jon R wrote: |
| Some priority 1 failures, such as missing titles on frames or missing alt attributes on images, can easily be positively identified by automated software. |
A missing alt attribute on images is not a priority 1 failure. The alt attribute is not the only way of providing a text equivalent to an image. At worst this should only be marked as a warning at priority 1, that for example, it is only a best practice to have an alt attribute on an image.
Share:
| Jon R wrote: |
| Since your post was basically a statement masquerading as a question, I'll ask you a question: why is your question relevant? |
I think you put it very well earlier: "...we're on the same side!"
Always assume good intention.
_________________
My CV type thing and my Life of Ben (Blog). Nigel Peck's Accessify Forum Requirements.
Share:
| Isofarro wrote: |
| A missing alt attribute on images is not a priority 1 failure. |
Share:
| Cerbera wrote: |
| Accusing Grant of asking disengenuious questions is neither civilised nor professional behaviour, imho. |
I said no such thing. However, I said I would answer "genuine questions". That question was not genuine - Grant already knew the answer, and was trying to make a point. That's fair enough, which is why I addressed the point he was making.
| Cerbera wrote: |
| Always assume good intention. |
I am trying hard to do so, however you yourself are not practicing what you preach - it is certainly not "assuming good intention" when you try and re-write what I said into an accusation that I did not make.
Share:
Well this has started off well, then. _________________
The Watchmaker Project - my personal blog
29digital Design Studio - freelance web design/development
Share:
Question: How does the Sitemorse program arrive at a numerical score? Are all compnents of WCAG 1.0 which you can test weighted equally?_________________
Accessibility != Bobby
Share:
| vigo wrote: |
| Question: How does the Sitemorse program arrive at a numerical score? Are all compnents of WCAG 1.0 which you can test weighted equally? |
I'm not sure if this was meant to be a free-for-all, however in answer to your question: the exact calculation used to arrive at a "score out of 10" for a site is available here: https://secure.sitemorse.com/benchmark.html
(In summary, all checkpoints of equal priority have equal weight.)
Share:
Just a quick one off of the back of your last response Jon; the test goes over multiple parts (x)html validity, speed, accessibility etc. are these all weighted in equal measure to derive the league tables?_________________
Red Ant
Share:
| Richard Conyard wrote: |
| Just a quick one off of the back of your last response Jon; the test goes over multiple parts (x)html validity, speed, accessibility etc. are these all weighted in equal measure to derive the league tables? |
Well, that's answered comphrensively in the URL I gave, but the short answer is no, different categories such as performance or errors have different weightings.
Share:
| Jon R wrote: |
| Since your post was basically a statement masquerading as a question, I'll ask you a question: why is your question relevant? |
Trying to get back on track...
It's very relevant, because SiteMorse regularly produces news releases like this:
| SiteMorse, 27th April 2006 wrote: |
| Accessibility Compliance – results of automated testing against the mandatory requirements of Priority 1 (A) Accessibility (sites are also tested for AA compliance): 25 (last month 21) sites passed all Accessibility A tests, 58 (last month 54) sites had less than 10% of pages with failures. 77 (last month 80) sites failed AA tests on every page. |
The disclaimer given reads:
| SiteMorse wrote: |
| The range of tests [Web Accessibility Initiative WAI] that can be completed automatically are limited, 100% compliance with the automated tests does not mean 100% compliance with the requirements. |
If it isn't 100% then what is it? Without information about the proportion of checkpoints being reliably tested by your software it's impossible to put any value on the results or your reports.
If it's 4 out of 16 then it's next to useless as an absolute or a comparative measure, since the known facts are dwarfed by the unknown. If it's 12 out of 16 then the statistical merit of the results is getting close to significant.
_________________
Dan Champion, Champion IS, Mooch Marketing, Revish
Share:
Jon, thanks for your answer.
At the start of this thread I stated that the questions are:
| Quote: |
| ...designed to interrogate the validity of SiteMorse test results and the accuracy of the league tables. |
I will ask questions that do just that and my second question was completely in line with that statement. You publish league tables based on how one site performs next to another. How many checkpoints you can comprehensively measure is a critical aspect of the accuracy of these tables. That is why my question is relevant.
If you do not want to answer a question in future, please say that you have no comment and we will move on.
I will note that in your answer to Q.2, you have not provided an exact number although you suggest that I believe the answer to be "none".
| Jon R wrote: |
| I'm not sure if this was meant to be a free-for-all |
Actually it wasn't. I would prefer to keep this one-on-one so we don't wander off topic. Would anyone mind if we kept it like that? We can have some general Q and A at the end if Jon is happy with that.
The next question is similar to Question 2. It is relevant to web managers that I have met and spoken to who believe that some web sites with a higher SiteMorse accessibility ranking than their own may in fact be far less accessible- bringing into question the validity of automated accessibility testing in league tables. This is an opportunity for you to respond.
Q.3 Is it theoretically possible that a web page that has 0% WCAG priority 1 checkpoint compliance could still be awarded 100% SiteMorse accessibility compliance?
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Share:
All times are GMT
You cannot post new topics in this forumYou cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum