SiteMorse Q & A
Home / Site Building & Testing / SiteMorse Q & A
OK, Thank you Jon for getting back. A couple of points to pick up on here. You've asked some questions which I need to respond to and raised some points that have not been addressed before. I'll respond to these, you can make some final statements if you wish and then I'd like to hand over the discussion to other forum members. If you would like to keep the discussion one-on-one then just let me know and we'll make that request to the forum.
Can I just ask members to try and stay civilised and objective as per Jon's request. I appreciate that this is sometimes difficult when differences of opinion are so strong (as I have found myself), but only discussion of a nature that is not personal or purely subjective (unless an opinion is asked) is going to prove a useful resource.
| Jon R wrote: |
| However, you have still not explained how such a deliberately-constructed test page can even be considered to possess such a quality as "accessibility" or "inaccessibility". |
I don't really understand this response. You seem to be either asking me to explain to you how the pages can be used to test your own tool or you are saying that a deliberately-constructed test page cannot be used to test SiteMorse's ability to check priority 1 compliance. I think that the Juicy Studio's test cases demonstrate instances of inaccessibility very clearly.
What about Bruce's page? His page is not constructed as a trial for automated tools (as far as I am aware). It delivers genuine content. SiteMorse failed to highlight any of the accessibility errors on the page.
How does SiteMorse identify when an error is deliberately-constructed and tell it apart from and error that is genuine?
| Jon R wrote: |
| I don't believe you did answer the question. You have now got closer to answering it, but the obvious next step is - why do you think that? Using the specific example I already gave, let us suppose that SiteMorse generated a priority 2 'fail' if a document contained over, say, 1kB of textual content but did not contain any headings (it does not currently do such a thing, this is just for the sake of argument). You would presumably believe that to be "bad". Why? |
Historically SiteMorse has not tested for the presence of headings on a page and passed sites that don't have a single item of heading markup.
I would not endorse reliance on a tool to tell me if any page required or didn't require a heading. It is not possible to predict intended usage of all pages on the internet. Therefore if pages are found without headings then that needs to be checked manually. To fail the page without having the faintest idea of it's intended purpose and then to publish that result as part of a league table puts pressure on authors to add headings to content when in a particular instance they may not be necessary or counter-productive - this is the key problem of the SiteMorse tool that sets it apart from other accessibility auditing tools.
| Jon R wrote: | ||
Steady on. I have gone to considerable effort in this discussion to keep things civilised and objective. I would appreciate it if you would do me the same courtesy I am doing you. |
I too have found it a considerable effort. Please forgive me for my lapse.
| Jon R wrote: |
| Also note that SiteMorse does not base anything on "passing" accessibility checkpoints |
It should be noted that the SiteMorse only reports failures. Of course, if a report shows that a site has no failures i.e 10/10 for accessibility then this can still mean that the site is inaccessible. Can we agree on this point?
Also, would it be reasonable to presume that many who have read the report will, by implication, believe that if no failures are found in a particular test then they have passed that checkpoint?
| Jon R wrote: | ||
Indeed, given that people do not, in general, do such a thing. |
I have given one example where people have done such things. When the discussion is opened up to other members I am sure that you would like to be informed of other instances where SiteMorse testing may have been side-stepped in order to achieve a higher SiteMorse League Table position and also where the tool may have requirements to "avoid failure of" SiteMorse tests.
| Jon R wrote: |
| No, they are not. The scores are indisputable facts (that's kind've the point of them, after all), but the accessibility or otherwise of your test pages is highly questionable. |
This was the whole point of asking you to test them.
For those who have not had a chance to visit Gez's page There are tests for several priority 1 checkpoints. To test WCAG Checkpoint 1.1 there is one example of an image with the word "Lemon". The alternative text given is "apple". This simple test measures whether the automated tool can identify an invalid text alternative. This error which is easily detected by a manual check was not correctly identified by the SiteMorse tool which reported that no priority 1 errors were found, even though this error was clearly present.
So while it is an indisputable fact that the SiteMorse tool found no errors on the page it is also an indisputable fact that the tool is unable to detect this often serious error which is commonplace on the web and gave the page top marks.
Reference: Inaccessible page by Juicy studio
| Jon R wrote: | ||
Often it is, yes. |
So SiteMorse can "often" tell that one website is more accessible than another.
Thank you. (While I would contest that the result is incidental) Can we agree now that the League Tables are a rough measure of accessibility and not a definitive guide?
| Jon R wrote: |
| The gist of your argument seems to be that because it is not possible to measure everything, we should measure nothing. |
Not at all. You misunderstand me. I use automated tools myself daily (although SiteMorse isn't my choice). I have stated before on this forum and elsewhere that they are an incredibly useful way of supplementing Manual auditing. However, the results of automated testing alone are very limited to verifying only a few AA checkpoints, while all checkpoints can be fully tested manually.
| Jon R wrote: |
| Bear in mind that what we do with automated testing is simply not possible with manual testing - it would be too time-consuming and expensive. |
This is typical of the type of statement that we are used to seeing in SiteMorse press releases.
I think I have been able to demonstrate over the course of this discussion that what you do with automated testing is far more limited than manual auditing. The SiteMorse reports that I have seen state that time saved manual testing on the 2 sites that we have highlighted in this discussion would have been 2 hours in one instance and 29 minutes in the other. It then highlights all the other checkpoints that need manual testing in addition to this time. What metric is used to measure this imaginary time-saving isn't even relevant. I am sure that someone with even the most basic idea of web-accessibility would easily be able to spot at least 1 of the errors missed by the SiteMorse tool within a minute.
While you mention time-saving and cost... For less than the cost of an annual subscription to SiteMorse, a web developer could invest in any one of the commercially available accessibility testing tools - many of which are demonstrably more comprehensive than SiteMorse and sit on the developer's desktop or server. They can be used to test any number of pages in any part of the site at any time that is convenient for the developer. Compliance to any of these tools should in theory result in a higher SiteMorse League Table ranking as they all do the same job to a greater or lesser degree (for anyone still concerned by such matters).
| Jon R wrote: |
| Even if an organisation buys in external consultants to manually verify the accessibility of a web site, they will usually only examine a few pages. |
My response and final statement:
The report generated from the examination of a few pages by an experienced consultant would be many times more insightful and comprehensive than any report generated by the SiteMorse tool. Bear in mind also that if an organisation were to buy external consultancy then the consultant usually has a tool to assist in finding errors. An automated report with manual verification is usually included in the bargain. In my opinion I doubt that any organisation who has had such comprehensive service - especially if it included disabled user testing would feel comfortable on relying on automated testing thereafter.
Some final statements are invited from Jon and then over to the forum. I know some of you must be quite eager to post your questions and comments. I'd like to thank Jon once again for agreeing to take part in this discussion. I'll be around to enter into the discussion if needs be.
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Share: 
| Grant Broome wrote: | ||
I don't really understand this response. You seem to be either asking me to explain to you how the pages can be used to test your own tool or you are saying that a deliberately-constructed test page cannot be used to test SiteMorse's ability to check priority 1 compliance. |
SiteMorse is a tool intended for checking real-life web sites. If what you are giving it as input is not a real-life web site, then the output is meaningless - garbage in, garbage out.
Plus, as I have repeatedly pointed out, and you have repeatedly avoided answering, the very concept of "accessibility" does not apply to such a test page so to criticise SiteMorse for not scoring it in any particular way is ridiculous.
| Grant Broome wrote: |
| What about Bruce's page? His page is not constructed as a trial for automated tools (as far as I am aware). |
Bruce's 'page' isn't a page at all, it's just an alternative stylesheet for csszengarden. It's interesting as an example of what you can do with CSS, but that's it. The HTML is excellent from an accessibility point of view, and even after the CSS is applied it would appear to pass all the Priority 1 checkpoints except perhaps one, and all the Priority 2 checkpoints except three (and if the CSS is removed, which many browsers and tools let you do, it passes them all). Why do you think that the page should be scored badly for accessibility?
| Grant Broome wrote: |
| I would not endorse reliance on a tool to tell me if any page required or didn't require a heading. It is not possible to predict intended usage of all pages on the internet. Therefore if pages are found without headings then that needs to be checked manually. To fail the page without having the faintest idea of it's intended purpose and then to publish that result as part of a league table puts pressure on authors to add headings to content when in a particular instance they may not be necessary or counter-productive |
That's the whole point of my question - I am interested to see if you can think of any situation where it would be counter-productive or even unnecessary to add at least one heading to a page (with more than minimal content). I can't think of any, off the top of my head, but maybe I'm not being imaginative enough.
| Grant Broome wrote: |
| It should be noted that the SiteMorse only reports failures. Of course, if a report shows that a site has no failures i.e 10/10 for accessibility then this can still mean that the site is inaccessible. Can we agree on this point? |
Certainly (with the caveat that accessibility is not a binary true/false property, and therefore it is better to say that a site "has accessibility problems" rather than just "is inaccessible"). That's why we point out in every league table, and every report, that manual testing is also necessary.
| Grant Broome wrote: |
| Also, would it be reasonable to presume that many who have read the report will, by implication, believe that if no failures are found in a particular test then they have passed that checkpoint? |
No. We go specifically out of our way to explain that manual testing is necessary. A SiteMorse report never says "pass" with respect to accessibility or individual checkpoints - the two options are "check" or "fail".
| Grant Broome wrote: |
| When the discussion is opened up to other members I am sure that you would like to be informed of other instances where SiteMorse testing may have been side-stepped in order to achieve a higher SiteMorse League Table position and also where the tool may have requirements to "avoid failure of" SiteMorse tests. |
I would be interested in any anecdotes along those lines, yes, although they would be much more useful if they included information indicating why, if such a thing has been done, they did it that way rather than fixing the issue properly, and also whether or not the site as a result is actually "worse" in some way as opposed to merely unimproved.
| Grant Broome wrote: |
| For those who have not had a chance to visit Gez's page There are tests for several priority 1 checkpoints. To test WCAG Checkpoint 1.1 there is one example of an image with the word "Lemon". The alternative text given is "apple". This simple test measures whether the automated tool can identify an invalid text alternative. |
It measures nothing. Gez knew before he made the page, as you knew before asking me to run SiteMorse on it, that not only does no current accessibility tool identify the "error" (if error it is), it is impossible for any such tool ever to exist (absent the invention of artificial intelligence). An almost identical example is given in our standard documentation discussing automated testing, where we show a picture of a duck with the 'alternative text' "picture of a dog" and explain that SiteMorse will not detect such an issue.
| Grant Broome wrote: |
| Can we agree now that the League Tables are a rough measure of accessibility and not a definitive guide? |
Certainly. If you had asked me that right from the outset I would have readily agreed. This is why I particularly object to being called "unethical" and similar - not only do we not try to hide this, we go out of our way to point it out!
| Grant Broome wrote: | ||
Not at all. You misunderstand me. I use automated tools myself daily (although SiteMorse isn't my choice). I have stated before on this forum and elsewhere that they are an incredibly useful way of supplementing Manual auditing. |
I don't believe I did misunderstand you. Your reply above is a total non-sequitur as a response to my point. Right from the beginning, this thread has been specifically concerned with the usefulness or otherwise of the league tables - you have never claimed, of course, that automated testing is not useful in and of itself. You have been claiming that the league tables are "bad" because they do not fully measure accessibility, and therefore presumably you are saying they should not exist. My response is that your alternative is in fact worse than the current situation, and partly measuring accessibility is better than not measuring it at all.
| Grant Broome wrote: |
| However, the results of automated testing alone are very limited to verifying only a few AA checkpoints, while all checkpoints can be fully tested manually. |
It is totally untrue to say that "the results of automated testing alone are very limited to verifying only a few AA checkpoints" - even you would agree, I'm sure, that at the absolute minimum they are also useful for identifying areas where checkpoints may have been violated; for example by locating multimedia content so that it can be manually inspected.
| Grant Broome wrote: | ||
This is typical of the type of statement that we are used to seeing in SiteMorse press releases. I think I have been able to demonstrate over the course of this discussion that what you do with automated testing is far more limited than manual auditing. |
No, you're missing the point - yes, in some ways automated testing is of course limited with respect to manual testing. However, in other ways, manual testing is more limited than automated testing. For example, testing, as we do, thousands of pages every day is simply impossible with manual testing.
| Grant Broome wrote: |
| The SiteMorse reports that I have seen state that time saved manual testing on the 2 sites that we have highlighted in this discussion would have been 2 hours in one instance and 29 minutes in the other. It then highlights all the other checkpoints that need manual testing in addition to this time. What metric is used to measure this imaginary time-saving isn't even relevant. I am sure that someone with even the most basic idea of web-accessibility would easily be able to spot at least 1 of the errors missed by the SiteMorse tool within a minute. |
Again you're missing the point. The estimate of time saved is just that - an estimate of the time saved. The time taken to fully test a particular set of pages manually is much greater than the time taken to fully test a particular set of pages when assisted by SiteMorse. SiteMorse of course does not do the job for you completely automatically, since that would be impossible. It saves you time by helping you do the job.
| Grant Broome wrote: |
| While you mention time-saving and cost... For less than the cost of an annual subscription to SiteMorse, a web developer could invest in any one of the commercially available accessibility testing tools |
You appear to have completely changed the subject and now be talking about pricing. That would appear to be completely off-topic compared to this thread so far, but I think you are mistaken if you think that SiteMorse's pricing is anything other than extremely competitive.
| Grant Broome wrote: |
| many of which are demonstrably more comprehensive than SiteMorse and sit on the developer's desktop or server. They can be used to test any number of pages in any part of the site at any time that is convenient for the developer. |
Sitting on the developer's desktop or server is in fact a disadvantage. Testing being external is important - otherwise, problems will be missed. SiteMorse can also test "any number of pages in any part of the site at any time that is convenient" - it is under the customer's control.
| Grant Broome wrote: |
| Compliance to any of these tools should in theory result in a higher SiteMorse League Table ranking as they all do the same job to a greater or lesser degree |
Certainly. It would indicate a problem with the league tables (or the other tools you are talking about) if this was not true.
| Grant Broome wrote: |
| Some final statements are invited from Jon and then over to the forum. |
To partly summarise points that have already been made in this thread:
- The SiteMorse league tables accurately measure what they are advertised as measuring - the results of automated testing (which you have agreed is good as a "rough measure of accessibility").
- Even if a person does not like the SiteMorse league tables, that is no criticism of the SiteMorse product itself. As you have agreed, automated testing is "an incredibly useful way of supplementing manual auditing".
- I believe the SiteMorse league tables have been a benefit in, among other things, raising the awareness of accessibility issues, and are therefore a good thing.
If other people wish to ask further questions, I'll still be watching this thread. However, I'm not going to respond if it just degenerates into a pointless slanging match again.
Share:
I'd just like to say thanks to both Grant and Jon for both doing a good job so far at not only finding points that they agree on but - on the whole - doing a pretty good job of remaining civil and polite on what can be an emotive topic.
I'm someone who does a fair amount of accessibility testing - both manual and automatic - but then I believe that most of us regulars on this forum would probably fall into this category.
Jon,
I've mentioned this before, but I just wanted to see what other people (including yourself) think. Obviously it would entail additional work, but could SM incorporate some check into their league tables whereby they maybe check five pages manually from a handful of the sites tested per block - maybe the top ten? - and somehow incorporate that into the grading.
This would demonstrate the benefits of manual testing in catching some other errors, and encourage webmasters not to rely solely on an automated tool. It would also mean that once you've reached a certain standard you can't expect to maintain your SM league table position simply by ticking the boxes that pass automated tests and would hopefully encourage people to go further with accessibility.
...obviously I appreciate the scale would prevent this from being done on every site you check, but I think being done on some sample basis would have considerable benefits.
_________________
Jack Pickard The Pickards Information Services| Blog | Twit
Share:
Hi Jon,
As one of the more clear cut issues involved in the Sitemorse test is the performance testing, would you be able to explain a few of the processes involved in gaining the results:
- Are the performance tests ran from multiple locations? (And on what speed connections?)
- How many concurrent requests are made during the course of the testing?
- When refering to "56k modem", "512k ADSL" and "corporate network" timings - are the tests ran on physical lines the equivalent of these, or are the results infered?
The reason I ask these questions is that I have seen various reports for sites I am involved in which report homepage timings of in excess of 15 seconds, and average speeds of less than 5kB/s. These sites are hosted on a 4Mb pipe and high end servers, and we would receive many complaints from clients and users if the performance was anywhere near the reported figures in real usage. (This is supported by our own independent tests which we have ran from multiple locations).
It would be useful to be able to explain to clients why the figures Sitemorse provide seem to differ from those observed.
Many thanks
Share:
| Jon R wrote: |
| The SiteMorse league tables accurately measure what they are advertised as measuring - the results of automated testing... |
Hi Jon
For this reason (i.e. that the league tables don't measure accessibility per se) amongst others we've decided to opt-out of the league tables. This involves us either blocking the SiteMorse spider by sniffing the user-agent string, or blocking the ip address from which the spider operates. Obviously neither of these solutions are 100% reliable, since either could change arbitrarily.
Why doesn't the SiteMorse spider honour robots.txt, unlike the W3C validator and most other automated tools? It would make it a whole lot easier for those who don't wish to be "measured" by the SiteMorse tool. Alternatively will SiteMorse honour requests from site owners for their sites to be added to a list which the tool won't test (which I note is the case with SiteMorse's site itself)?
Dan
_________________
Dan Champion, Champion IS, Mooch Marketing, Revish
Share:
Hi Jon,
I think things got unnessasarily heated back there, and certain issues were circled somewhat.
I think we can all agree that tools fit into the process. Automated tests are useful to help find problems, but not for evaluating actual accessibility. (My position on the matter).
I've two questions:
1. This may not be for you to answer, but unfortunately Sitemorse's marketing seems to undermine your previous statements:
| Jon R wrote: |
| We go specifically out of our way to explain that manual testing is necessary. A Sitemorse report never says "pass" with respect to accessibility or individual checkpoints - the two options are "check" or "fail". |
From the latest Sitemorse press release:
| Sitemorse wrote: |
| Priority 1 (A) Accessibility (sites are also tested for AA compliance): 48% (last month 40%) sites passed all Accessibility A tests. |
Just a minor example, but it certainly seems that Local Authorities (used to?) perceive Sitemorse's league tables as equating to actual accessibility. Sitemorse's marketing doesn't really help in that regard, more due to approach than specific words.
Oh, I almost forgot the question: Is this likely to continue?
2. More importantly, there is no transparency of the criteria used.
Sitemorse is a tool, it can help to discover accessibility issues and other useful website problems. When talking about passing or failing, it is against an interpretation of the WCAG guidelines. That's fine, any automated tool would have to do that.
The heading issue earlier brought this up immediately - it would have been a useful discussion (on another thread) to establish when people should be warned/informed that their page did not have a suitable structural element.
However, for Sitemorse's results to have meaning (with regard to accessibility), the criteria used need to be freely available and transparent.
I recommend, urge and challenge Sitemorse to publish their testing criteria.
It would actually help Sitemorse. There is already a public initiative in Europe to provide a common set of criteria for evaluating web sites (hat tip to Dennis Kessler for that). There are also open source checkers available (e.g. Hera).
Sitemorse's value is not from being a black-box test of accessibility, it's being a service that helps site owners monitor and understand the performance of their site.
Therefore I would suggest you put this type of argument to rest, and publish the criteria (as I asked Laurence Shaw to about 2 years ago). This would enable people to know what pass/fail actually means, kind of like this Cynthia Says / Bobby comparison.
Kind regards,
-Alastair
Share:
| danchamp wrote: |
| Why doesn't the SiteMorse spider honour robots.txt, unlike the W3C validator and most other automated tools? |
Some search engines (particularly Google) go out of their way to give site owners control over how their site is spidered. For example, <meta> tags are recognised for site owners to have per-page control. Some even recognise rel="nofollow" to allow site owners per-link control.
The site owner has to pay for the resources being used by spiders. They should be able to opt out by at least one costless method. The robots.txt format is one such method and has widespread industry support.
_________________
My CV type thing and my Life of Ben (Blog). Nigel Peck's Accessify Forum Requirements.
Share:
| JackP wrote: |
| I've mentioned this before, but I just wanted to see what other people (including yourself) think. Obviously it would entail additional work, but could SM incorporate some check into their league tables whereby they maybe check five pages manually from a handful of the sites tested per block - maybe the top ten? - and somehow incorporate that into the grading. |
The problem with your idea is that it is simply impossible to test manually even a few pages of every site we test automatically - we'd be testing literally thousands of pages a month by hand.
If we tested only some sites from each league table site set, then we couldn't incorporate this into the score because then the sites wouldn't be on a level playing field - the handful we picked for manual inspection would have either a disadvantage or an advantage compared to the rest.
Share:
| Ross wrote: |
| As one of the more clear cut issues involved in the Sitemorse test is the performance testing, would you be able to explain a few of the processes involved in gaining the results: |
OK. Again, I'm not sure if this is on-topic for accessifyforum or not, but anyway...
| Ross wrote: |
| Are the performance tests ran from multiple locations? (And on what speed connections?) |
They are run from a connection close to the LINX on very high speed connections - the servers are located in a large data-centre so have excellent network provision.
| Ross wrote: |
| How many concurrent requests are made during the course of the testing? |
A maximum of 5 - similar to a couple of normal users with standard browsers.
| Ross wrote: |
| When refering to "56k modem", "512k ADSL" and "corporate network" timings - are the tests ran on physical lines the equivalent of these, or are the results infered? |
These are calculated to indicate the maximum possible speed that could be achieved using each technology - e.g. if your site's front page consists of 128kB of data, and your ADSL can achieve a maximum of 64kB/sec, then the minimum possible time to download it is 2 seconds (assuming the server can serve at 64kB/sec or more).
| Ross wrote: |
| The reason I ask these questions is that I have seen various reports for sites I am involved in |
I don't want to get bogged down with discussions of individual sites here. However I will say that if SiteMorse is reporting particular performance figures, then they indicate the genuine performance of that server at the time the test was carried out.
Share:
| danchamp wrote: |
| For this reason (i.e. that the league tables don't measure accessibility per se) amongst others we've decided to opt-out of the league tables. This involves us either blocking the SiteMorse spider by sniffing the user-agent string, or blocking the ip address from which the spider operates. Obviously neither of these solutions are 100% reliable, since either could change arbitrarily. |
The User-Agent header will not change arbitrarily - it will always contain the string "b2w ". It's what we recommend people to filter on to remove SiteMorse requests from their access statistics.
| danchamp wrote: |
| Why doesn't the SiteMorse spider honour robots.txt, unlike the W3C validator and most other automated tools? |
There are several reasons. To begin with, bear in mind that, unlike most search engines etc, SiteMorse does not just randomly scan every site it can find. It only scans sites it is told to. This means sites that people have requested to be inspected, or league table sites - which are pretty much restricted to public authorities and large corporates. If a site owner requests their site to be tested, that surely overrides any potential instruction from robots.txt.
In addition, most people when writing their robots.txt consider only search engines. They perhaps don't even realise that they will be affecting other tools too. Therefore the contents of their robots.txt, in general, has very little application to non-search-engine spiders.
Share:
| Alastc wrote: | ||||
1. This may not be for you to answer, but unfortunately Sitemorse's marketing seems to undermine your previous statements:
From the latest Sitemorse press release:
|
OK, this may seem pedantic, but I said a report will never say 'pass' with respect to accessibility checkpoints, whereas your quote is from some text describing sites passing automated tests. As I said, we do try very hard to make it clear what we are and are not testing. However, since I am not involved in marketing I can't get too involved in discussion of marketing materials, although I can pass on any feedback.
| Alastc wrote: |
| Oh, I almost forgot the question: Is this likely to continue? |
Sorry, I must be missing something - is what likely to continue?
| Alastc wrote: |
| 2. More importantly, there is no transparency of the criteria used. |
I strongly disagree with that. The criteria used are publicly available from the W3C and IETF - that's the point of the tool. If you make your site well-performing and fully compliant to HTML and WCAG standards, it will get full marks. (The specific benchmarks for performance are shown at the URL I gave earlier in this thread.) Explanations of all the different diagnostics SiteMorse can produce can be seen from within a report by clicking the 'additional information' links.
| Alastc wrote: |
| The heading issue earlier brought this up immediately - it would have been a useful discussion (on another thread) to establish when people should be warned/informed that their page did not have a suitable structural element. |
Yes, I certainly agree. It's a pity that Grant didn't seem to want to discuss that bit. I would definitely welcome discussion on that point, although I think it would probably be best if a new thread was started in which to hold it.
Share:
Just a quick aside: apparently there is some confusion over the 'check' accessibility diagnostics. These do not affect a site's score in any way - they are provided simply for the benefit of the site's developer. Only 'fail' diagnostics count against the 'Accessibility SiteMorse Mark'.
Share:
| Jon R wrote: |
| Only 'fail' diagnostics count against the 'Accessibility SiteMorse Mark'. |
Thank you Jon, that is very useful to know. With that fact, and that all checkpoints that can generate an automatic fail are weighted equally, interpreting the tool's output becomes a lot easier. I'll bow out again now, just wanted you to know I'll give credit where I think its due
_________________
Accessibility != Bobby
Share:
| Quote: |
| The criteria used are publicly available from the W3C and IETF |
An automated tool can't do that, you've agreed with that before so I'll try an example or two to clarify.
Wasn't there a discussion before about whether suitable alt text for an image could include ".jpg"? (Something to do with GAWDs if I remember correctly.) Accessibility tools often check for common issues such as authoring tools including the file name as a alt text. That isn't mentioned in the WCAG 1, although in most cases would catch problems.
Another example would be checking link targets. A tool should check that for each link text used, that text only refers to one URL. However, this can run into issues such as Windows servers being case-insensitive, and *nix servers are not.
Bobby annoyingly reports a alt-text error when you have an image with null alt text and text within a link, even if the image should have a null alt.
For each checkpoint there are things that tools can and should check for that try and find as many cases where there might be a problem in the page.
That European initiative (Unified Web Evaluation Methodology) I mentioned is part way through producing a huge document that shows what tests can be carried out programmatically and/or manually, and what confidence level they have for each test.
Take UWEM's tests for Checkpoint 1.1 as an example.
The Sitemorse tool must have criteria which interpret and perhaps extend the guidelines. That is a good thing, after all it is 7 years since WCAG 1 was published.
However, for pages to pass or fail according to a tool, the criteria must be as public as the results.
| Quote: |
| Explanations of all the different diagnostics SiteMorse can produce can be seen from within a report by clicking the 'additional information' links. |
But not from the league tables, from what I've seen. Do you have to buy a report to see them?
Share:
| Jon R wrote: | ||
These are calculated to indicate the maximum possible speed that could be achieved using each technology - e.g. if your site's front page consists of 128kB of data, and your ADSL can achieve a maximum of 64kB/sec, then the minimum possible time to download it is 2 seconds (assuming the server can serve at 64kB/sec or more). |
This is very interesting but very difficult to believe as I have seen you quote speeds for a site in 1 league table and in the next month ( using the same home page ) you have given totally and massivly different speeds.
The page in question was CMS based XHTML and the wording on a couple of news snippets had changed but thats all so effectivly the page was the same weight.
Anyway I digress - my point is why don't you make it clearer, in your reports, that the results are calculated and not real - people believe ( d ) these.
_________________
If it can go wrong it will. So don't worry about it.
Share:
All times are GMT
You cannot post new topics in this forumYou cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum