Legal advice from automated testing tools?
A little rant, just posted over on Accessify: Legal advice from automated testing tools?
| Quote: | ||
|
As has already been noted on previous occasions, automated accessibility testing tools can be useful ... but only if their results are not merely taken at face value, but backed up by human testing and plain common sense. All too often these tools simply follow accessibility guidelines by the letter, adding their own arbitrary (and often secret) heuristics to test what can't be tested programmatically, and give a report containing false positives or false negatives (see for instance Isofarro's excellent article on SiteMorse). In this light, I find the accessibility check carried out as part of SilkTide's sitescore quite amusing. Testing one of my sites, the tool came across a single invalidly encoded character (an em dash from a copy/paste straight out of a Word document). Yes, this makes the page's markup invalid, which in turn makes it fail WCAG 1.0 checkpoint 3.2 "Create documents that validate to published formal grammars". Now, an automated tool would indeed be correct to flag this up as an issue (and fail the page for level AA and AAA), but SilkTide's choice of words leaves a lot to be desired (and, to the cynic in me, sounds a lot like fear mongering intended to sell their consultancy services). Under the ominous "British legal requirements" heading, the sitescore report states that the site:
This is wrong on at least three levels:
But, overall, it's nice to know that an automated tool can now also give us such valuable legal advice...who needs lawyers anymore? |
Patrick H. Lauke / splintered
Share: 
| Quote: |
This is wrong on at least three levels:
|
I'm not quite sure how you can make a categorical statement that something is wrong and then qualify it with a statement that entertains the idea that it could be right.
I thought that part of the value of validity was that it gives our sites the very best chance of being compatible with today's and tomorrow's UAs, which may or may not be so strict as to bork on an invalid character or more generally, on a page made invalid by that invalid character.
i.e. So long as we simply don't know, the point of validity is partially to minimise the chance of 'surprises'.
(Validity 'only' being a level 2 criteria notwithstanding) Are we now to consider for ourselves which markup invalidations we think might be important and choose to address or ignore them on that basis? Can our judgement really be that infallible?
(Given that we're now post October 1999 and post October 2004, I actually think the point about perpetuating false beliefs about certain DDA dates is academic. Whichever the date, it's now in effect [something which the SilkTide report makes fairly clear] and that's really the only dinstinction that matters, imho.)
Fwiw, I do agree with the skeptical gist of your article. As you suggest, it does sound like a (typical) attempt to drum up business by drumming up a slightly hysterical take on your situation.
It smacks of those who turn up on your doorstep and tell you that they've noticed some tiles missing from your roof - and then attempt to sell you on their generally over-priced and frequently underskilled roofing repair services.
Anyhoo, the fact that SilkTide have now integrated faux legal commentary into accessibility reports is indeed a potentially unfortunate precedent.
Share:
| Bill Posters wrote: |
|
I'm not quite sure how you can make a categorical statement that something is wrong and then qualify it with a statement that entertains the idea that it could be right. |
i'm not saying it's right, and i do say earlier on in the piece that the validator is 100% right to flag this up as an issue. what i am saying, though, is that in this case it's an error that yes, should really be fixed, but its presence certainly is not a showstopper that automatically makes the site completely inaccessible and therefore unlawful.
| Quote: |
| Are we now to consider for ourselves which markup invalidations we think might be important and choose to address or ignore them on that basis? Can our judgement really be that infallible? |
all validation errors have to be addressed, but again my comment was in light of the faux legal advice.
| Quote: |
| (Given that we're now post October 1999 and post October 2004, I actually think the point about perpetuating false beliefs about certain DDA dates is academic. Whichever the date, it's now in effect [something which the SilkTide report makes fairly clear] and that's really the only dinstinction that matters, imho.) |
true, of course. but it goes to show the "depth" of their knowledge in the area.
Patrick H. Lauke / splintered
Share:
| redux wrote: |
| i'm not saying it's right, and i do say earlier on in the piece that the validator is 100% right to flag this up as an issue. what i am saying, though, is that in this case it's an error that yes, should really be fixed, but its presence certainly is not a showstopper that automatically makes the site completely inaccessible |
…to the best of your current knowledge.
If you know for certain which types of invalidations might possibly be a problem in future and which ones never will, please let me in on the secret.
| Quote: | ||
true, of course. but it goes to show the "depth" of their knowledge in the area. |
For sure. We were both just being pernickety.
Share:
Actually, it was just you, Bill.
Redux is quite correct in his article. His site is being delivered as text/html, in which unregocnised characters will be ignored or displayed using the generic "unsupported character" glyph (usually a square outline or an inversly coloured question mark). It is not ideal, obviously, but it is a largely inconsequential error. It only takes a basic understanding of Web technologies to realise that some errors are more severe than other.
Although no error is desireable, SilkTide have completely misrepresented the severity of the problem.
I ran Project Cerbera through it and generally the results made sense, although they are clearly biassed towards selling the company's services.
(EDIT) SilkTide results for SilkTide. I wonder how their site scored at 9.6 when they don't use "the latest standards (XHTML)" and use tables for layouts, along with various other things things which their tool warns about.
| SilkTide Results wrote: |
| Design makes proper use of modern technology (no table-based layout) (more detail) |
Their homepage uses layout tables, too. The form for the website test URL is in one and recent news is in one. There is presentational markup (like <b> tags) throughout.
I can't believe how brazenly false the results for their own site are. It's *almost* like they've skewed their scoring algorithm to favour their own site, and overlook its faults.
Last edited by Ben Millard on 05 Dec 2005 05:51 pm; edited 3 times in total
Share:
| Cerbera wrote: |
| Actually, it was just you, Bill.
Redux is quite correct in his article. His site is being delivered as text/html |
Presumably you know which site it was, so you have me at a disadvantage.
Fwiw, in the event it's one of those in his his sig…
Splintered is XHTML 1.1 sent as application/xhtml+xml (though it appears that it's also being sent as text/html to less capable browsers*).
Photographia is XHTML 1.0 sent as text/html.
* Redux, is there something clever going on with that?
The method I use to assertain whether a browser can handle application/xhtml+xml shows IE5/Mac as incapable and yet the markup for splintered still uses the XHTML 1.1 doctype.
(The method I use has seemingly been reliable in all other occasions.)
- - -
As someone who's not overly familiar with automated accessibility tests, is there some worthwhile info to be gained from the SilkTide test? Or is it another candidate for the type of lambasting that SiteMorse allegedly warrants?
Share:
Putting my 2p worth in... I quite like the silktide system.
It's a bit basic and the terms it comes back with well are a bit silly/scarmongering which smacks of the marketing team giving it a once over, but as a quick and free grounding to the uninitiated it's okay.
It comes back to the same truth, automated tools are only part of the armoury, and shouldn't be relied upon in isolation.
Share:
| Richard Conyard wrote: |
| It's a bit basic and the terms it comes back with well are a bit silly/scarmongering which smacks of the marketing team giving it a once over, but as a quick and free grounding to the uninitiated it's okay. |
Yeah, I agree. Ran my site over in it and it highlighted validation errors (oops, will fix later) but said I was leaving myself open to being sued under the DDA. Which is a bit harsh for a personal site, particularly when there's no cash involved. Still, I accept I need to fix them, but telling me I'm going to get my ass sued off (or whatever the exact phrasing was) is a little OTT.
Oh, and it said the main concept on my site was "feeling stalky". Which was amusing if a little frightening. But I like the look of it on the whole. It's a nice little tool - albeit like any other tool it has its limitations.
Jack Pickard The Pickards Information Services| Blog | Twit
Share:
| JackP wrote: |
| Yeah, I agree. Ran my site over in it and it highlighted validation errors (oops, will fix later) but said I was leaving myself open to being sued under the DDA. Which is a bit harsh for a personal site, particularly when there's no cash involved. Still, I accept I need to fix them, but telling me I'm going to get my ass sued off (or whatever the exact phrasing was) is a little OTT. |
Especially since it could be easily argued that since your site is mostly a personal site, that it does not fall under the scope of the DDA in the first place, and so even if it's the most inaccessible site in the world you couldn't be sued for it...
But I'm just being picky there.
... and still feeling amused because their report told me off for swearing
Although I do wonder what I lost 0.3 points on in terms of design.
Share:
heh...i lose major points on splintered because i'm not using enough images and, because there's no forms, i'm not making use of any interactivity throughout and my site is therefore dull and boring or something. hey, they've got artificial intelligence running there...Patrick H. Lauke / splintered
Share:
| redux wrote: |
| my site is therefore dull and boring or something |
it's not your site that's dull and boring
only joking [as you don't want to offend a moderator, now do you?]
Jack Pickard The Pickards Information Services| Blog | Twit
Share:
| Quote: |
| probably unlawful |
That's quite an assumption.
The tool can't tell if you're a service provider. And it can't tell whether you've made reasonable adjustments to make your services available to disabled people if you are a service provider.
Plus everyone gets the same invalidly encoded em dash, so it can't possibly be discrimination.
Anyway, it's for a judge to decide whether or not discrimination has occurred, not a Web design company who can't even get the name of the DDA correct.
Share:
| ectoplasm wrote: |
| The tool can't tell if you're a service provider. |
Not wanting to seem the apologist, but it's probably assuming that anyone who feels the need to run their site through their test is likely to be a service provider, rather than an 11 year old girl with a homepage about her hamster 'Mr Sniffy' (or someone with an equally out-of-bounds site).
It's probably a safe bet, but still it's an assumption they should've avoided putting into writing.
I tried to test my main site and it came back saying they were unable to find a website at the address I requested.
(The url was perfectly good, of course) Ho-hum.
Share:
[quote="Bill Posters"]
| ectoplasm wrote: |
| a homepage about her hamster 'Mr Sniffy' |
Wow! What's the URL? I can't believe someone else has a hamster called "Mr Sniffy"!
www.brucelawson.co.uk
Web Evanglist, Opera, WaSP Accesibility Task Force
Study the Web Standards Curriculum
International Lothario (retired)
Share:
| ectoplasm wrote: |
| The tool can't tell if you're a service provider. |
And yet it could be argued that providing information IS providing a service. There is information about my family tree on my site. If that information was not available to users with disabilities, I would say that it's right to argue I am discriminating against them.
Where I believe I would be safer under the DDA is that it is not reasonable to expect a single person maintaining a home or hobby site (that does not have any financial transactions as part of it) to be expected to achieve the same standards mandated for a large company.
I would like to argue that in my case, it would be reasonable for ME to do this, as I have the knowledge, but I still think a court would find in my favour if I was sued because a) I specifically say "if you have problems accessing my site, contact me and I'll do my best to fix your problem" and it would certainly be reasonable for that line of approach to be tried first and b) they wouldn't want to set a precedent where hobbyists/personal sites could be sued under the DDA.
| Bill Posters wrote: |
| Not wanting to seem the apologist, but it's probably assuming that anyone who feels the need to run their site through their test is likely to be a service provider [...] It's probably a safe bet, but still it's an assumption they should've avoided putting into writing. |
I agree on both counts! And I'd just like to reiterate that I think it's a nice tool... I'd like to see it improved, but I would expect I'd be using it again.
Jack Pickard The Pickards Information Services| Blog | Twit
Share: