SiteMorse Q & A
Home / Site Building & Testing / SiteMorse Q & A
| Cerbera wrote: |
| Somebody accessing a UK site from Australia will get different performance from someone in Japan, both will be different from someone in America, all will be different from someone in the UK. |
Certainly true. However our service is measuring UK performance for UK sites. If people wanted a "how fast is our site from Australia" service they'd presumably go find one.
| Cerbera wrote: |
| A data centre in one part of the UK will get different performance compared with a data centre of equal power in a different part of the UK, too. |
Not significantly it won't.
| Cerbera wrote: |
It's a well known fact that performance differs depending on location, especially amongst online gamers.  |
On-line games are a totally different type of network application to web sites. There is no comparison.
| Cerbera wrote: |
| If the SiteMorse speed tests are all from one location and are only conducted one time per page, they are useless for gathering any meaningful measurement of speed. |
This is totally untrue.
| Cerbera wrote: |
| This is made worse by extrapolating the performance of domestic connections from the high-powered commercial connection of a data centre. |
Why?
| Cerbera wrote: |
| If the speed testing method was published, site owners would be aware that they results are inaccurate. |
Since they are not inaccurate, that is false.
Share: 
Do you know how the Internet works, Jon? From your responses, it seems that you don't._________________
My CV type thing and my Life of Ben (Blog). Nigel Peck's Accessify Forum Requirements.
Share:
| Jon R wrote: | ||
|
| Cerbera wrote: |
|
Not true at all. ..............This is made worse by extrapolating the performance of domestic connections from the high-powered commercial connection of a data centre. |
Its this last point from Cerbera I was trying to make Jon, when I said they are calculated and not real.
Or have I misunderstood and you actually test with all the different types of connection?
_________________
If it can go wrong it will. So don't worry about it.
Share:
| jim barter wrote: | ||
I would really like to see this point addressed. |
Sorry, I snipped that bit originally because I couldn't see any way to answer it without being rude. To put it charitably, Grant's reply made little sense in response to my statement.
| jim barter wrote: |
| Jon, How can the concept of accessibility not apply to a page designed to indicate why automated accessibility tools are not the be all and end all of accessibility testing? Sorry - but this is a fundamental concept, failure to grasp this is indicative of failure to grasp accessibility. |
Exactly my point - I think anybody who thinks that the concept of accessibility applies to that page is indeed failing to grasp accessibility. If a page is not intended to convey any information, or provide any service, then how can anyone be impeded in accessing these non-existent services and information?
Share:
| Cerbera wrote: |
| Do you know how the Internet works, Jon? From your responses, it seems that you don't. |
I'd say the same to you, but then I'd be lowering myself to your level.
You're now trying to start that "slanging match" we all wanted to avoid, so I'm going to leave it there for the moment I think.
Share:
| Jon R wrote: |
|
Exactly my point - I think anybody who thinks that the concept of accessibility applies to that page is indeed failing to grasp accessibility. If a page is not intended to convey any information, or provide any service, then how can anyone be impeded in accessing these non-existent services and information? |
I guess this is what's puzzling me - how does one measure, in any objective manner, a notion as vague as whether or not a page conveys information. Given that, how does one measure whether or not said information can be accessed, without human intervention of some sort?
I think I'm missing something about automated testing here.
_________________
Jim O'Donnell
work: Royal Observatory Greenwich
play: eatyourgreens
Share:
| Jon R wrote: | ||||
Sorry, I snipped that bit originally because I couldn't see any way to answer it without being rude. To put it charitably, Grant's reply made little sense in response to my statement. |
Still not addresed, Don't make me "do a Paxman".
| Jon R wrote: | ||
Exactly my point - I think anybody who thinks that the concept of accessibility applies to that page is indeed failing to grasp accessibility. If a page is not intended to convey any information, or provide any service, then how can anyone be impeded in accessing these non-existent services and information? |
No, the exact opposite of your point. please read it again.
_________________
http://www.opsi.gov.uk/acts/acts1995/Ukpga_19950050_en_8.htm#mdiv57
Last edited by jim barter on 18 May 2006 12:53 pm; edited 1 time in total
Share:
| Jon R wrote: | ||
Not significantly it won't. |
That's not true in my experiance Jon. A site in Manchester on the end of an 66Mbps linked through Stoke would have a much slower response than a server connected directly to LINX core or the BT Central/SuperJANET network. If that wasn't so I wouldn't be paying as much as I do for hosting
_________________
Accessibility != Bobby
Share:
| Jon R wrote: |
| you can get free example reports from the web site. |
I could do with a link then, I can't find one. If you are referring to the ones where you enter your email address, I get those every month, and there is no link through. (And please ask whoever creates those to add an unsubscribe link to the emails.)
Could you add that information to the area which has the benchmarks?
Speed issue: there seems to be confusion as to where it's a page weight test, or a measured speed test on particular pipe sizes. To be honest, that doesn't interest me much (availability rather than accessibility), but it would be nice if that were clear.
Robots.txt: The W3C HTML validator does not spider sites, people have to point it to a page at a time. The W3C link checker does spider a site, and does follow the robots.txt .
| Jon R wrote: |
| Everyone else has managed to mind their manners and keep things polite in this thread, Grant - why can't you? |
I have to agree, the attitude coming across hasn't been helpful, intentional or not.
| Jon R wrote: |
| I've seen that page before. To be honest, I don't understand what the point of it is. It certainly doesn't add much to the basic WCAG document - it has all this nice algorithmic-sounding stuff like "select elements with x attribute" but then simply goes and ignores the actual issue with glib statements like "Decide whether the text alternative appropriately represents the non-text-element's function". |
It isn't saying that each will be automatically tested, but it describes (in a way that can be applied to manual or automatic tests) what will pass or fail.
For example, this could be automatic:
| Quote: |
| Select non-text-elements without an alt attribute. The non-availability of any text alternative leads to an error. |
This could not:
| Quote: |
| Select non-text-elements without an empty text alternative. The presence of an empty alt attribute indicates the possibility of a pure decorative image. If that is not the case, an error must be flagged. |
If Sitemorse would show what it does check in a similar manner to this, then it would be transparent. That is what I'm suggesting, and I think it would help Sitemorse's position.
Share:
At Woking Borough Council, we recently bought some credits for the SiteMorse tool, mostly so that we could try to figure out exactly what our scores in the monthly league tables actually meant. Like Alastc, we were unable to find anything on the SiteMorse site that explained, in depth, what precisely was being tested for each of the categories for which we would receive a score.
I've written up some observations on what we found, for anyone who's interested, but the long-and-the-short of it is this: the actual test results were quite useful, but still required a certain amount of judgement to interpret -- waters which can only be muddied further by the inevitable lack of detail in the league table charts themselves.
Cheers,
Andrew.
PS: Sorry for the too-early post -- I pressed ctrl-s rather than shift-s by mistake. Blasted accesskeys!
_________________
Andrew Green
Article Seven / Buy Our Honeymoon / Elements or Lower
Share:
| Jon R wrote: | ||
I'd say the same to you, but then I'd be lowering myself to your level. You're now trying to start that "slanging match" we all wanted to avoid, so I'm going to leave it there for the moment I think. |
My question was (and still is) genuine. You are the only SiteMorse developer who has (very kindly) taken the time to speak to us here. The site speed tests (and other issues) might not be within your area of expertise.
Your responses show you are unaware of some realities of Internet communication. I therefore asked if you are aware how data is sent around the Internet at all. This question was to help me ascertain what level of knowledge you have on this specific issue.
Several members have expressed concerns about the accuracy of SiteMorse's speed testing methods. I am trying to find your level of knowledge on this issue so we may discuss it in a mutually informed way.
Now, please, do you know how the Internet works?
_________________
My CV type thing and my Life of Ben (Blog). Nigel Peck's Accessify Forum Requirements.
Last edited by Ben Millard on 18 May 2006 03:35 pm; edited 2 times in total
Share:
| Jon R wrote: | ||
I believe I have shown considerable good faith in spending a not-insignificant amount of time coming here to discuss issues with the members of this forum, and yourself in particular. It is extremely rude of you to then claim, with no basis whatsoever, that despite clear evidence to the contrary, I'm just ignoring you. Everyone else has managed to mind their manners and keep things polite in this thread, Grant - why can't you? |
What I have experienced time and again is SiteMorse representatives consistently ignoring comments from the community. Frequently leaving forums with many unanswered questions as we are seeing again here. Jon, my statement is based on a history of dialogue with SiteMorse. I am not aware of any instance where SiteMorse has made changes to their tool based on advice from this forum.
On a personal level, you have accused me of being vague, your answers to my questions have been evasive, you've failed to acknowledge my responses to your questions and then call me rude for suggesting that I claim that you are ignoring me - not that I ever made such a claim.
The discussion was an attempt to glean some facts about the SiteMorse product. Right at the beginning, before you even answered my first question I made the statement:
| Quote: |
| This set of questions is designed to interrogate the validity of SiteMorse test results and the accuracy of the league tables. |
I don't think you have the right to get upset if I make challenging statements. You are a representative of SiteMorse and agreed to enter into a discussion about the validity of the tool. I will not apologise for my statement because only time will tell if I am right or not.
I do think that it's very impolite to make claims about how you have invested your time in this discussion without acknowledging those who have invested theirs in an effort to understand and offer feedback that could be used to improve your product. I also think that it's incredibly poor business practice to ignore peer review and customer feedback, but that is your choice. Best of luck to you.
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Share:
I think that's about it.
I would like to thank Jon again for his time and for entering this discussion and all those that participated in this Q and A. I hope that those who have read this thread have gained a greater understanding of the automated testing and the SiteMorse tool.
I must state that I do believe that automated testing tools have their place in web accessibilty testing as a useful way of assisting manual testing - best performed by disabled users on their own assistive technologies. This discussion was about the validity of the SiteMorse testing tool and the value of SiteMorse league tables which are based solely on the results of automated testing and do not incorporate any form of manual verification.
The discussion has been a little heated at times due to the strong differences in opinion about the SiteMorse tool and more specifically the affect of league tables on accessibility but I hope that the responses and comments will be found useful.
During the discussion, there were some clear statements about the way that the SiteMorse tool handles automated testing.
I just want to summarise some of the key topics of discussion:
Testing
- SiteMorse agreed that none of the priority 1 checkpoints could be fully checked using automated testing
- SiteMorse acknowledge that accessibility can only be comprehensively measured via manual testing
- SiteMorse state that an automated tool can be useful for pinpointing areas that require manual testing
- A web page that violates 11 out of 17 Priority 1 checkpoints scored 10/10 when tested with the SiteMorse tool (SiteMorse had previously declined to test the site and publish the results)
- SiteMorse do not agree that a web page designed to test the limitations of automated tools can be used to test the SiteMorse tool
- There were several requests for SiteMorse to publish their testing criteria publicly. SiteMorse claim that they have done this in their reports which are available to existing customers only. An alternative location for the criteria was not given
- SiteMorse state that they only claim that a site has “failed” certain checkpoints and never use the word “pass” to describe test results
- SiteMorse do not acknowledge that the opposite of “fail” is “pass” in the context of their report
- In contrast to this claim, the latest SiteMorse press release states a percentage of websites that “passed all accessibility A tests”
League Tables
- SiteMorse accept that League tables are only a rough guide to how accessible websites are as only a small proportion of checkpoints could be fully verified using automated testing.
- SiteMorse claim that League tables can “often” tell whether one site is more accessible than another.
SiteMorse errors
- SiteMorse acknowledged that their tool does not test for the presence of headings in websites even though it fails sites that have heading ordering issues.
- If a website has issues with heading order, It is currently possible to gain a higher SiteMorse ranking by removing all of the headings from the website.
- SiteMorse do not acknowledge that their tool sometimes erroneously reports errors (false positives) - should be noted that this information is from another thread but relevant to this point although I would be happy to revise this statement if SiteMorse were to make this concession.
Server response time (not part of the original discussion but relevant to League Tables)
- SiteMorse would not acknowledge that Server response times differ depending on location
- SiteMorse does not conduct actual modem testing on websites but extrapolates the speed from the broadband connection
- Page download time is only measured on the homepage of the website and does not account for re-use of assets on subsequent pages
During the discussion, some information for web managers on how to exclude their sites from League Tables was given. This may be useful as the SiteMorse tool ignores the robots.txt file that would be normally used in such instances:
SiteMorse user agent always uses the string "b2w" which can be used to identify and block the user agent . How this is used will depend on your platform. I have been in touch with Daniel Champion who is currently doing some work on this. If you are interested in excluding your website from future SiteMorse League tables I gather that you should be able to get some useful information on his blog www.blether.com sometime in the near future.
Thanks again to all those that have taken part in what has been a very interesting discussion.
_________________
Grant Broome
Blog
CDSM
Shaw Trust
Share:
The internet works on the OSI 7-layer model and the TCP/IP model. The TCP/IP network model contains four layers, unlike the OSI model, which contains seven layers unsurprisingly. Obviously industrial routers and media are major influencing factors as are nodes. Yes I've passed Cisco CCNA 2 so do know how to use/program IOS (Internetwork Operating System) I'd say the speed testing will be subjective to some extent. Testing one page for speed is rather short-sighted to say the least._________________
};-) http://www.xhtmlcoder.com/
WVYFC chose the Yorkshire Air Ambulance as the main charity to fund raise for in 2006
Share:
| Grant Broome wrote: |
| SiteMorse user agent always uses the string "b2w" which can be used to identify and block the user agent . How this is used will depend on your platform. I have been in touch with Daniel Champion who is currently doing some work on this. If you are interested in excluding your website from future SiteMorse League tables I gather that you should be able to get some useful information on his blog www.blether.com sometime in the near future. |
I've posted some information on my site about blocking SiteMorse and other unwelcome robots. It only covers Apache in any depth, if anyone has experience of restricting access under IIS, or links to any resources please let me know and I'll add them.
_________________
Dan Champion, Champion IS, Mooch Marketing, Revish
Share:
All times are GMT
You cannot post new topics in this forumYou cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum