At VoiceStream, the majority of our systems have a web front end. We look at user perception & satsfaction as being a combination of Performance (tradition definition - availability, response time, etc) and Usability (does the service/system align with user business processes). We are looking at adding Usability Testing results and improvement goals in our SLA. Am I way out of bounds in doing this? Do you see Usability Testing as fitting with the SLA concept of -Define -Measure -Assess -Set objectives -Monitor -Improve?

What are your opinions?

-- Stephen Bardin (stephen.bardin@voicestream.com), December 13, 2001

Answers

Stephen, You pose an interesting question. The approach taken at VoiceStream is somewhat uniqe. Normally, Availability is separated from the usual Performance indicators and becomes a category unto itself. However, that is more of a question of semantics than substance. One can include virtually anything in an SLA. The real keys to consider when deciding which factors to include as metrics in an SLA are: Measurable - Is this an indicator that can be measured? If you can't measure it, then you can't include it in the SLA. Relevance - Is this metric relevant to the service covered by the agreement? Meaningful - Does this indicator have any meaning to the user? For example, end-to-end response usually is very meaningful. However, technical measurements such as packets dropped, will usually be meaningless. Controllable - Can this metric reflect something that can be controlled? If not, forget it. It has no place in an SLA. Cost - How much will it cost to measure this? Can you afford the measurement?

From you message, it appears that you have a methodology for measuring what you describe as Usability. Presumably, that measurement is affordable. Control does not seem to be a question in this case. From my perspective, the open question seems to be whether or not this indicator will be meaningful to the user in an SLA. You have to answer that question. (Alternatively, you may want to employ Usability as a predictor of user satisfaction, but not include it in the SLA.)

To give you a more detailed answer on this question, I would need to have a better understanding of your environment.

Rick Sturm

-- Rick Sturm (sturm@enterprisemanagement.com), December 27, 2001.