Changes to the TL 9000 Measurements Handbook
TL 9000 measurements handbook R5.5
Earlier this year, I blogged about the TL 9000 standard and its Measurements component. TL 9000 is unique among management system standards in its requirement that every certified organization submit prescribed monthly measurements to its Measurements Repository System (MRS).
The organization can then turn around and obtain aggregate Performance Data Reports from the MRS which allow it to compare its performance against its industry peers – in other words, a free benchmarking service, which is a big part of the TL 9000 value proposition.
As part of its new TL 9000 Roadmap initiative, QuEST Forum has begun to issue what are being called point releases to deliver small, focused updates to the TL 9000 Requirements and Measurements Handbooks in an agile, timely manner. The first of these point releases is Measurements Handbook (MHB) R5.5, which was published earlier this year and became effective on June 30th.
It’s not a full handbook update. It comprises just three elements, representing additions or changes to the current MHB R5.0. Here are the details.
Incident restore rate (IRR)
The IRR measurement applies to just five Product Categories:
- 7.3.2 Network Operations Center,
- 9.1 Voice,
- 9.2 Wireless,
- 9.5 Internet Access, and
- 9.8 Video Broadcast Services.
If you report measurements against one of the above, then congratulations: your organization is about to start being measured on its overall responsiveness to incidents – that is to say, how good it is at restoring service to normal levels, following the identification of an incident, within the timeframe expected by your customer. The clock starts ticking when the organization is notified of an incident (either by the customer or by a network surveillance process) and stops when service has been restored.
The length of time it took to do that is then compared with the restoration target, typically set in a customer contract or SLA (failing which, per the organization’s own internal targets – failing which, a default of 2 hours for a central office, 4 hours for a remote).
The monthly IRR measurement is simply the percentage of incidents due to be restored in the month which were actually restored on-time. Note that service can have been restored in response to an incident without the underlying cause of the incident having been resolved. Don’t confuse IRR (which deals with incidents) with FRT (Fix Response Time, which deals with problem reports). FRT measures on-time closure of problem reports; IRR measures on-time restoration of service in response to incidents. (An incident won’t necessarily give rise to a problem report at all.)
Software fix quality (SFQ)
SFQ measures the effectiveness of an organization’s software fix processes. Specifically, it quantifies the risk to customers of introducing defective software fixes into their in-service releases.
The measurement used to be a simple percentage: the number of fixes reported as defective in the reporting month, divided by the number of fixes released. But there was a problem: the fixes reported as defective may well have been released in an earlier month. In any given month, the number of reported defective fixes could exceed the number of released fixes – resulting in an SFQ calculation of greater than 100%. Even worse, if no fixes happened to have been released in the reporting month, you could wind up with division by zero (which will horrify you if you’re a mathematician).
So MHB R5.5 introduces a fix to the Software Fix Quality measurement itself. It has made it an annualized measurement, just like the Field Return (FR) measurements. Now, instead of calculating it as described above, you multiply the number of fixes reported as defective in the month by an annualization factor (typically 12, for 12 months in a year) and divide it by the total number of fixes released in the 12 months up to and including the reporting month.
This makes the new SFQ an annualized measure of the percentage of defective software fixes per year. It not only makes better mathematical sense, but it also smooths out the measurement from month to month. (Calculated the old way, as a snapshot of a single month, the numbers could be very volatile.)
Normalized one-year return date (NYR)
This is an easy one! The NYR measurement was not perceived to be value-added any longer; so it’s been removed. Just quit reporting it! (But continue submitting your ERI, YRR and LTR data: those measurements haven’t changed.)
What do i have to do?
First, get hold of the new and changed MHB R5.5 content. Good news: it’s available as a free download from the tl9000.org website. The corresponding Product & Service Category Tables (PCT) R5.5 are there too, under Handbooks.
You may submit data per the new definitions beginning with the measurements for the month of June 2017, and must use MHB R5.5 beginning with December’s data. You will upgrade to MHB R5.5 just as you have to any previous Measurements Handbook revision: you will have to have submitted at least one month’s worth of measurements using MHB R5.5 (and PCT R5.5) before the external audit at which you transition to the new revision.
Need more info? On tl9000.org, under Handbooks, you’ll find a more detailed presentation on the R5.0-to-R5.5 delta, and examples of how to calculate IRR and the revised SFQ, as well as the new MHB R5.5 and PCT R5.5 materials. There’s also a tiny, perfect delta training course available (as e-Learning, under Training) for less than what you spend on coffee in a week. And, of course, you can feel free to contact the TL 9000 experts at NQA with your questions or concerns on this or any other topic.
Author: Rick Hill, TL 9000 Program Manager