Risks of Risk Assessment in Multiple Small Illumination Sources During Winter Conditions

["Rob, grandpa of Ryan, Trevor, Devon & Hannah" <rmslade () shaw ca>]

Risks of Risk Assessment in Multiple Small Illumination Sources During Winter 
Conditions
Robert M. Slade, version 1.0, 20121220

Testing can be used to demonstrate the presence of bugs, but never their absence.
          - testing aphorism

ABSTRACT

As follow-up research to the study "Risk Assessment and Failure Analysis in 
Multiple Small Illumination Sources During Winter Conditions" (first published in 
2003, and available at http://catless.ncl.ac.uk/Risks/26.26.html#subj11 ), the 
author has undertaken a multi-year study attempting to reduce the level and risks 
of failure in the illumination network required for celebration of the Northern 
Hemisphere Mid-Winter Party Period and Gift Giving Season.  (The nodes in this 
network currently stand at approximately 900 sources, and a significant portion 
may be noted at https://twitter.com/rslade/status/281528406917656577/photo/1 .)

Testing of nodes (also known as "bulbs") and subnets (also known as "strings") has 
been a major component of the risk reduction strategy.  However, recent studies 
have indicated that testing itself may be a contributing factor in node and subnet 
failures.

INTRODUCTION

In terms of risk management, it is well known that there comes a point of 
diminishing returns in the process.  The father of quality control, Walter Deming, 
noted that there was such a thing as too much quality assessment ( cf. 
http://victoria.tc.ca/int-grps/books/techrev/bkdeming.rvw ).  Despite the greater 
accuracy of assessment, very few enterprises engage in full quantitative risk 
analysis, preferring the less accurate but less costly (in terms of time and 
resources) qualitative risk analysis.

This study looks specifically at the testing component of the risk management 
process, and notes the probability that testing may contribute to total risk or 
failure.

TESTING IN THE LIGHT CYCLE

For details of the light sources and portions of the process, we refer readers to the 
earlier study ( http://catless.ncl.ac.uk/Risks/26.26.html#subj11 ).  A brief outline 
of the light source cycle is in order at this point.

Towards the end of September, the female members of the household, in 
preparation for upcoming events, start to ask the male members of the household 
whether any purchases or other preparation is necessary.  (This generally 
corresponds to the initiation phase of the cycle.)  The male members of the 
household point out that Canadian Tire does not start selling Christmas lights or 
decorations until November.  (This portion of the communication protocol is not, 
as many suppose, for information purposes, but to deflect discussion from the fact 
that the notes on necessary purchases and replacements, made last year, are 
packed away with the Christmas decorations, and are therefore inaccessible.  
Students of security may note that this is a good illustration of the importance of 
all three pillars of security: the confidentiality and integrity of the information is 
maintained, but availability is not.)  Testing at this point in the cycle might be 
useful, but is, unfortunately, impossible.

At some point in November, the male members of the household will have run out 
of excuses for not retrieving the Christmas decorations from storage.  At this 
point there is usually a mass retrieval of the decorations, and assessment of any 
items requiring replacement or supplement, or any perishable items which must be 
purchased each year.  (This corresponds to the requirements phase.)  Testing of 
light nodes and subnets may be done at this point.

This retrieval/requirements phase is generally followed by a design/planning phase. 
 To many researchers, it would appear that the ultimate result varies little from 
year to year, and that the design and planning is not necessary.  However, mature 
researchers will note that, as one becomes, well, "more experienced" in these 
matters, one notes a failing of memory as to the exact process from previous 
years, and sometimes even more recent events are difficult to ...

I'm sorry, where was I?  Oh, yes.

Testing and failure rectification can be undertaken during the design phase.  Some 
researchers feel that this assessment point can be skipped, but experienced 
researchers know that failed nodes will inevitably be discovered on the back of the 
tree in such cases.

During the implementation phase, testing tends to be somewhat informal.  Since 
the light nodes are being placed individually, failure of a node is generally obvious.  
However, if testing and rectification is not planned into the process, researchers 
inevitably find themselves balanced precariously on a stool at the back of the tree, 
with no replacement nodes, when a dead node or subnet is discovered.

The maintenance phase of the cycle generally runs from the first Sunday of 
Advent until January 6th (Feast of the Epiphany, last of the twelve days of 
Christmas).  Testing at this period is by observation.  Unfortunately, very much 
like testing, observation can usually tell you which nodes are shining, but not 
which ones are not.  As per the earlier study, it should be noted that a single node 
failure does not generally result in subnet failure, but that cumulative failures do.  
Therefore, failure to observe and rectify individual node failures frequently result 
in subnet failures at some point during this phase.  Rectification following subnet 
failure at this point is extremely difficult, and usually impossible.

The termination phase of the cycle involves "undecorationing," and return of 
items to storage.  Testing is possible at this point of the cycle, but is made 
problematic by a) fatigue, and b) haste in returning items to storage in order to 
allow for "spring cleaning."

RESULTS OF TESTING AT DIFFERENT CYCLE PHASES

Initially, this study looked at testing by observation during the maintenance phase. 
 It was felt that by observation and ongoing rectification, nodes and subnets could 
be maintained, and would therefore be in good order upon retrieval the following 
year.

Unfortunately, the following year some nodes and subnets were found to be dead.  
Therefore, testing at the termination phase was added.  This had the advantage of 
allowing notes to be taken during rectification, so that replacements could be 
purchased in advance, the year after.  As previously noted, this information was 
maintained, but was not available at a time when it would be useful.

Therefore, testing was added during the requirements phase.  All subnets were 
tested upon retrieval, replacements were purchased (if one could fight through the 
crowds at Canadian Tire), and rectification was done prior to implementation.  
During implementation phase on that study, it was found that nodes and even 
subnets were still showing as failed.  This led to the addition of an additional 
testing point during the design/planning phase.

During this past cycle, all nodes and subnets were tested and rectified during the 
termination phase.  Upon retrieval, subnets were tested and any failures rectified.  
During planning, subnets were again tested and failures rectified.  During 
implemenation, provision was made for rectification within the process.  So far, in 
the maintenance phase, failures have been rectified as soon as observed.  (One 
subnet failure was noted.  The attempt to rectify it was successful, but this is 
considered anomalous.)  Failure rates between testing points have been observed as 
high as 14% of total nodes.)

CONCLUSION

The results of the data collected are inescapable.  Testing results in failure.

ACKNOWLEDGEMENT

This study would not have been undertaken without the encouragement and 
support of Gloria J. Slade.

======================  (quote inserted randomly by Pegasus Mailer)
rslade () vcn bc ca     slade () victoria tc ca     rslade () computercrime org
  Great wits are sure to madness near allied.    - John Dryden, 1681
victoria.tc.ca/techrev/rms.htm http://www.infosecbc.org/links
http://blogs.securiteam.com/index.php/archives/author/p1/
http://twitter.com/rslade
_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.