The SLA Lesson: software bug blues

I have been hugely busy in the past weeks with several projects, so the blogging got stuck... I will try to avoid this in the future. Now back to my latest experience

Part of every Information Security Management System is the incident management process. It is as process in which the company identifies a problem which is occurring or has occurred, and performs steps to contain it, minimize the impact, identify the root cause and take measures to prevent the incident from recurring.

The incident in question is a dreaded application blocking - a company of 1000 employees uses a custom made fully integrated CRM/ERP system, which exhibited complete or partial non-responsiveness of several minutes for a period of nearly two hours. This situation was identified at several departments, while the rest of the company is functioning as usual.

As soon as the call came in, the incident response team was formed and the problem was analyzed. After 15 minutes, the problem was identified. Accounting has started a program which should run once a week and affects the billing information of most Key Customers. This program was started at it's usual time, with usual parameters. The problem was rectified by stopping the processing and postponing it for after business-hours

Upon further investigation of the incident it was identified that the problem has occurred before, at regular intervals, but was never reported as an incident. The situation has been handled by the IT department, who communicated the problem to the software company which created the software as a bug.

When i requested a status update from IT on this bug report, i received a shocking information: The software company has closed the bug report with a status of DENIED

So I called the release manager at the software company, and i got an even bigger shock: He explained that the software company decided to deny this bug report due to overwhelming change requests and bug reports from our company. In his words, this bug was a mere nuisance since it blocked part of the software for about an hour once a week - just run it during lunch!

At this point, the incident was no longer just an incident, it became a support contract issue, so i reported the situation to management and asked for their involvement.

This incident is a very good lesson in the different priorities and focus of the parties involved:
For a user of the system any problem can be a show stopper.
For the manufacturer of the system, the same problem can be played down to an importance of an itch. There can be many reasons for such a difference in opinion, but here are a few:
  1. There are insufficient human resources to address the issue
  2. There are profitable change requests or projects to to address, so this element is merely postponed since the software company will not see a profit from engaging their resources into correcting this problem.
  3. The problem is caused by a design flaw in the system, that is either very difficult or impossible to rectify in a reasonable time and within reasonable budget
The only way to increase the value of the users' incident to the manufacturer is through applying proper controls and penalties in the support contract. That is why security incidents history and results should also be used as a very valid input into the preparation and negotiation of the SLA

Popular posts from this blog

Cracking a TrueCrypt Container

Image
This week i tried to open an old TrueCrypt container. It turned out that i had forgotten the password. So I endeavored into the realm cracking the TrueCrypt container. Here are my experiences The problem I have a TrueCrypt container in which i hold my personal documents. The container is created with TrueCrypt 6.1a. Since i haven't been using the documents for a while, the password slipped from my mind. I a moment of desperation I tried to crack the password. The preparation To automate the process, I used the true.crypt.brute tool in version 1.9b. It is a very straightforward tool to use, but it has one drawback - it tries to crack based only on a pregenerated wordlist. That means that you need to generate your possible passwords list and let it rip. First, i created a simple encrypted volume with a 2 character password to check the software. It went through 819 passwords within 45 seconds and decrypted the password. This would mean that the brute force crack would run
Read more

Essential Management Semantics - Responsible vs Accountable

I've had a discussion at the office about who is responsible for a certain activity. And as expected, the junior colleagues got into a discussion of who is more and who is less responsible for the activity. The Information Technology Infrastructure Library ( ITIL ) defines two distinct roles:  Responsible and Accountable If you open Websters dictionary ( www.websters.com ) and look up the adjective " responsible " you get the following description: answerable or accountable, as for something within one's power, control, or management If you do the same for " accountable " here is what you get: subject to the obligation to report, explain, or justify something; responsible; answerable . It is a common sense to assume that "accountable" and "responsible" are synonyms. But both in Management and IT their meaning differs slightly and that makes all the difference: Accountable is the PERSON  (singular) who answers for
Read more

Steganography - Passing through the defenses

Image
Steganography is still considered to be a part of the obscure tools of secret agents and corporate spies. However, steganography tools are widely available, and anyone can use them. Most of these tools are now available online. But a lot of systems currently perform some form of resampling or filtering of images. This poses an interesting challenge -  how survivable is steganography in filters? This also gave us a great reason to publish another set of pictures (albeit cropped) of Lena Söderberg ;) Here is our original image Proposed Counter-Steganography System The filter system will need to be cost-effective, minimally intrusive and not prone to error. Since there may be many different steganography alghorithms, the filter system should not try to read such messages. Doing so will require an entire farm of filter servers. Instead, the systems will resort to a much simpler mechanism: Modify all passing images so that the original hidden data is compromised. Use only minute c
Read more