Buggy Customers

Yesterday was one of those crazy days which start out with too much to do and end with you having done none of it despite being furiously busy. I participated in two web meetings (one in Italy and one in Boston) and a conference call and produced two documents – most of which were not on the plan as the day started. One of those documents was a quick position paper that essentially said, "No, there's nothing wrong with our software – you just have to think..."
There are strict requirements in the pharmaceutical industry to ensure that crucial events performed with software managing electronic records are captured and time stamped. An inspector from the FDA or other regulatory agency may want to know exactly what happened, who did it and when it occurred. Open Text Livelink runs a comprehensive audit trail and timestamps all of the events.
But in the process of validating their Livelink system according to regulatory requirements a customer discovered that when they viewed the same audit trail using two different computers as web clients the recorded times differed, and not even by an exact number of hours --> Big red flag waving, major problem, slam on brakes, system can’t be validated, panic, it is the software vendor’s problem, make them fix it yesterday, call Martin, fix it or make it go away…
Martin looks at problem, takes deep breath, answer simple = client hasn’t thought this through.
So what was the problem you ask Dear Reader (if there even be one!)? Well Livelink gets its time from the server(s) on which it runs (both application and database server(s)). Typically systems are configured to get their time from a time-server – this may be internal or external, in either case an atomic clock available on the Internet (e.g. at NIST) is usually the final authoritative source of information (good overview). As long as the servers from which Livelink gets time information have accurate and synchronized clocks, then the timestamps are fine.
So where’s the problem? Well users get confused – that old ‘human factor’ thing I keep running into. Livelink has an option to either represent time to users exactly as ‘known’ to Livelink (i.e. ‘server time’) or to reinterpret this time to each user’s local time – this feature is called the “Timezone Offset” and is a configured by a Livelink Administrator and applies to any client computer accessing a given Livelink system. The aim of this function is to enable users to conveniently understand events according to their local time.
To illustrate the usability benefits (i.e. trying to address the human factor) of the Timezone Offset feature consider the following example:
A Livelink server is configured to record and display times according to Eastern Standard Time but does not have Timezone Offset enabled.
A user working in Pacific Time adds a document as 10:00am.
If that user subsequently refers to the audit log they will see that their document was added at 1:00pm, not 10:00am.
However, if Timezone Offset is enabled, then whenever Livelink sends time and date information to the user in the Pacific Time zone it reinterprets the time to their local setting, so the user would see that they added the document at 10:00am as expected, not 1:00pm. This reinterpretation of time applies to all time events, not just those related to a particular user.While the Timezone Offset feature can aid users, it presents particular challenges in a validated system. In order for Livelink to reinterpret server time to a user’s local time, it has to determine the user’s current time. It does this by querying the time as known to the user’s client computer and compares this to the Livelink server time, calculating the difference between these two. This calculated time difference is then applied to all time references provided to a given user. The times in the authoritative Livelink record are not being changed, only their representation to users to keep them happy. If a given user’s computer clock is accurate then the times shown to the user correspond exactly to the times in the Livelink record, although they are adjusted to the corresponding local time equivalent.
In the real world unless adequate controls are enforced this can lead to unexpected variations. It is of course essential that the clock of any computer accessing Livelink be correct – not only must the ‘displayed time’ be correct (e.g. 05:13:12pm), but also the correct time zone must be set. In general the clocks employed in PCs are not accurate. Also, when users travel they often fail to reset their computer clocks or reset the displayed time but not the time zone. As a simple example, in a Livelink system when the Timezone Offset feature is activated and two computers differ by 2 minutes and 12 seconds, then all time events shown by Livelink will differ by this amount when compared between the two computers.
So how to address this? 1.) Ensure that the clocks of client computers are correctly set or, 2.) Establish procedures to deal with apparent time variations. If neither of these can be achieved, then the Timezone Offset feature should not be used. Is simple is it not?
So Martin writes this all out in more formal form (i.e. passive voice, qualified sentences, using official corporate template, converts to secure PDF, etc.), e-mails to sales and technical people on account, and then gets back to serious work on the next unscheduled task. Sigh…