A Funny Thing Happened on the Way to the Standard

Another bylined article I've submitted, which talks about how adoption of a global standard in an industry sector disrupts the market for supporting software:

Sometimes the consequences of a course of action lead to unexpected events, especially when different, apparently independent market forces and trends converge. Take the new regulatory submission standards for example. There has been a fascinating interplay of business and process requirements, human factors, new technologies and software market forces, which unfortunately, has made the implementation of submission management systems a considerable challenge.

Traditionally every regulator has had their own unique requirements for submission content and format, and often these regional requirements differed significantly according to product type (drug, device, diagnostics, etc.) and stage of development. The efforts of the International Conference on Harmonization (ICH) to develop a common submission structure applicable to applications at different stages have therefore been welcome. While there are still regional differences in organizational detail and required content, the resulting ICH Common Technical Document (CTD) format is a great advance that enables extensive document reuse and global process coordination in companies.

So what else happened? Why can’t companies immediately benefit from the ICH’s efforts? The development of the CTD came at a time when there was also considerable pressure to move from paper to electronic formats. Sponsors sought the efficiency benefits of internal electronic processes in submission assembly, review and approval, and regulators concluded that the continuing growth in the size of submission would ultimately render paper submissions impractical. In the case of the FDA this has become especially critical as the reviewers are moving to new facilities with a smaller not larger, paper document room!

As the CTD was developed, another ICH group followed immediately with specifications for the electronic form – the eCTD. While the CTD is relatively specific, the electronic form is more specific in its requirements. Basically submission managers have more discretionary choices in assembling a CTD that they do with an eCTD. The greater specificity of the eCTD is a consequence of both design and technology choice.

The eCTD working group chose an emerging technology, XML, to describe how eCTD submissions should be formatted. XML is a relatively new and powerful tool that is being widely deployed in many industries as a way to describe information and transfer it between systems. XML is extremely flexible, but can also be used to create hierarchical formats with strict controls. The eCTD’s Document Type Definition (DTD) essentially requires that electronic submissions follow a specific format based on the CTD, but with strict requirements at each level, i.e. only certain documents with specific names can go in certain folders. Software can be developed that automatically follows the DTD specifications, making sure that users can only add the right documents in the right places, and also to automatically validating assembled, electronic submissions.

The key point about a standard like the eCTD is that it does not matter what software product is used to assemble a submission, the resulting electronic submission is exactly the same, since it must comply with the standard. Open standards are disruptive to software markets. In the case of the eCTD, it is enabling separation of the submission management process into discrete steps.

A submission might be prepared with one tool but could be viewed with another. As a result, a number of vendors provide free eCTD viewers. Here we see that one aspect of the overall submission process, namely viewing, has become a commodity. Vendors can no longer include viewing as part of the value proposition of their product. But this commoditization of submission management is occurring at more than the viewing step.

The eCTD is designed as a standard submission format to ensure that the submission sent by a sponsor is exactly recreated at the regulator. But submission transfer is not restricted to a completed submission. Parts of draft submissions can be prepared in different locations within a company, or even by partners and service organizations, and then sent electronically to another system for final assembly. Again, each of these submission sections could be prepared and then assembled using different software products from different vendors, since what is being transferred must conform to the same standard (i.e. the eCTD). An open standard levels the playing field in a way that provides flexibility to end users, but it also presents big challenges to commercial software vendors.

A third example of the mixing-and-matching of submission products is the use of a submission assembler from one vendor and a validator from another. Yet another example relates to printing. While the eCTD is an electronic format, since it is based on the CTD, paper submissions can be printed from electronic submissions. Since regulatory authorities in Europe currently require that paper copies be submitted with each eCTD, this is an important requirement. Some software vendors now provide standalone eCTD printing products – again these products can print any valid eCTD prepared by software from other vendors, not just submissions prepared in a specific system.

When submissions were only paper, the available software products were monolithic, end-to-end systems that were required to do everything. Their proprietary nature made it very difficult for users to use other software tools during submission assembly and publishing. Now as we have seen, users can mix-and-match different tools to suite their needs.

Clearly the transition to electronic submissions in an eCTD format has created a technological discontinuity in the software marketplace. Not only can aspects of the process be performed by standalone tools, the traditional, established submission publishing products are not suitable – the new requirements need a new approach. Just a few years ago, before the eCTD was completed, there were two primary vendors in a mature, stable marketplace. At the Drug Information Association’s (DIA) North American December 2004 eCTD/CTD meeting six vendors presented their eCTD solutions, and at the February 2005 document management meeting 10 vendors presented eCTD tools in a submission workshop. The DIA organized these events to help life sciences companies that are facing difficult decisions.

While the growing number of vendors and products suggests users have a wider choice, the increased market competition has resulted in tremendous price erosion. The long-term viability of these specialty vendors is now in question since the number of customer companies has not dramatically increased to compensate for the lower product prices, i.e. the customer base remains the same and while the market size measure by total revenue is decreasing. In addition, this market is being divided between many more vendors, each selling products for far less. The current situation is reminiscent of the dot com era, but on a much smaller scale – small, venture-backed, startup companies are desperately competing to buy market share in a market that cannot sustain them.

Another technological trend is also emerging to disrupt the submission publishing business – the continuing development of powerful document management systems. Traditional paper submissions in life sciences had very specialized requirements for managing large document collections beyond the capabilities of early document management systems. In essence, the submission market developed because the available document management systems were inadequate. In the same manner, paper submission production required unique features to manage large print jobs on production printers, and both paper and early electronic submission formats before the eCTD required complex PDF manipulation at a time when PDF technology was relatively new.

The standard, corporate submission management system configuration is usually a hybrid. A document management system is used to manage the lifecycle of all components, electronic documents, and these are then ‘sent’ to a submission publishing system for assembly, manipulation and printing. These two systems have traditionally been sourced from different vendors, so synchronization between the two has typically been a problem for users, especially as document management systems were often highly and uniquely customized in each company.

Nowadays document management systems are far more sophisticated – indeed they are merging into a new category called enterprise content management (ECM). Modern ECM systems have many tools to support large collections of documents, as well as a range of collaborative process-support, tracking and reporting tools, and given the simplified, XML-based requirements of the eCTD, do not require much extension to manage the entire submission process, packaging and exporting the electronic documents that they already hold. There is significant benefit in using one, primary software product for the entire process.

There are clearly conflicting market forces in play. On the one hand we see ECM systems being capable of handling more of the submission process, and on the hand other specific steps of the submission process can be handled by purpose-build tools from several vendors. It may be that ECM systems will come to manage the overall process and be supplemented by specialty eCTD tools for specific steps in the process, e.g. validation and printing.

In summary, the development of a relatively standard, open format for electronic submissions in all regions, which should have made submission management a much easier process for life sciences companies, has created new challenges by disrupting the market for the necessary software support tools. Users are faced with a dilemma of choosing between a wide range of new and often incompletely developed tools from many different vendors, and are concerned both about the features they require and the long-term support for those tools (i.e. vendor viability). In some cases companies have simply resorted to building their own submission management software, at least in the interim. As history has shown however, pharmaceutical companies often build software for their own immediate needs, but ultimately they have to transition to commercially available products.

As with all markets, after this current period of uncertainty created by significant change, there will be maturation and stabilization. Already the ICH eCTD and the regional standards have stabilized after a period of rapid evolution, and the FDA for example, has committed not to make further major changes without two years notice. This provides a firm base for the impending market shakeout. The resulting balance between ECM/document management systems, submission management solutions and specialty software tools for specific submission tasks is as yet unknown.