Testing as a Shared Service in an Information Sharing Environment

When it comes to testing in an information sharing ecosystem, the needs of a test lab have unique characteristics. Such a lab needs to offer:

• Global accessibility. With the work force now being spread across the continent and potentially around the world, shared services including testing need to support a Global Reach.
• Continuous availability. Today's schedule compression and Agile project timelines dictate that development and testing occur at all hours and any day of the week. An effective information sharing testing platform must be operationally available 24x7x365.
• Support for Test Driven Development (TDD). Modern software development lifecycles, particularly Agile processes, depend upon a testing model that allows for frequent execution of suites of automated testing that help developers verify working parts of their code and identify non-working parts so they can focus their development efforts appropriately.
• Multi-user capability. By definition, information sharing involves multiple parties collaborating to exchange data in a syntactically and semantically interoperable way. The test lab must have the capacity to support multiple testers and testing organizations concurrently.
• Self-service. Because testing among participants in an information sharing environment is ongoing, the test platform can't depend on administrators or proctors to help facilitate testing or review test results. It needs to be fully self-service and as automated as possible.
• Hosting in the cloud. Given the needs around accessibility and availability, it makes sense that the test lab would be delivered from the cloud.
• Ability to test standards conformance. The test platform needs to incorporate a robust rule-oriented model for testing context and conformance to the standards appropriate to the information sharing environment.
• Capability for negative testing and exception handling. Testing the "happy path" provides only limited assurance of successful interoperability and conformance. Negative testing seeks to test the interaction among systems in ways that are unexpected but which nevertheless occur in production situations. It also ensures that structured exception handling logic in the code is sufficiently comprehensive and effective. This kind of testing is crucial for avoiding some of the production issues that are typically the most difficult to isolate and solve.
• Detailed reporting. The ideal test platform doesn't simply report success or failure of a test. It provides highly detailed statistics and troubleshooting information to help developers pinpoint the root cause of defects, correct them efficiently, and re-test quickly. For example, it's the difference between knowing your car won't start without knowing the reason, and instead knowing immediately that the starter has failed and needs to be replaced.
• Standardized test data. Effective interoperability testing requires a sufficiently large sample of test data that allows for proper exercise of all relevant aspects of the interoperability among the tested systems. For example, interoperability test data in the healthcare domain demands millions of individual "synthetic patient" data records for simulating a range of medical events across multiple systems and organizations.
• Communities of Interest. The shared testing environment can amplify its effectiveness by offering a mechanism for organizing Communities of Interest that attract organizations testing against the same standards and information domains. It allows these organizations to share their challenges and lessons learned, benefitting from the aphorism, "a rising tide lifts all boats."

Information sharing standards provide a common language and set of expectations that enable interoperability among message-based systems. Ideally, data exchange standards should permit data to be shared among various kinds of users regardless of application or application vendor. Facilitating interoperability among multiple systems involves the development of information exchange gateways and adapters. Gateways provide a "bridge" from one Organization (system) to another, enabling the secure transfer of information and allowing gateway-to-gateway communication. Every system must have a gateway service to exchange information with another system. Adapters attach to the gateways acting as a translator for both the gateway and application it represents for the message that is sent and received during a transaction. These gateways include the necessary interoperability standards to enable a secure and accurate transaction from one system to another.

Standards-based, interoperable solutions are necessary for effective and secure information exchange. As an example from the health care industry, a significant percent – some estimates are as high as 85% – of the information needed to care for a patient at some point needs to cross enterprise boundaries; interoperability is mandatory to ensure that information flows readily and securely.1

Additionally, interoperability provides an effective tool in combating fraud, waste, and abuse (FW&A) – an area that inflicts staggering costs on already burdened systems in various disciplines. Medicare and Medicaid FW&A costs taxpayers billions of dollars each year; beyond representing an economic concern, fraud risk affects all aspects of compliance, including patient safety.2 FW&A can be greatly reduced when medical claims data can be linked to an Electronic Health Record (EHR) and exchanged between the payer and provider. Utilizing advanced analytics, a provider can detect certain types of fraud, waste, and abuse before administering a procedure. The payer can evaluate the attached medical claim and either validate and pay the claim or invalidate the claim.3 Ensuring interoperability between the payer and provider significantly prevents fraudulent claims within the health care system, lowering health care costs to providers.

Ensuring interoperability impacts decision making as well. For example, a major component of health information interoperability centers around the accuracy of information and content sent between two or more health care systems. Providing immediate patient information and medical data to health care providers mitigates the risk of performing unnecessary medical procedures during a patient encounter. Doctors are given the freedom of abundant and accurate patient data securely exchanged among health care systems, to identify and perform necessary procedures. The end result is improved quality of care and avoidance of unnecessary costs. Similar benefits apply to interoperability in every domain from finance to law enforcement to logistics.

Organizations must have a solid understanding of Technical (syntactic), Semantic and Process Interoperability concepts to truly grasp the challenges and barriers to success. Technical (syntactic) Interoperability allows computers to reliably exchange data. This is accomplished with messaging protocols and data formats. Semantic Interoperability allows computers to unambiguously and consistently determine the meaning of the data for presentation and decision support – that is, the interoperating systems interpret the data in the same way. This can be accomplished with the use of standardized terminologies (e.g., ICD-10, SNOMED, etc.). Process Interoperability allows computers to exchange data and track workflows as records move between organizations. This is accomplished with standardized business process notation or workflow protocols.

AEGIS' experience in health information interoperability and conformance testing has revealed key insights in to what makes it so difficult. The nature of interoperability means there will always be external constraints affecting any one system's testing. As the cliché goes, "It takes two to Tango."

Interoperability involves at least two systems that need to remain in step with each other as they exchange and interpret shared information. A change to one exchange partner's system could affect interoperability with all its other partners.

Decisions made by one organization's CIO can impact that organization's interoperability with many other organizations. The bottom line is that interoperability testing cannot be viewed in terms of a point in time – because once that point in time has passed, interoperability can no longer be assured. One of the hardest aspects to understand when it comes to interoperability testing is that is must be continuous. Only when testing for continuous interoperability is in place can stakeholders be assured that their information sharing initiatives will likewise carry on without interruption. Continuous interoperability requires a robust automated testing approach and tools that focus on that important outcome.

The testing needs to begin early. If interoperability testing is not performed until a system is preparing to join an information exchange ecosystem, the probability of encountering serious defects is high. Interoperability testing needs to begin during the earliest development stages of a new system or release. The testing platform needs to be made available to the development team to use when they need it. Given the nature of distributed development teams and Agile development processes, such a platform needs to be available from anywhere in the world and any time of day or night. A cloud-based service is ideal for fulfilling this need.

The test platform needs to truly provide a production-like environment for effective interoperability testing. Such a platform needs to implement instances of information exchange gateways – not just 'stubs' or placeholders for those systems. Why? Because such placeholders inevitably involve certain short cuts and assumptions that don't necessarily represent how the various systems interoperate in the real world. This can easily lead to invalid testing results obviating the effectiveness of the testing.

Considering all the development teams working to produce interoperable solutions, this common testing platform that represents the full information sharing ecosystem needs to be able to support multiple development teams testing concurrently without adversely affecting other teams' results.

Testing can't become a bottleneck to the development team's velocity. Therefore, each team needs to be able to handle its own testing on the platform in a self-serve manner. Ideally, the test platform allows the development team and/or their testing body to create test cases or select from existing standards-based tests and repeatedly execute their selected suite of tests in an automated fashion.

At the same time, testing can't be limited to positive or "happy path" testing – that is, testing to ensure merely that the system does what it is supposed to do. In production, unexpected things happen and if systems don't handle those situations (exceptions) gracefully, they are vulnerable to failing in dramatic ways such as incurring a sustained outage or exposing secure information. Therefore, interoperability testing, like all software testing, needs to include negative testing. Negative testing in message-based information exchange addresses scenarios such as malformed messages. Testing such situations and identifying related defects during testing is critical because these are among the most difficult problems to chase down when they occur in production.

Finally, effective interoperability testing needs to not only indicate defects, but also provide sufficient supporting details for the development team top efficiently identify the root cause, correct the problem, and re-test it. This is the kernel of TDD. Testing environments offering highly visible results, drill-down capability in to message traffic at the source of the defect, and pointers to the problem(s) can dramatically shorten the troubleshooting cycle and speed the development and delivery of the system in to production. A related feature of the testing platform is a sufficient quantity of standardized test data available to the systems under test (SUT) that allow for exercising all aspects of a transaction between systems. Developing correct test data in an interoperability testing environment requires careful analysis and design of that data as well as a configuration management approach that protects its integrity.

Interoperability and standards conformance testing is not easy, but with the right testing tools and environment it doesn't need to be hard either.

1"Breaking Barriers to Interoperable Health Care." Government Health IT Conference and Exhibition. HIMSS. 2012. http://www.govhealthitconference.com/interoperability.aspx

2Testimony of Daniel R. Levinson, Inspector General, U.S. Department of Health & Human Services. March 2011. hhttp://www.finance.senate.gov/imo/media/doc/030211DL1.pdf

3Hanson, Susan P. "Fraud Control: New Tools, New Potential." Journal of AHIMA 77, no.3 (March 2006): 24-30. http://library.ahima.org/xpedio/groups/public/documents/ahima/bok1_030850.hcsp?dDocName=bok1_030850