Chris Gill from Washington University in St Louis, introduced the second technical session.

The papers in this session will touch on distributed control and sensing in networked medical device systems. These are real time embedded networked system infrastructures for MDSS, and the development, assurance and medical practice-driven models for high confidence medical device software.

These systems are characterized as being made up of interlocking systems of (control) systems – interoperability, where devices in the system have an overall control structure. Such systems also embody various algorithms for processing data from sensors or other systems. There is invariably a human in the loop when operating these systems. As they develop, these types of systems must overcome systems integration and performance issues. A formal framework for embedded and hybrid systems is also needed.

Other key issues that must be addressed for these systems of systems include:

  • Interoperable data, devices, computation, communication
  • Security and privacy
  • Large-scale medical information management
  • Interaction of devices with different levels of criticality (e.g., managing safety and criticality)
  • Multiple-level QoS tradeoffs and QoS guarantees
  • Design for certification

Give the interoperable nature of these systems, co-development and co-assurance is a given. From and model based analysis, design and implementation of a system must be considered early on. Achieving user-centered designed for medical devices is also a challenge.

Finally, two additional issues for consideration: How do we balance highly configurable interoperable systems made up of multiple devices. Also, new ideas in “science of certification” and regulatory practice are for “Cyber-physical Systems” in HCMDSS.

Formal Methods Based Development of a PCA Infusion Pump Reference Model: Generic Infusion Pump (GIP) Project
David Arney, Raoul Jetley, Paul Jones, Insup Lee, Oleg Sokolsky, University of Pennsylvania, FDA/OSEL

This project developed a generic pump safety model, based on a study of accepted formal techniques for developing safety critical embedded systems. They applied an extended finite state machine system model to create a hazard model.

GIP system structure includes a pump controller, patient monitor, alarm manager, caregiver interface, hardware and generic infusion pump. All of this is connected to a network to log data. Requirements were gathered from numerous sources for a generic PAC pump. Sample requirements were provided.

The hazards and FMEA were divided into software, hardware and usage scenarios. Hazards were divided into failure mode, description, severity, probability, delectability, causes, and mitigating strategies. Sample hazards included calculation error, wrong does entered, air in line, and output occlusion.

A conventional state machine model was used to create a reverence model for pump operation. Multiple communicating state machines to model operation scenarios.

Test generation covered all states, transitions between states, and modified conditions/decision coverage. Test cases were used with a test driver or application interface that mapped variable sin the model to structure or functions in the implementation.

The testing was successful and an open source package will be available for device manufactures to use. Future work will put together a full running system.

They’ve had a very hard time to get any pump manufacturers to work with them. Right now they are looking for future collaborators to extend their testing and implementation research.

The have also considered what would be involved in putting this software test system through the medical device approval process. Current FDA process does not clearly map to software generated software.

This is similar to automated software testing done by software vendors – something that the vast majority of device vendors have little or no experience.

Point-of-Care Support for Error-Free Medication Process
J. W. S. Liu, C. S. Shih, P. H. Tsai, H. C. Yeh, P. C. Hsiu, C. Y. Yu, W. H. Chang, Academia Sinica, National Taiwan University, National Tsing Hua University

Jan Liu, from the Institute of Information Science Academia Sinica (http://sisarl.org/), presented Point of Care Support for Error Free Medication Process. The process starts with a prescription, the creation of a prescription order, ending with a personal medication dispenser. Liu presented their motivation and goals, use scenarios, architecture and implementation, and ended with calls for collaborators.

Over 25% of admissions to nursing homes related to improper use of meds, and 125,000 die annually due to non-compliance with meds. Medications regimens can be very confusing – take one twice a day, one 4 times per day, another 3 times with meals, etc. Consequently, a number of patient dispensing products have been developed – all with considerable limitations (like programming the old VCR).

The project made the following assumptions: a personal dispenser manages all meds and health supplements of the user; directions used by the dispenser are based on complete medication record of the user; and some network connection is available. They also assumed that the pharmacist knows all the meds a patient is taking (rather than an individual physician). The user’s habits and preferred meds times are taken in to account when possible. And the dispenser is allowed to monitor and record medication history.

The system receives data on prescriptions via a third party prescription order entry system. The system takes the orders as input to an authoring tool that looks at ordered drugs, drug interactions, the patient’s EMR, and then creates a medication schedule specification (presently XML).

The dispenser contains wells into which individual bottles for each medication are housed. The bottles will have RFID tags for automatic reading (bar codes were rejected due to the complexity of having users scan the tag). They dispenser knows which drug is in which socket due to RFID. An annunciator (on the dispenser or via cell phone message) indicates when to take a medicine, and a tray opens with the drug to take.

If compliance errors occur, appropriate messages are generated for the patient. They identified 8 events and actions that could occur alone or in combination. Recognizing that similar scheduling and compliance issues exist in acute care delivery as with self administered drugs, they have also targeted ambulatory and acute care systems. They noted that current acute care medication dispensing systems are not well integrated into other systems. Detailed workflow for meds administration in an acute care environment was presented.

Core competencies include underlying models and algorithms for medication scheduling and dispatching; component-based dosing, the architecture and implementation of dispensers; and device software quality assurance. They’ve been challenged developing reliable dispensing mechanisms. Questions remain regarding device and user interaction, and user behavior models (e.g., what if patient wants to travel and can’t take dispenser).

This dispensing is based on accurate and complete drug data that is in a format that can be used by the system. They are currently using drug libraries to take advantage of their concise and consistent descriptions. In addition to creating a consistent nomenclature and structure for describing a prescription, there are also interactions to take into consideration. Numerous constraints like temporal distance, diagnosis, maximal and minimal constraints must also be supported for safe and effective medication administration.

They are looking for collaborators in medication prescription content and representation. They would like to work with physicians, pharma, and over vendors in the meds administration chain.

A Survey of Software Engineering Techniques in Medical Device Development
Raimund Feldmann, Forrest Shull, Christian Denger, Martin Huest and Christin Lindholm, Fraunhofer Institute, Lund University, SWEDEN

This was the most depressing presentation of the entire workshop. The cozy insular world of medical devices has caused the industry to sorely lag similar industries with regards to product development methodologies and resulting embedded software quality.

The motivation for the survey was that software is integral to embedded systems and clinical workflows. There is a tremendous amount of innovation in the IT field, and software makes up a major portion of embedded system product and development costs. Studies predict that medical device R&D investments will increase to 33% of the overall budget by 2015. In 1996 10% of medical devices recalls were caused by software-related issues. In June of 2006, software errors in medical devices make up 21% of design defects.

They wanted to characterize medical device software regarding software engineering methodologies, benchmark this with other embedded system industries, and to explore current challenges. The goal was to use the results as input for better integration of software engineering standards into applied processes and to develop new methods tailored to the medical device domain.

The survey targeted medical device and medical information systems with a web based survey. The survey addressed countries world wide, and was done June to September 2006. The survey was promoted on the Internet and via direct mail.

The survey was not fully randomized, but appears to be consistent with similar studies. Admittedly, the survey may not be indicative of the state of the entire industry; however, large percentage of respondents was alike in having a primary focus of safety critical software. Results seem to be consistence with experience of domain insiders from which the received feedback.

Unlike many US centric studies, this study’s respondents were from around the world. North American respondents only made up 13% of the sample. Software teams were made up of 1 to 10 engineers made up 49% of respondents. The importance of software to the final product was rated as very important in 84% of responses. And safety-critical devices were involved in 76% of responses.

Most vendors both develop their own software and use some third party software. Only 36% of engineers were computer scientists – respondents noted difficulty in finding these engineers. The activity that causes the most software problems was reported to be the activities related to requirements (63%). Next biggest problem was architecture (16%), then related to implementation (10%). Analysis of survey responses indicated vendors have problems with changing requirements, out of date embedded software architectures, generally miss opportunities for software component reuse potential, and have poor code maintainability. Improvements in requirements engineering seems most promising to gain significant improvements. This contrasts with the fact that most effort is spent on test (40%).

So, how are requirements specified? Natural language dominants requirements as always or frequently used (52%). Structured notations (e.g., use cases) are only used by 40% of respondents, and only 2% use formal languages for describing requirements, architecture and design. Even Microsoft uses more robust requirements gathering that that used by most medical device vendors.

Tool usage was infrequent, with poorly defined processes. A scant 12% use tools to automate testing. Tools are most frequently used in risk management (63%) with Microsoft Excel noted as the most frequently used risk management tool.

Techniques to ensure software quality lead with black-box functional testing (74%), inspection and reviews (56%), structural testing techniques (29%), and software risk analysis (FMEA) at 18%. Less than 5% of medical device vendors collect code and design metrics for all their projects. Sadly, if you don’t measure it, you can’t manage it.

The studies conclusions noted that there was no significant difference of the perceived challenges between small and large companies, who where focused on usability, maintainability, reliability, and safety. While the importance of software is well appreciated, less formal processes, notations, and tools are predominant. Most software quality problems occurred in the early development phases.

Defect prevention instead of detection is sorely needed by the industry. The industry also needs to optimize requirements activities. Systematic techniques for requirements elicitation and management are needed; traceability is key. Investing in software architectures is needed, and could serve as a foundation for plug and play interoperability. This would include systematic reuse with software product line architectures and component-based development.

In my experience, the major barrier to software product line architectures is the way companies in the medical device industry are organized. The vast majority of companies are organized into silos that match their target markets, divide along product categories or both. Each silo group has their own marketing and R&D, sales is usually less fragmented. Getting these individual R&D groups to agree on a product line architecture is near impossible without the very active participation of senior management to provide adult supervision.

A very low adoption of configuration management, combined with the long life cycle of medical devices, creates problems in resolving software errors, and determining where and when the error was introduced. This only gets worse when connecting devices into a system.

Raimund also noted the Department of Defense is publishing best practices for software

If you want a full report, email Raimund at [email protected]

Technical session questions:
I asked if vendors have a hard time finding and maintaining engineers because product life cycles are so long and projects so infrequent. The survey did find this to be an issue. Another attendee noted that many graduated computer scientists are not educated with the appropriate skills for embedded systems engineering. This was acknowledged, and it was noted that the accumulated knowledge and training is key.

With their research in other industries, how did the medical device compare? Compared with the automation industry and some other industries, the medical device industry is way behind.

In Taiwan, vendors have found that computer scientists are poorly prepared for developing consumer electronics embedded systems. They have turned to electrical engineers for embedded software.

The amount of software errors has doubled over the period reported in the study – it was noted that the functionality in medical devices has also increased, and it was suggested that in fact, error reports by functionality may have gone down.

Requirements at Kaiser are frequently gathered by clinicians trained as systems analysts because it is easier to train them to write requirements than to take a systems analyst and train them on nursing. They also use natural languages because the training hurdle for structured requirements languages is very high. Raimund noted that requirements can certainly be gathered initially via natural language, but subsequent analysis should elicit structured consistent requirements for generating specifications.

After the Technical session, there was a brief period where Abstract Talks were presented. Here are some of the abstracts that were presented.

SOA Medical Device Interoperability

Model Driven Architecture: OMG (not “oh my god” but “object management group”) – model driven architecture solutions. Case Study: tele-surgical robot.

MD PnP Case Study: Use Case Demo: X-ray and Ventilator. During surgery ventilation is frequently interrupted to take certain x-rays, the system provided a reduced ventilation rate and synced the x-ray with breaths at either inspiration or expiration. This was another example of simple device interoperability that does not exist commercially.

Workflow Based Integration Framework for Medication Use: to eliminate drug related injuries, focused on communications and avoiding drug interactions. To reuse existing technology, they used a workflow engine to integrate disparate systems at authoring, database, prescription entry system, and dispenser. The workflow includes doctor, pharmacist and user/patient as actors.

Software Certification for Distributed, Adaptable Medical Systems: Challenges and Paths Forward, from BBN Technologies. BBN works on highly distributed high reliability defense systems and wants to transfer their techniques to medical devices. The new generation of distributed adaptable software systems presents numerous challenges. These complex systems have distributed resource usage and system configurations which change in real-time. They are designed to allow more effective usage of limited resources to meet evolving user needs by dynamic adjustments and reconfigurations. These systems are difficult to Certify due to required (implicit and explicit) scheduling of distributed interactions while maintaining system safety. Conventional approaches like exhaustive testing, documentation, code review, and other formal methods for high-confidence software are no longer economically feasible for highly complex, dynamic, distributed systems. As complexity increases there is the inherent potential problem of state explosion – where permutations and variables rapidly increase to become unmanageable.
A particular stumbling block is absence of methodology for dealing with dynamic resource management. Current methods assume the static allocation of states. Interesting.

Christopher Gill presented Cyber-Physical Systems Software for HCMDSS. Chris suggested that current operating systems and middleware are not well suited to controlling even single physical domains. Lee has several good observations about how time got left out of computing; abstractions, like processing threads that don’t consider time. Patient physiology requires that the semantics of multiple domains be integrated rigorously. Failure cascades involving multiple cyber and physical elements can occur. New work has laid a foundation for better dealing with cyber-physical systems. Timed coordination models, timed futures and hierarchical scheduling are useful. A new cyber-physical operating system and middleware would be ideal.

Tammara Massey from UCLA presented, Towards Reconfigurable Embedded Medical Systems. This presentation went to crossing the chasm from individual embedded systems to broader systems that ranged from multiple different peripherals to multiple interoperable embedded systems. She described two different projects, one a reconfigurable gait study system, the other was fabric based sensors.

Marwan Abdeen presented FDA: Between Process and Product Evaluation. Process oriented software validation and certification has been adopted by the FDA. While the FDA recommends that software validation start at the beginning of the software development process, it does not specify specific development processes or models. This paper questioned the clarity and effectiveness of the FDA’s process oriented approach to software validation, contrasting the FDA approach to a more proscriptive one that included specific tools and methodologies. They concluded that the FDA validation approach leads to confusion in the minds developers. The absence of requirements for metrics and measurement of the development process was presented as a shortcoming of the FDA’s approach. The Common Evaluation Methodology (frequently used for security and defense applications) was offered as a more rigorous alternative to the FDA’s QSR.