Summary

The terms model and model-based can be interpreted very broadly. This has not least to do with the fact that a model is always created for a specific purpose and the set of legitimate models is at least very large, if not infinite.

What they all have in common, however, is that they are simplified or reduced to what is essential. The aim is always to facilitate the activities it carries out.

General advantages and disadvantages of model-based methods:

The advantages lie in the improvement or support of work processes.

The necessary tool support can be considered a disadvantage. This can be further subdivided into tool costs, familiarization with new tools and the danger of tool lock-ins (strong ties to proprietary data formats, etc.).

However, the disadvantages mentioned can be greatly reduced with a considered choice of tools/development tools. Of course, even then it is still necessary to weigh up the advantages and disadvantages. At IMT, we are convinced that every project, regardless of size, can benefit from the use of model-based methods.

Examples for application of model-based methods at IMT:

For the development of specific electronic and mechanical components, we use CAD systems such as “Altium Designer” or “SOLIDWORKS 3D CAD”. The majority of such systems are already established on the market.

When developing larger systems, we mostly use MBD. In addition to the benefits of rapid prototyping, this also enables early fault detection through the simulation/testing approaches offered by the “in-the-loop” processes. Among other things, IMT relies on “MATLAB/Simulink” for this. For example, this is used in the development of measuring instruments.

If the product also has to meet strict standard requirements, it makes sense to combine this with an MBSE approach, which supports proof of conformity with the standard. For example, this is the case in the development of medical technology products.

Model-based methods are often used to implement smaller projects such as test automation tools, which allows “boilerplate code” to be generated. Among other things, IMT uses its own “DATAFLOW Designer” software tool, which follows an MBSE approach.

Introduction

The (EN) ISO 13485 in its 2016 edition, as well as the FDA Guidances and the European Medical Device Regulation (MDR) require software used in the quality management systems of medical device manufacturers to be validated. While ISO 13485 and the EU MDR do not go into detail on validation should look like, the US FDA published guidelines on January 11, 2002: “General Principles of Software Validation; Final Guidance for Industry and FDA Staff”. The ISO and IEC working groups have also considered this topic and published ISO/TR 80002-2 in 2017: “Validation of software for medical device quality systems”.

The aim of this article is to demonstrate a simple way to set up tool validation.

Software List

Both ISO 13485 (in chapter 4.1.6) and the FDA Guidance (in chapter 4.8) require software validation to be commensurate with the risk of the software, regardless of the size of the company or the resources available. Accordingly, a quality management system requires an overview of the software applications used and their intended use. Preferably, this overview will also document whether the software is

• “Off the shelf” software (i.e. software that can be purchased as a mass product in stores),
• adapted software
• or even software explicitly developed for the intended use.

For standard software such as Microsoft Word, which is installed in the thousands, the probability that an error will be found through the swarm intelligence of its thousands of users is probably much higher than through tool validation. This circumstance should be taken into account in tool validation. It is possible that the process for standard software, which is only used for non-critical purposes, has already been completed at this stage.

Figure 1: Software list

Software Validation Plan

The software validation plan should include the following elements:

• Description of the software including intended use, intended users and their environment
• Software requirements
• Test specifications for software requirements
• Risk management plan for the software to be validated

These elements can either all be documented in one document with different sub-chapters or they can be outsourced to different documents. The latter is particularly suitable for more extensive requirements documents.

Description of software

The chapter or document describing the software should describe the intended use, the intended users and their environment. Use case diagrams can be beneficial to give a simple, visual overview:

User requirements in the software

The user requirements should be in a testable format and given a unique ID. At IMT, we mostly use the following syntax:

for example, such a requirement could be

or

Both requirements have a unique ID and can be tested. Requirements that cannot be tested are to be avoided. For example, “fast” should be replaced by a time such as “2 seconds”, or “bright” by “under an inspection lamp with 1500 lx”.

Test specifications for user requirements

Each user requirement requires at least one test. In addition to a unique ID, the test specification includes the test instruction and the acceptance criterion or expected result. A test for the UR-001 above could look like this:

Risk management plan for this software

Since the effects of a software error in the quality management system are different from those of a medical device, the risk management plan must also be adapted accordingly. In particular, the definitions of the impact categories must be considered differently. Depending on the area of application, the same impact categories take on completely different meanings. The impact category “critical” potentially means the death of a patient in the case of a medical device, but the loss of data in the case of medical device tracking software.

Testing and assessment

Before the validation report can be produced, the software must be tested and the risks assessed.

Testbericht

The software must be tested according to the test specification. In the process, not only a “pass/fail” must be logged for each test step, but also the actual behavior observed, so that the test can be re-enacted. Therefore, the software and its environmental conditions must also be logged. This includes:

• Name and manufacturer of the software
• Version, build number
• Operating system
• 32/64-Bit environment
• Peripherals used
• ….

Date, examiner and 4-eye-principle evaluation

Using the above example, the report could look as follows:

Risk analysis

The risk analysis should assess all known problems. In the case of software that is distributed for use in a regulated environment, a corresponding list of known anomalies is often also published. It is worthwhile to check the manufacturer’s support website or to contact support. In addition to problems known to the manufacturer, shortcomings such as missing features must also be assessed. If a risk is not acceptable, control measures must be defined to reduce the risk. As medical device manufacturers who are used to risk management according to ISO 14971, we recommend following the same process.

Software validation report

The software validation report should include the following elements:

• Identify software and environment
• Test report(s) or reference(s) thereto
• Risk management report on the known anomalies of the software
• Evidence that all requirements have been tested. This is most easily mapped via a traceability matrix.
• Formal evaluation and approval of the software for use

NB: For “smaller” applications, it is possible to produce the test report, risk analysis and validation report in one document.

Identifying the software and environment, test report

Often there is a matrix of validated versions and environments for the software. If this is the case, it should be mapped accordingly in the validation report:

Risk management report on the known anomalies of the software

The validation report shall further assess the residual risk arising from the use of this software. The focus is on the acceptability of the known anomalies and/or missing functions.

Traceability

Another aspect to cover is to prove that all requirements for the software have been tested. Depending on the scope of the requirements and the documentation of the tests, two ways of documenting this have emerged:

Forward linking of the requirements to the tests

A column is added to the requirement table where the requirement is checked:

This variant is particularly suitable for a small scope of requirements and if the entire validation is created in one document.

Traceability Matrix

In the case of extensive requirements documents or if the test implementation has its own IDs, it is advisable to create a traceability matrix that contrasts the requirements with the test specification and possibly the test report:

If several reports need to be mapped for different runtime environments and software versions, this can be mapped very easily in a traceability matrix by adding additional columns.

Formal evaluation and approval

Last but not least, the entire package of documentation should be evaluated and documented with an approval decision.

Summary

As shown in the images, the validation of the software in the quality management system is nothing more than the systematic, documented execution of the upper left and right corners of the V-model widely used in software development:

This includes:

• Intended benefits and requirements
• Risk management
• Validation of user requirements
• Assessing the risks
• Assessing the overall validation and approval of the software for its intended use.

Introduction

Embedded „system design“, „system architecture“, „software architecture“, what exactly is behind these terms? What do they mean? What are the definitions? Why is it important to differentiate between these two terms?

This article deals with the answers to these questions.

The system

Here, we are dealing with technical systems, Or more specifically, with embedded systems. An embedded system is defined as follows:

“An embedded system is an electronic computer that is integrated (embedded) in a technical context. In this context, the computer either performs monitoring, control or regulating functions or is responsible for some form of data or signal processing, for example in locking or unlocking, encoding or decoding or filtering.”

Source: Wikipedia

Very similar to this definition (source https://www.embedded-software-engineering.de):

“An embedded system is a binary-valued digital system (also called a computer system) that is embedded in and interacts with a surrounding technical system. Here, the computer usually takes over monitoring, control or regulation functions, but is often also responsible for some form of data or signal processing.”

These definitions can be clarified with the following diagram:

Figure 1: General embedded system with the desired function/responsibility, surrounded by external systems and one or more users.

For the sake of simplicity, we usually simply refer to the “system” we are dealing with. Throughout the remainder of this article, we will often only refer to the „system“, which means the corresponding technical or embedded system.

Architecture and design

In order to explain the terms, we must demonstrate how such a system is built. Figure 1 presents the embedded system as a black box. The structure of the system has not yet been defined. The components from which the system is to be built must be defined. This activity is known as “design”. A “system design” is therefore about defining and determining the system structure. Based on the diagram (Figure 2), the result of this design activity could look as follows:

Figure 2: Result (architecture) the first design iteration to define the components (architecture elements).

Design activities tend to be repetitive. This means that the components—referred to here as architectural elements—are decomposed until they are sufficiently defined. Only then can the creation (development) of the system begin.

The result of this design activity is called “architecture”. Figure 2 is therefore a first “system architecture” (strictly speaking, Figure 2 is a reference architecture of a general embedded system), although the term architecture still needs to be defined more precisely in this context. There is no single established definition of the term architecture. You can find a list of definitions here: Definition Software Architecture. While these are definitions of software architecture, they also apply to the term system architecture. The distinction lies in the scope—more on this below.

We use the following definition of architecture (from „Software Architecture in Practice – Third Edition“):

Definition of (software) architecture:

“The (software) architecture of a system is the set of structures needed to reason about the system, which comprise (software) elements, relations among them, and properties of both.”

It contains the following 3 sentences:

1. The architecture of a system is built from a set of structures.
2. These are the structures needed to be able to infer the system.
3. The structures include the elements, their relationships and their properties.

So, what does that mean?

The structures consist of the elements, their relationships and their properties. Architecture consists of structures, which are needed to ensure that the system behaves as required. The whole area of non-functional requirements is also meant here, in addition to the fulfillment of functional requirements. For example, requirements regarding further development usually influence the structure of the system. However, not all structures in a system belong to architecture. Not all structures are architecture-relevant. These structures are then no longer referred to as architecture, but as design.

“Design” is on the one hand the activity that leads to an architecture (see above), and on the other hand the term for all non-architectural results of the design activity.

Many definitions simplify things by stating that the rough structure or the structures defined early in the design process are architecture. All later or more detailed structures are design. However, these definitions are insufficient, as more detailed structures may well be necessary to meet the required demands. Also, in agile development, not everything relevant is defined early in the design process.

System versus software

Finally, the distinction between “system architecture” and “software architecture”: As already mentioned above, they differ in scope. The system architecture includes the structures of various system elements, such as hardware, software and mechanics. The software architecture, on the other hand, comprises the structures that define the structure of a software element (an element from the system architecture). This can be seen in the diagrams below:

Figure 3: System architecture diagram with hardware and software elements.

In the further decomposition of a software element, we eventually end up with the software architecture:

Figure 4: Software architecture diagram of software elements “Control Software”.

Summary

The most important terms and their definitions are summarized again in the following table:

Table 1: Terms and their meaning/definition

Introduction

The documentation for system and software architectures is explicitly required for medical systems with safety-critical requirements, such as patient monitoring systems or ventilators. In addition, proof must be provided that an architecture has also been implemented according to its definition. But even outside the regulated area, the accurate description of an architecture is useful for most applications, because the description of an architecture is not only proof that requirements have been fulfilled, but it also helps to detect errors in the specification at an early stage. This article shows which (auxiliary) means can be used to describe an architecture and what can be followed.

Description of architecture

As explained in the article on system and software architecture, the architecture of a system is “the set of structures needed to infer the system, or to ensure that the system satisfies the required properties. ”

Admittedly, this definition is rather abstract and shows no concrete reference to hardware, electronics or software. However, a system consists of different structures such as software, electronics or mechanics. In addition, a system can also be characterized by logical structures or temporally sequential processes. In this sense, the term “the architecture” can be misleading, as the singularity of the term architecture might suggest that an architecture consists of a single image. However, practical experience shows that a system must be viewed from many different angles (views) and thus must also be described. For example, a system may be divided into logical units and functions that do not necessarily correspond to the physical division. This can be shown by the following, highly simplified, example:

Figure 1: Logical and physical view of a client-server application and an associated scenario to connect the views.

But which is the right or best view, and most importantly, the one recognized by regulations?

An architecture description usually consists of showing different views, making connections between views, and making the fulfillment of different requirements clear. In this respect, the description of an architecture consists of several views, which highlight different aspects. That is, there is no one true view, but the description of an architecture takes into account different views equally [1].

Views of an architecture

The views are an essential part of the international ISO/IEC/IEEE 42010 standard [link to https://www.iso.org/standard/50508.html], which provides guidance on how to describe the architecture of a system. This standard was published in 2011 and is the result of a joint ISO and IEEE revision of the earlier (software-heavy) IEEE 1471 standard [link to https://standards.ieee.org/standard/1471-2000.html]. The original IEEE 1471 standard specified how to describe the architecture of a system. Further requirements, such as the structure of an architecture as well as requirements for description language, were added in 42010. However, the standard does not require a specific architecture description or description language. Instead, the practice of describing an architecture should be standardized.

Views are intended to represent a system by considering stakeholder requirements (concerns) based on defined viewpoints. Figure 2 shows how views, viewpoints, requirements and stakeholders are related.

Figure 2: Dependencies between views, positions, requirements and stakeholders based on the ISO 42010 standard

Due to IMT’s many years of experience in the field of system and software design, the following process has been established to create an architecture description, which is strongly based on the ISO 42010 standard:

• Identifying stakeholders
• Identifying description concerns/requirements
• Defining appropriate points of view (viewpoints)
• Creating views, based on the viewpoints
• Verifying the architecture using scenarios, repeating steps 4 and 5.

The individual steps are described in more detail in the following chapters.

Identifying stakeholders

In order to define the viewpoints in a targeted and, above all, systematic manner, the first step is to identify the stakeholders who work with and for the system, or make decisions about the system. These may include developers, product managers, risk managers, end users, production staff, project managers, etc.

Identifying concerns and requirements

The requirements for the architecture description should be recorded and documented based on the stakeholders identified. In addition to the context interview held with the stakeholders, information on requirements for architecture documentation can also be found in the user or system requirements. Table 1 shows how this might look using Figure 1 as an example.

Table 1: Identifying stakeholders and system-architecture expectations

Definition of appropriate points of view

The viewpoint is characterized by a set of requirements from the corresponding stakeholders, as well as different conventions regarding model types, notations and techniques for the views based on them. Table 2 gives an example of how the viewpoints can be defined. The corresponding notations must be defined in addition to the tabular list. This can be done either in each view separately, or, in terms of the 42010 standard, in a separate viewpoint-specific section.

Table 2: Definition of viewpoints

Viewpoints

A viewpoint is a type of template that can be (re)used across multiple systems and/or is interchangeable among architects. This template is then used to create system-specific views based on it. A well-known collection of viewpoints is the “4+1 view model https://www.cs.ubc.ca/~gregor/teaching/papers/4+1view-architecture.pdf” by Kruchen or the arc42 template , which are used in software systems especially. However, the use of viewpoints is not limited to software architectures—they can be applied to any system.

The definition and choice of viewpoints is primarily determined by the concerns of the identified stakeholders. In the following chapters, we present three possible views, although this set is not applicable to every system.

In general, it is recommended to keep the number of views used to a reasonable minimum, as redundancies increase with each additional view. In addition, each view must be maintained, tracked and, most importantly, kept consistent throughout the course of the project. The resulting effort increases with the number of views. In this case, less is more—provided that the views allow the requirements to be proven.

The logical view

The logical view is often used as the (primary) viewpoint for devices consisting of hardware and software. In the process, the system is broken down into its individual logical parts and arranged. In this view, functional requirements can be assigned to components and possible functional redundancies within the system can be identified. Explicit assignment of system requirements makes sense, as this ensures traceability. Proof of traceability is required, especially for safety-critical systems. This applies not only to system architectures, but also to subsystem and software architectures.

Communication paths and timing sequences/dependencies can also be defined within the logical view, depending on the architectural language. A combination is presented, as logical division and the associated processes are extremely dependent. Alternatively, the processes can be represented in a dedicated view, in which case the design of the logical view and process view is usually repetitive.

The physical view

As with the logical view, different physical viewpoints can expand the description and understanding of the system. Physical views can be, for example, electronic, pneumatic or electromechanical views, which are useful depending on the requirements of the system. A physical view can also make sense for distributed software systems, in which implementing non-functional requirements such as scalability, availability or performance can be demonstrated.

The development view

The development view is often only implicit, but it is also very important for a smooth project workflow. This shows how the system to be developed can be divided into small work packages in order to assign them to individual developers or development teams. Not only can internal requirements such as reusability or the selection of development tools be taken into account, but this view also enables development costs and deadlines to be monitored throughout the course of the project. The development view is often not part of the formal system architecture, but is created as part of project planning.

Verification of architecture

Running scenarios is a popular tool for architecture verification. The most important key scenarios are defined depending on the requirements. A scenario is not a view in its own right, but builds on existing views by linking them. Therefore, a complete architecture description is the basis for verification, or rather, the scenarios can only be run if the architecture has been fully described. Analogous to the description of the logical or physical view, the sequence of the scenario is presented in diagram form. Object interaction diagrams are often used to represent the interaction between logical and/or physical elements (see also Figure 1). Such scenarios are not only used to verify an architecture, but also to help understand the structure of a system.

Summary

Often, different stakeholders are involved who place different demands on a system. In order to address these concerns specifically and to verify that they have been fulfilled, several views are needed to describe an architecture. These views can be derived from generally defined viewpoints. The viewpoints consist of universally applicable conventions and are thus interchangeable and reusable across multiple systems. Finally, an architecture description can be verified with the help of scenarios to demonstrate fulfillment of the most important requirements.

[1] Basic principle (II) of ISO 42010