Real-Time Systems. Design Principles for Distributed Embedded Applications. Herman Kopetz. Second Edition (811374), страница 85
Текст из файла (страница 85)
The following techniquesimprove the testability:1. Partitioning the system into composable subsystems with observable and in thedomains of value and time well-specified interfaces. It is then possible to testeach subsystem in isolation and to limit the integration effects to the testing ofthe emerging behavior. Probe-effect free observability of subsystem outputs atany level of the architecture was one of the motivations to provide multicastingin the basic message transport service (see Sect. 4.3) of the time-triggeredarchitecture.29412 Validation2. Establishment of a static temporal control structure such that the temporalcontrol structure is independent of the input data. It is then possible to test thetemporal control structure in isolation.3.
Reducing the temporal dimension of the input space by introducing a sparsetime-base of proper granularity. The granularity of this time-base should besufficient for the application at hand but should not be any smaller. The smallerthe granularity of the sparse time-base, the larger the potential input space in thetemporal domain. By decreasing the size of the input space or by increasingthe number of non-redundant test cases the test coverage, i.e., the fraction of thetotal input space that is covered by the tests, can be improved.4. Publication of the ground state of a node in a g-state message at the periodicreintegration point. The ground state can then be observed by an independentcomponent without probe effect.5. Provision of determinism in the software such that the same output messageswill be produced if the same input messages are applied to a component.Because of their deterministic properties and their static control structure, timetriggered systems are easier to test than event-triggered systems.12.2.2 Test Data SelectionDuring the test phase, only a tiny fraction of the potential input space of a computersystem can be exercised.
The challenge for the tester is to find an effective andrepresentative test-data set that will give the designer confidence that the systemwill work correctly for all inputs. In this section, we present some methods for testdata selection:Random Test Data Selection. Test data are selected randomly without any consideration of the program structure or the operational profile of use.Requirements Coverage. In this method, the requirements specification is thestarting point for selecting the test data.
For each one of the given requirements, aset of test cases is designed to check whether the requirement is satisfied. Thehidden assumption in this criterion is that the set of requirements is complete.White-box Testing. The internal structure of a system is examined to derive a set oftest data such that some kind of coverage criterion is satisfied, e.g., that allstatements have been executed or that all branches of a program have been tested.This test data selection criterion is most effective for unit testing, where theinternals of the component implementation are available.Model-based Test Data Selection: The test data is derived from a model of thesystem under test and a model of the physical plant.
Model-based test data selectioncan be automated, since the correctness of test results can be related to a performance criterion of the physical process.12.2 Testing Challenges295Example: Consider the case where a controller of an automotive engine is tested versus amodel of this engine. The model of the engine has been extensively validated with respect tothe operation of the real engine and is assumed to be correct. The control algorithms that areimplemented in the controller determine the performance parameters of the engine suchas energy efficiency, torque, pollution, etc. By observing the performance parameters ofthe engine we can detect anomalies that are caused by a misbehavior of the controllersoftware.Operational profile.
The basis of the test data selection is the operational profile ofthe system under test in the given application context. This test data selectioncriterion misses rare events.Peak load. A hard real-time system must provide the specified timely service underall conditions covered by the load- and fault-hypothesis, i.e., also under peak loadsthat are caused by rare events. The peak-load scenario puts extreme stress on thesystem and should be tested extensively.
The behavior of the system in above-peakload situations must also be tested. If peak load activity is handled correctly, thenormal load case will take care of itself. In most cases it is not possible to generaterare events and peak load in the real-world operational environment. Thereforepeak-load testing is best performed in a model-based test environment.Worst-Case Execution Time (WCET).
To determine the WCET of a task experimentally, the task source code can be analyzed to generate a test data set that isbiased towards the worst-case execution time.Fault-Tolerance Mechanisms. Testing the correctness of the fault-tolerance mechanism is difficult, because faults are not part of the normal input domain.
Mechanisms must be provided that can activate the faults during the test phase. Forexample, software- or hardware-implemented fault injection can be used to testthe correctness of the fault-tolerance mechanisms (see Sect. 12.5).Cyclic systems. If a system has a cyclic behavior (many control systems are cyclic),the crossing of a particular phase of the cycle is a repetitive event in the temporaldomain. In many cyclic systems it is sufficient to test all events that occur in a singlecycle.The above list of test data selection criteria is not complete. A survey study onthe effectiveness of different test data selection criteria is contained in Juristo et al.[Jur04].
Selecting the test data by using a combination of the above criteria seems tobe more effective than relying on a single criterion in isolation.In order to be able to judge the quality of a test-data set, different coveragemeasures have been introduced. A coverage measure describes the degree to whicha test data set exercises the SUT. Common coverage criteria are:1.2.3.4.5.Function coverage – has every function been exercised?Statement coverage – has every statement of the source code been executed?Branch coverage – has every branch instruction been executed in all directions?Condition coverage – has every Boolean condition been fully exercised?Fault coverage – have the fault-tolerant mechanisms been tested for every faultthat is contained in the fault hypothesis?29612 Validation12.2.3 Test OracleGiven that a set of test cases has been selected, a method must be provided todetermine whether the result of a test case produced by the SUT is acceptable ornot.
In the literature, the term test oracle is used to refer to such a method. Thedesign of an algorithmic test oracle is a prerequisite of any test automation, whichis needed for reducing the cost of testing.In practice, the judgment whether the result of a test case is in conformance with a naturallanguage representation of the model of the user’s intention is often delegated to a human.Model-based design and model-based testing can help to partially solve the problem.The structured design process, discussed in Sect.
11.2, distinguishes between thePIM (Platform-Independent-Model) and the PSM (Platform-Specific Model) of acomponent. An executable representation of the complete interface behavior (in thedomains of value and time) at the PIM level of a design can act as the reference forthe adjudication of a test result at the PSM level and thus help to detect implementation errors. The oracle challenge is thus shifted from the PSM level to the PIM level.Since the PIM is developed in an early phase of the design, errors can be capturedearly in the lifecycle, which reduces the cost associated with correction of the errors.The LIF specification of a component (see Sect.
4.4.2) should contain inputassertions and output assertions. Input assertions limit the input space of thecomponent and exclude input data that the component is not designed to handle.Output assertions help to immediately detect errors that occur inside a component.Both input assertions and output assertions can be considered to act as a test-oraclelight [Bar01].
Since the PIM is not resource constrained, the wide use of inputassertions and output assertions at the PIM level can help to debug the PIM specification. In the second phase, when the PIM is transformed to the PSM, some of theseassertions can be removed to arrive at an efficient code for the target machine.In a time-triggered system, where the temporal control structure is static, thedetection of temporal errors in the execution of a program at the PSM level isstraightforward and can be automated.12.2.4 System EvolutionMost successful systems evolve over time.
Existing deficiencies are corrected andnew functions are introduced in new versions of the system. The validation of thesenew versions must be concerned with two issues:1. Regression testing. Checking that the functionality of the previous version (thatmust be supported in the new version) has not been modified and is still correct.2. New-function testing. Checking that the functionality that is new to the latestversion is implemented correctly.Regression testing can be automated by executing the test-data set of the previousversion on the new version.
The anomalies that have been detected in the old12.3 Testing of Component-Based Systems297version (see Sect. 6.3) can be the source of new test cases that put extraordinarystress on the system. Bertolino [Ber07] lists six questions that have to be addressedwhen designing a specific test campaign:1. WHY do we perform the test? Is the objective of the test campaign to findresidual design errors, to establish the reliability of a product before it can bereleased or to find out whether the system has a usable man–machine interface?2.
![](https://s.studizba.com/z.php?f=/templates/si/i/noavatar.png&w=100&h=100&t=1"){kind=avatar}
