ISO/IEC/IEEE 29119-1:2022
(Main)Software and systems engineering — Software testing — Part 1: General concepts
Software and systems engineering — Software testing — Part 1: General concepts
This document specifies general concepts in software testing and presents key concepts for the ISO/IEC/IEEE 29119 series.
Ingénierie du logiciel et des systèmes — Essais du logiciel — Partie 1: Concepts généraux
Le présent document définit les concepts généraux relatifs aux tests logiciels et présente les principaux concepts employés dans la série ISO/IEC/IEEE 29119.
General Information
Relations
Buy Standard
Standards Content (Sample)
INTERNATIONAL ISO/
STANDARD IEC/IEEE
29119-1
Second edition
2022-01
Software and systems engineering —
Software testing —
Part 1:
General concepts
Ingénierie du logiciel et des systèmes — Essais du logiciel —
Partie 1: Concepts généraux
Reference number
ISO/IEC/IEEE 29119-1:2022(E)
© ISO/IEC 2022
© IEEE 2022
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2022
© IEEE 2022
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the
respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: +41 22 749 01 11
Fax: +41 22 749 09 47
Email: copyright@iso.org Email: stds.ipr@ieee.org
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
ii
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
Contents Page
Foreword .v
Introduction .vii
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Software testing concepts .16
4.1 Introduction to software testing. 16
4.1.1 Overview . 16
4.1.2 Relationship to quality management . 16
4.1.3 Verification and validation . 16
4.1.4 Test item . 17
4.1.5 Static and dynamic testing . 17
4.1.6 Exhaustive testing and sampling . 18
4.1.7 Testing as a heuristic . 18
4.1.8 Purpose of testing . 18
4.1.9 Test basis . 19
4.1.10 Test oracle . . . 19
4.1.11 Test independence . 19
4.2 Test plans and test strategies . 19
4.2.1 General . 19
4.2.2 Risks and risk management . 20
4.2.3 Risks and requirements as the basis of a test strategy .20
4.2.4 Test approaches . 21
4.2.5 Testing in development and maintenance life cycles .22
4.2.6 Domains and system characteristics . 23
4.2.7 Test strategy contents . 23
4.3 Test frameworks . 24
4.3.1 Test processes . 24
4.3.2 Test documentation .28
4.3.3 Documentation requirements .30
4.3.4 Configuration management and testing .30
4.3.5 Tool support .30
4.3.6 Process improvement and testing .30
4.3.7 Test metrics . . 31
4.4 Test design and execution . 31
4.4.1 Test model . 31
4.4.2 Model-based testing . 32
4.4.3 Scripted and exploratory testing . 33
4.4.4 Test design techniques . 33
4.4.5 Experience-based testing .34
4.4.6 Retesting and regression testing . 35
4.4.7 Manual and automated testing . 35
4.4.8 Continuous testing . 35
4.4.9 Back-to-back testing . 35
4.4.10 A/B testing . 36
4.4.11 Mathematical-based and fuzz testing.36
4.4.12 Test environments .36
4.4.13 Test data management . 37
4.5 Project management and testing . 37
4.6 Communication and reporting . 37
4.7 Defects and incident management .38
Annex A (informative) System characteristics and testing – examples .39
iii
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
Annex B (informative) Testing roles .46
Bibliography .47
IEEE Notices and Abstract.48
iv
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed
for the different types of ISO/IEC documents should be noted. This document was drafted in
accordance with the rules given in the ISO/IEC Directives, Part 2 (see www.iso.org/directives or
www.iec.ch/members_experts/refdocs).
IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating
Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its
standards through a consensus development process, approved by the American National Standards
Institute, which brings together volunteers representing varied viewpoints and interests to achieve the
final product. Volunteers are not necessarily members of the Institute and serve without compensation.
While the IEEE administers the process and establishes rules to promote fairness in the consensus
development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of
the information contained in its standards.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents) or the IEC
list of patent declarations received (see https://patents.iec.ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to
the World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see
www.iso.org/iso/foreword.html. In the IEC, see www.iec.ch/understanding-standards.
ISO/IEC/IEEE 29119-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information
technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and
Software Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards
Development Organization cooperation agreement between ISO and IEEE.
This second edition cancels and replaces the first edition (ISO/IEC/IEEE 29119-1:2013), which has been
technically revised.
The main changes are as follows:
— Testing terms and their definitions that are not covered within this document have been removed.
This has led to this document being renamed from ‘Concepts and definitions’ to ‘General concepts’.
— The coverage of test concepts has been made more concise and re-ordered.
— The concept of test sub-processes has been removed due to its complexity and replaced with
additional coverage of the instantiation of test processes.
— The expected content of a test strategy has been clarified.
v
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
— A simplified test design process is described, with the derivation of test cases now based on test
models rather than on test conditions.
— The coverage of metrics and measures has been moved from an annex into the body of the document.
— The annex explaining how testing fits into different life cycle models has been removed.
— A new annex providing examples of how systems from different domains are associated with certain
characteristics and test approaches has been added.
A list of all parts in the ISO/IEC/IEEE 29119 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards
body. A complete listing of these bodies can be found at www.iso.org/members.html and
www.iec.ch/national-committees.
vi
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
Introduction
The purpose of the ISO/IEC/IEEE 29119 series is to define an internationally agreed set of standards for
software testing that can be used by any organization when performing any form of software testing
and using any life cycle.
It is recognized that there are many different types of software, software organizations, and
methodologies. Software domains include information technology (IT), personal computers (PC),
embedded, mobile, scientific and many other classifications. Software organizations range from
small to large, co-located to world-wide, and commercial to those providing a public service. Software
development methodologies include object-oriented, traditional, agile and DevOps. These and other
factors influence software testing. The ISO/IEC/IEEE 29119 series can support testing in many different
contexts.
This document facilitates the use of other parts in the ISO/IEC/IEEE 29119 series by introducing the
general concepts on which the ISO/IEC/IEEE 29119 series is built.
A general introduction to software testing is provided. The role of software testing in quality
management and as part of verification and validation is described; and its implementation in the form
of both static and dynamic testing is defined. The impracticality of exhaustive testing and the need for
sampling are explained; and the importance of the test basis and test oracle are described. The benefits
of test independence are introduced.
Test plans and test strategies are described in the context of risk-based testing, which is the
recommended approach to strategizing and managing testing that underlies the ISO/IEC/IEEE 29119
series and provides the basis for test prioritization and focus. Test levels, test types and test design
techniques (and corresponding measures) are described in the context of their inclusion as part of the
test strategy.
Various test frameworks are presented, including test processes (and test process improvement), test
metrics, test documentation, configuration management and tool support.
The performance of test design and execution based on the use of a test model is described. Several of
the most important test design and execution choices are considered, including scripted and exploratory
testing approaches, the importance of test design techniques for the creation of test cases, test patterns,
retesting and regression testing, manual and automated testing, back-to-back and A/B testing.
Several activities that directly support test design and executions are introduced, including test
environments, test data management, communications and reporting and defect and incident
management.
Annex A briefly describes a number of system characteristics and suggested associated test approaches.
If a tester can identify which of the system characteristics apply to the system they are testing, then
they should consider whether the specialized testing listed for the characteristic is appropriate for
inclusion in their test strategy.
Annex B introduces several generic testing roles and briefly describes their responsibilities.
The test process model that the ISO/IEC/IEEE 29119 series is based on is defined in detail in
ISO/IEC/IEEE 29119-2. ISO/IEC/IEEE 29119-2 covers the software testing processes at the
organizational level, test management level and for dynamic test levels. Testing is the primary approach
to risk treatment in software development. This document defines a risk-based approach to testing.
Templates and examples of test documentation that are produced during the testing process are defined
in ISO/IEC/IEEE 29119-3. Software testing techniques that can be used during testing are defined in
ISO/IEC/IEEE 29119-4.
While this document is informative, ISO/IEC/IEEE 29119-2, ISO/IEC/IEEE 29119-3 and
ISO/IEC/IEEE 29119-4 are normative, meaning that they include requirements for anyone wanting to
claim conformance to these standards. Users who want to use the standards but have good reasons
vii
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 7 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
for not following every requirement (e.g. for someone following an agile approach to development and
testing) can claim tailored conformance as long as the level of tailoring and its rationale are described
and agreed. Specific details of conformance are provided in the relevant conformance clause in each of
the standards.
The ISO/IEC/IEEE 29119 series can be used in isolation or can be used as part of a larger set of standards
that cover other aspects of the software life cycle. For instance, some users use ISO/IEC/IEEE 12207
to define software system life cycle models appropriate to their products and services (and some
may use the corresponding systems engineering standard, ISO/IEC/IEEE 15288), and reference the
ISO/IEC/IEEE 29119 series for their software testing needs.
Together, the ISO/IEC/IEEE 29119 series aims to provide stakeholders with the ability to manage and
perform software testing in any organization.
viii
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 8 ----------------------
INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-1:2022(E)
Software and systems engineering — Software testing —
Part 1:
General concepts
1 Scope
This document specifies general concepts in software testing and presents key concepts for the
ISO/IEC/IEEE 29119 series.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO, IEC and IEEE maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at https:// www .electropedia .org/
— IEEE Standards Dictionary Online: available at https:// dictionary .ieee .org
NOTE For additional terms and definitions in the field of systems and software engineering, see
ISO/IEC/IEEE 24765, which is published periodically as a “snapshot” of the SEVOCAB (Systems and software
Engineering Vocabulary) and is publicly accessible at https:// www .computer .org/ sevocab.
3.1
A/B testing
split-run testing
statistical testing (3.131) approach that allows testers to determine which of two systems or components
performs better
3.2
accessibility testing
type of usability testing (3.131) used to measure the degree to which a test item (3.107) can be operated
by users with the widest possible range of characteristics and abilities
3.3
activation function
transfer function
formula associated with a node in a neural network (3.50) that determines
the output of the node (activation value (3.4)) from the inputs to the neuron
3.4
activation value
output of an activation function (3.3) of a node in a neural network (3.50)
1
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 9 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
3.5
actual result
set of behaviours or conditions of a test item (3.107), or set of conditions of associated data or the test
environment (3.95), observed as a result of test execution (3.99)
EXAMPLE Outputs to screen, outputs to hardware, changes to data, reports, and communication messages
sent.
3.6
AI-based system
system including one or more components implementing AI (3.7)
3.7
artificial intelligence
AI
capability of an engineered system to acquire, process and apply knowledge and skills
3.8
autonomous system
system capable of working without human intervention for sustained
periods
3.9
autonomy
ability of a system to work for sustained periods without human
intervention
3.10
back-to-back testing
differential testing
approach to testing (3.131) whereby an alternative version of the system is used to generate expected
results (3.35) for comparison from the same test inputs
EXAMPLE The alternative version can be a system that already exists, a system developed by an independent
team or a system implemented using a different programming language.
3.11
base choice
base value
input parameter value that is normally selected based on being a representative or typical value for the
parameter
3.12
boundary value analysis
specification-based test design technique (3.94) based on exercising the boundaries of equivalence
partitions (3.30)
3.13
branch testing
structure-based test case (3.85) design technique based on exercising branches in the control flow
(3.22) of the test item (3.107)
3.14
cause-effect graph
graphical representation of decision rules (3.25) between causes (inputs described as Boolean conditions
(3.21)) and effects (outputs described as Boolean expressions)
3.15
cause-effect graphing
specification-based test design technique (3.94) based on exercising decision rules (3.25) in a cause-effect
graph (3.14)
2
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
3.16
classification
machine learning (3.44) function that predicts the output class for a given
input
3.17
classification tree
hierarchical tree model of the input data to a program in which the inputs are represented by distinct
classifications (relevant test aspects) and classes (input values)
3.18
combinatorial testing
combinatorial test design techniques
class of specification-based test design techniques (3.94) based on exercising combinations of P-V pairs
(3.56)
EXAMPLE Pairwise testing (3.57), base choice (3.11) testing.
3.19
compatibility testing
type of testing (3.131) that measures the degree to which a test item (3.107) can function satisfactorily
alongside other independent products in a shared environment (co-existence), and where necessary,
exchanges information with other systems or components (interoperability)
3.20
completion criteria
conditions under which the testing (3.131) activities are considered complete
3.21
condition
Boolean expression containing no Boolean operators
EXAMPLE “A < B” is a condition but “A and B” is not.
3.22
control flow
sequence in which operations are performed during the execution of a test item (3.107)
3.23
decision
type of statement in which a choice between two or more possible outcomes controls which set of
actions will result
Note 1 to entry: Typical decisions are simple selections (e.g. if-then-else), to decide when to exit loops (e.g. while-
loop), and in case (switch) statements (e.g. case-1–2-3-.-N).
3.24
decision outcome
result of a decision (3.23) that determines the branch to be executed
3.25
decision rule
combination of conditions (3.21) (also known as causes) and actions (also known as effects) that produce
a specific outcome in decision table testing (3.27) and cause-effect graphing (3.15)
3.26
decision table
tabular representation of decision rules (3.25) between causes (inputs described as Boolean conditions
(3.21)) and effects (outputs described as Boolean expressions)
3
© ISO/IEC 2022 – All rights reserved
© IEEE 2022 – All rights reserved
---------------------- Page: 11 ----------------------
ISO/IEC/IEEE 29119-1:2022(E)
3.27
decision table testing
specification-based test design technique (3.94) based on exercising decision rules (3.25) in a decision
table (3.26)
3.28
decision testing
structure-based test case (3.85) design technique based on exercising decision outcomes (3.24) in the
control flow (3.22) of the test item (3.107)
3.29
dynamic testing
testing (3.131) in which a test item (3.107) is evaluated by executing it
3.30
equivalence partition
equivalence class
class of inputs or outputs that are expected to be treated similarly by the test item (3.107)
3.31
equivalence partitioning
test design technique (3.94) in which test cases (3.85) are designed to exercise equivalence partitions
(3.30) by using one or more representative members of each partition
3.32
error guessing
test design technique (3.94) in which test cases (3.85) are derived on the basis of the tester’s knowledge
of past failures, or general knowledge of failure modes
Note 1 to entry: The relevant knowledge can be gained from personal experience, or can be encapsulated in, for
example, a defects database or a “bug taxonomy”.
3.33
executable statement
statement which, when compiled, is translated into object code, which will be executed procedurally
when the test item (3.107) is running and may perform an action on program data
3.34
exhaustive testing
test approach (3.83) in which all combinations of input values and preconditions are tested
Note 1 to entry: In nearly all non-trivial situations, exhaustive testing is impossible, due to the large number of
possible tests.
3.35
expected result
observable predicted behaviour of the test item (3.107) under specified conditions based on its
specification or another source
3.36
experience-based testing
class of test case (3.85) design techniques based on using the experience of testers to generate test
cases
EXAMPLE Error guessing (3.32).
Note 1
...
INTERNATIONAL ISO/IEC/
STANDARD IEEE
29119-1
Second edition
Software and systems engineering —
Software testing —
Part 1:
General concepts
Ingénierie du logiciel et des systèmes — Tests logiciels —
Partie 1: Concepts généraux
PROOF/ÉPREUVE
Reference number
ISO/IEC/IEEE 29119-1:2021(E)
©
ISO/IEC 2021
©
IEEE 2021
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
COPYRIGHT PROTECTED DOCUMENT
© ISO/IEC 2021
© IEEE 2021
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may
be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting
on the internet or an intranet, without prior written permission. Permission can be requested from either ISO or IEEE at the
respective address below or ISO’s member body in the country of the requester.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
CP 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Geneva NY 10016-5997, USA
Phone: +41 22 749 01 11
Email: copyright@iso.org Email: stds.ipr@ieee.org
Website: www.iso.org Website: www.ieee.org
Published in Switzerland
© ISO/IEC 2021 – All rights reserved
ii PROOF/ÉPREUVE © IEEE 2021 – All rights reserved
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
Contents Page
Foreword .v
Introduction .vii
1 Scope . 9
2 Normative references . 9
3 Terms and definitions . 9
4 Software testing concepts .24
4.1 Introduction to software testing .24
4.1.1 Overview .24
4.1.2 Relationship to quality management .24
4.1.3 Verification and validation .24
4.1.4 Test item .25
4.1.5 Static and dynamic testing .25
4.1.6 Exhaustive testing and sampling .26
4.1.7 Testing as a heuristic .26
4.1.8 Purpose of testing .26
4.1.9 Test basis .27
4.1.10 Test oracle .27
4.1.11 Test independence.27
4.2 Test plans and test strategies .27
4.2.1 General.27
4.2.2 Risks and risk management .28
4.2.3 Risks and requirements as the basis of a test strategy .28
4.2.4 Test approaches .29
4.2.5 Testing in development and maintenance life cycles .30
4.2.6 Domains and system characteristics .31
4.2.7 Test strategy contents .31
4.3 Testing frameworks .32
4.3.1 Test processes .32
4.3.2 Test documentation .36
4.3.3 Documentation requirements .38
4.3.4 Configuration management and testing .38
4.3.5 Tool support .38
4.3.6 Process improvement and testing .38
4.3.7 Test metrics .39
4.4 Test design and execution .39
4.4.1 Test model .39
4.4.2 Model-based testing .40
4.4.3 Scripted and exploratory testing .41
4.4.4 Test design techniques.41
4.4.5 Experience-based testing .42
4.4.6 Retesting and regression testing .43
4.4.7 Manual and automated testing .43
4.4.8 Continuous testing .43
4.4.9 Back-to-back testing .43
4.4.10 A/B testing .44
4.4.11 Mathematical-based and fuzz testing .44
4.4.12 Test environments .44
4.4.13 Test data management .45
4.5 Project management and testing .45
4.6 Communication and reporting .45
4.7 Defects and incident management.46
Annex A (informative) System characteristics and testing – examples .47
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved PROOF/ÉPREUVE iii
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
Annex B (informative) Testing roles .54
Bibliography .55
IEEE Notices and Abstract .56
© ISO/IEC 2021 – All rights reserved
iv PROOF/ÉPREUVE © IEEE 2021 – All rights reserved
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are
members of ISO or IEC participate in the development of International Standards through technical
committees established by the respective organization to deal with particular fields of technical
activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
work.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the
different types of ISO/IEC documents should be noted. This document was drafted in accordance with
the rules given in the ISO/IEC Directives, Part 2 (see www .iso .org/ directives or www .iec .ch/ members
_experts/ refdocs).
IEEE Standards documents are developed within the IEEE Societies and the Standards Coordinating
Committees of the IEEE Standards Association (IEEE-SA) Standards Board. The IEEE develops its
standards through a consensus development process, approved by the American National Standards
Institute, which brings together volunteers representing varied viewpoints and interests to achieve the
final product. Volunteers are not necessarily members of the Institute and serve without compensation.
While the IEEE administers the process and establishes rules to promote fairness in the consensus
development process, the IEEE does not independently evaluate, test, or verify the accuracy of any of
the information contained in its standards.
Attention is drawn to the possibility that some of the elements of this document may be the subject
of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent
rights. Details of any patent rights identified during the development of the document will be in the
Introduction and/or on the ISO list of patent declarations received (see www .iso .org/ patents) or the IEC
list of patent declarations received (see https:// patents .iec .ch).
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and
expressions related to conformity assessment, as well as information about ISO's adherence to the
World Trade Organization (WTO) principles in the Technical Barriers to Trade (TBT), see www .iso .org/
iso/ foreword .html. In the IEC, see www .iec .ch/ understanding -standards.
ISO/IEC/IEEE 29119-1 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information
technology, Subcommittee SC 7, Software and systems engineering, in cooperation with the Systems and
Software Engineering Standards Committee of the IEEE Computer Society, under the Partner Standards
Development Organization cooperation agreement between ISO and IEEE.
This second edition cancels and replaces the first edition (ISO/IEC/IEEE 29119-1:2013), which has been
technically revised.
The main changes compared to the previous edition are as follows:
— Testing terms and their definitions that are not covered within this document have been removed.
This has led to this document being renamed from ‘Concepts and definitions’ to ‘General concepts’.
— The coverage of test concepts has been made more concise and re-ordered.
— The concept of test sub-processes has been removed due to its complexity and replaced with
additional coverage of the instantiation of test processes.
— The expected content of a test strategy has been clarified.
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved PROOF/ÉPREUVE v
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
— A simplified test design process is described, with the derivation of test cases now based on test
models rather than on test conditions.
— The coverage of metrics and measures has been moved from an annex into the body of the document.
— The annex explaining how testing fits into different life cycle models has been removed.
— A new annex providing examples of how systems from different domains are associated with certain
characteristics and test approaches has been added.
A list of all parts in the ISO/IEC/IEEE 29119 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www .iso .org/ members .html and www .iec .ch/ national
-committees.
© ISO/IEC 2021 – All rights reserved
vi PROOF/ÉPREUVE © IEEE 2021 – All rights reserved
---------------------- Page: 6 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
Introduction
The purpose of the ISO/IEC/IEEE 29119 series is to define an internationally agreed set of standards for
software testing that can be used by any organization when performing any form of software testing
and using any life cycle.
It is recognized that there are many different types of software, software organizations, and
methodologies. Software domains include information technology (IT), personal computers (PC),
embedded, mobile, scientific and many other classifications. Software organizations range from
small to large, co-located to world-wide, and commercial to those providing a public service. Software
development methodologies include object-oriented, traditional, agile and DevOps. These and other
factors influence software testing. The ISO/IEC/IEEE 29119 series can support testing in many different
contexts.
This document facilitates the use of other parts in the ISO/IEC/IEEE 29119 series by introducing the
general concepts on which the ISO/IEC/IEEE 29119 series is built.
A general introduction to software testing is provided. The role of software testing in quality
management and as part of verification and validation is described; and its implementation in the form
of both static and dynamic testing is defined. The impracticality of exhaustive testing and the need for
sampling are explained; and the importance of the test basis and test oracle are described. The benefits
of test independence are introduced.
Test plans and test strategies are described in the context of risk-based testing, which is the
recommended approach to strategizing and managing testing that underlies the ISO/IEC/IEEE 29119
series and provides the basis for test prioritization and focus. Test levels, test types and test design
techniques (and corresponding measures) are described in the context of their inclusion as part of the
test strategy.
Various test frameworks are presented, including test processes (and test process improvement), test
metrics, test documentation, configuration management and tool support.
The performance of test design and execution based on the use of a test model is described. Several of
the most important test design and execution choices are considered, including scripted and exploratory
testing approaches, the importance of test design techniques for the creation of test cases, test patterns,
retesting and regression testing, manual and automated testing, back-to-back and A/B testing.
Several activities that directly support test design and executions are introduced, including test
environments, test data management, communications and reporting and defect and incident
management.
Annex A briefly describes a number of system characteristics and suggested associated test approaches.
If a tester can identify which of the system characteristics apply to the system they are testing, then
they should consider whether the specialized testing listed for the characteristic is appropriate for
inclusion in their test strategy.
Annex B introduces several generic testing roles and briefly describes their responsibilities.
The test process model that the ISO/IEC/IEEE 29119 series is based on is defined in detail in
ISO/IEC/IEEE 29119-2. ISO/IEC/IEEE 29119-2 covers the software testing processes at the
organizational level, test management level and for dynamic test levels. Testing is the primary approach
to risk treatment in software development. This document defines a risk-based approach to testing.
Templates and examples of test documentation that are produced during the testing process are defined
in ISO/IEC/IEEE 29119-3. Software testing techniques that can be used during testing are defined in
ISO/IEC/IEEE 29119-4.
While this document is informative, ISO/IEC/IEEE 29119-2, ISO/IEC/IEEE 29119-3 and
ISO/IEC/IEEE 29119-4 are normative, meaning that they include requirements for anyone wanting to
claim conformance to these standards. Users who want to use the standards but have good reasons
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved PROOF/ÉPREUVE vii
---------------------- Page: 7 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
for not following every requirement (e.g. for someone following an agile approach to development and
testing) can claim tailored conformance as long as the level of tailoring and its rationale are described
and agreed. Specific details of conformance are provided in the relevant conformance clause in each of
the standards.
The ISO/IEC/IEEE 29119 series can be used in isolation or can be used as part of a larger set of standards
that cover other aspects of the software life cycle. For instance, some users use ISO/IEC/IEEE 12207
to define software system life cycle models appropriate to their products and services (and some
may use the corresponding systems engineering standard, ISO/IEC/IEEE 15288), and reference the
ISO/IEC/IEEE 29119 series for their software testing needs.
Together, the ISO/IEC/IEEE 29119 series aims to provide stakeholders with the ability to manage and
perform software testing in any organization.
© ISO/IEC 2021 – All rights reserved
viii PROOF/ÉPREUVE © IEEE 2021 – All rights reserved
---------------------- Page: 8 ----------------------
INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-1:2021(E)
1 Scope
This document specifies general concepts in software testing and presents key concepts for the
ISO/IEC/IEEE 29119 series.
2 Normative references
There are no normative references in this document.
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO, IEC and IEEE maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https:// www .iso .org/ obp
— IEC Electropedia: available at http:// www .electropedia .org
— IEEE Standards Dictionary Online: available at https:// ieeexplore .ieee .org/ xpls/ dictionary .jsp
NOTE For additional terms and definitions in the field of systems and software engineering, see
ISO/IEC/IEEE 24765, which is published periodically as a “snapshot” of the SEVOCAB (Systems and software
Engineering Vocabulary) and is publicly accessible at www .computer .org/ sevocab. ISO/IEC/IEEE 29119-1
includes all terms that are used in ISO/IEC/IEEE 29119 (all parts).
3.1
A/B testing
split-run testing
statistical testing (3.131) approach that allows testers to determine which of two systems or components
performs better
3.2
accessibility testing
type of usability testing (3.131) used to measure the degree to which a test item (3.107) can be operated
by users with the widest possible range of characteristics and abilities
3.3
activation function
transfer function
formula associated with a node in a neural network (3.50) that determines
the output of the node (activation value (3.4)) from the inputs to the neuron
3.4
activation value
output of an activation function (3.3) of a node in a neural network (3.50)
3.5
actual result
set of behaviours or conditions of a test item (3.107), or set of conditions of associated data or the test
environment (3.95), observed as a result of test execution (3.99)
EXAMPLE Outputs to screen, outputs to hardware, changes to data, reports, and communication messages
sent.
3.6
AI-based system
system including one or more components implementing AI (3.7)
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved PROOF/ÉPREUVE 9
---------------------- Page: 9 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
3.7
artificial intelligence
AI
capability of an engineered system to acquire, process and apply knowledge and skills
3.8
autonomous system
system capable of working without human intervention for sustained
periods
3.9
autonomy
ability of a system to work for sustained periods without human
intervention
3.10
back-to-back testing
differential testing
approach to testing (3.131) whereby an alternative version of the system is used to generate expected
results (3.35) for comparison from the same test inputs
EXAMPLE The alternative version can be a system that already exists, a system developed by an independent
team or a system implemented using a different programming language.
3.11
base choice
base value
input parameter value that is normally selected based on being a representative or typical value for the
parameter
3.12
boundary value analysis
specification-based test design technique (3.94) based on exercising the boundaries of equivalence
partitions (3.30)
3.13
branch testing
structure-based test case (3.85) design technique based on exercising branches in the control flow
(3.22) of the test item (3.107)
3.14
cause-effect graph
graphical representation of decision rules (3.25) between causes (inputs described as Boolean conditions
(3.21)) and effects (outputs described as Boolean expressions)
3.15
cause-effect graphing
specification-based test design technique (3.94) based on exercising decision rules (3.25) in a cause-effect
graph (3.14)
3.16
classification
machine learning (3.44) function that predicts the output class for a given
input
3.17
classification tree
hierarchical tree model of the input data to a program in which the inputs are represented by distinct
classifications (relevant test aspects) and classes (input values)
© ISO/IEC 2021 – All rights reserved
10 PROOF/ÉPREUVE © IEEE 2021 – All rights reserved
---------------------- Page: 10 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
3.18
combinatorial testing
combinatorial test design techniques
class of specification-based test design techniques (3.94) based on exercising combinations of P-V pairs
(3.56)
EXAMPLE Pairwise testing (3.57), base choice (3.11) testing.
3.19
compatibility testing
type of testing (3.131) that measures the degree to which a test item (3.107) can function satisfactorily
alongside other independent products in a shared environment (co-existence), and where necessary,
exchanges information with other systems or components (interoperability)
3.20
completion criteria
conditions under which the testing (3.131) activities are considered complete
3.21
condition
Boolean expression containing no Boolean operators
EXAMPLE “A < B” is a condition but “A and B” is not.
3.22
control flow
sequence in which operations are performed during the execution of a test item (3.107)
3.23
decision
type of statement in which a choice between two or more possible outcomes controls which set of
actions will result
Note 1 to entry: Typical decisions are simple selections (e.g. if-then-else), to decide when to exit loops (e.g. while-
loop), and in case (switch) statements (e.g. case-1–2-3-.-N).
3.24
decision outcome
result of a decision (3.23) that determines the branch to be executed
3.25
decision rule
combination of conditions (3.21) (also known as causes) and actions (also known as effects) that produce
a specific outcome in decision table testing (3.27) and cause-effect graphing (3.15)
3.26
decision table
tabular representation of decision rules (3.25) between causes (inputs described as Boolean conditions
(3.21)) and effects (outputs described as Boolean expressions)
3.27
decision table testing
specification-based test design technique (3.94) based on exercising decision rules (3.25) in a decision
table (3.26)
3.28
decision testing
structure-based test case (3.85) design technique based on exercising decision outcomes (3.24) in the
control flow (3.22) of the test item (3.107)
© ISO/IEC 2021 – All rights reserved
© IEEE 2021 – All rights reserved PROOF/ÉPREUVE 11
---------------------- Page: 11 ----------------------
ISO/IEC/IEEE 29119-1:2021(E)
3.29
dynamic testing
testing (3.131) in which a test item (3.107) is evaluated by executing it
3.30
equivalence partition
equivalence class
class of inputs or outputs that are expected to be treated similarly by the test item (3.107)
3.31
equivalence partitioning
test design technique (3.94) in which test cases (3.85) are designed to exercise equivalence partitions
(3.30) by using one or more representative members of each partition
3.32
error guessing
test design technique (3.94) in which test cases (3.85) are derived on the basis of the tester’s knowledge
of past failures, or general knowledge of failure modes
Note 1 to entry: The relevant knowledge can be gained from personal experience, or can be encapsulated in, for
example, a defects database or a “bug taxonomy”.
3.33
executable statement
statement which, when compiled, is translated into object code, which will be executed procedurally
when the test item (3.107) is running and may perform an action on program data
3.34
exhaustive testing
test approach (3.83) in which all combinations of input values and preconditions are tested
Note 1 to entry: In nearly all non-trivial situations, exhaustive testing is impossible, due to the large number of
possible tests.
3.35
expected result
observable predicted behaviour of the test item (3.107) under specified conditions based on its
specification or another source
3.36
experience-based testing
class of test case (3.85) design techniques based on using the
...
NORME ISO/
INTERNATIONALE IEC/IEEE
29119-1
Deuxième édition
2022-01
Ingénierie du logiciel et des
systèmes — Essais du logiciel —
Partie 1:
Concepts généraux
Software and systems engineering — Software testing —
Part 1: General concepts
Numéro de référence
ISO/IEC/IEEE 29119-1:2022(F)
© ISO/IEC 2022
© IEEE 2022
---------------------- Page: 1 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
DOCUMENT PROTÉGÉ PAR COPYRIGHT
© ISO/IEC 2022
© IEEE 2022
Tous droits réservés. Sauf prescription différente ou nécessité dans le contexte de la mise en œuvre, aucune partie de cette
publication ne peut être reproduite ni utilisée sous quelque forme que ce soit et par aucun procédé, électronique ou mécanique, y
compris la photocopie, ou la diffusion sur l’internet ou un intranet, sans autorisation écrite soit de l’ISO soit de l’IEEE, à l’une ou
l’autre des adresses ci-après.
ISO copyright office Institute of Electrical and Electronics Engineers, Inc
Case postale 401 • Ch. de Blandonnet 8 3 Park Avenue, New York
CH-1214 Vernier, Genève NY 10016-5997, USA
Tél.: +41 22 749 01 11
Fax: +41 22 749 09 47
E-mail: copyright@iso.org E-mail: stds.ipr@ieee.org
Web: www.iso.org Web: www.ieee.org
Publié en Suisse
ii
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 2 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
Sommaire Page
Avant-propos .v
Introduction .vii
1 Domaine d'application .1
2 Références normatives .1
3 Termes et définitions . 1
4 Concepts des tests logiciels .17
4.1 Introduction aux tests logiciels . 17
4.1.1 Vue d'ensemble. 17
4.1.2 Relation avec le management de la qualité . 17
4.1.3 Vérification et validation . 17
4.1.4 Élément de test . 18
4.1.5 Tests statiques et dynamiques . 18
4.1.6 Tests exhaustifs et échantillonnage . 19
4.1.7 Tests heuristiques . 19
4.1.8 Finalité du test . 19
4.1.9 Base de test. 20
4.1.10 Oracle de test . 20
4.1.11 Indépendance des tests . 20
4.2 Plans de test et stratégies de test . 21
4.2.1 Généralités . 21
4.2.2 Risques et gestion du risque . 21
4.2.3 Risques et exigences comme base d'une stratégie de test .22
4.2.4 Approches de test . 22
4.2.5 Tests dans les cycles de vie du développement et de la maintenance .23
4.2.6 Domaines et caractéristiques du système . 24
4.2.7 Contenu de la stratégie de test . 25
4.3 Frameworks de test.26
4.3.1 Processus de test . 26
4.3.2 Documentation des tests .30
4.3.3 Exigences relatives à la documentation . 32
4.3.4 Gestion de la configuration et tests . 32
4.3.5 Support outillé .33
4.3.6 Amélioration des processus et tests . 33
4.3.7 Métriques de test.34
4.4 Conception et exécution des tests .34
4.4.1 Modèle de test .34
4.4.2 Tests basés sur un modèle . 35
4.4.3 Tests avec script et tests exploratoires .36
4.4.4 Techniques de conception de tests .36
4.4.5 Tests basés sur l'expérience . 37
4.4.6 Re-tests et tests de régression .38
4.4.7 Tests manuels et automatisés .38
4.4.8 Tests en continu .39
4.4.9 Tests dos-à-dos . .39
4.4.10 Tests A/B . 39
4.4.11 Tests mathématiques et tests à données aléatoires.39
4.4.12 Environnements de test .40
4.4.13 Gestion des données de test .40
4.5 Gestion de projet et tests .40
4.6 Communication et génération de rapports . 41
4.7 Gestion des défauts et des incidents . 41
Annexe A (informative) Caractéristiques du système et tests — Exemples .42
iii
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 3 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
Annexe B (informative) Rôles de test .50
Bibliographie .51
Notes et résumés IEEE .52
iv
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 4 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
Avant-propos
L'ISO (Organisation internationale de normalisation) et l’IEC (Commission électrotechnique
internationale) forment le système spécialisé de la normalisation mondiale. Les organismes
nationaux membres de l'ISO ou de l’IEC participent au développement de Normes internationales
par l'intermédiaire des comités techniques créés par l'organisation concernée afin de s'occuper des
domaines particuliers de l'activité technique. Les comités techniques de l'ISO et de l’IEC collaborent
dans des domaines d'intérêt commun. D'autres organisations internationales, gouvernementales et non
gouvernementales, en liaison avec l'ISO et l’IEC, participent également aux travaux.
Les procédures utilisées pour élaborer le présent document et celles destinées à sa mise à jour sont
décrites dans les Directives ISO/IEC, Partie 1. Il convient, en particulier de prendre note des différents
critères d'approbation requis pour les différents types de documents ISO/IEC. Le présent document a
été rédigé conformément aux règles de rédaction données dans les Directives ISO/IEC, Partie 2 (voir
www.iso.org/directives ou www.iec.ch/members_experts/refdocs).
Les documents normatifs de l’IEEE sont développés au sein des sociétés de l’IEEE et des Comités de
Coordination des Normes du Conseil des Normes de l’Association des normes IEEE (IEEE-SA). L’IEEE
élabore ses normes par le biais d’un processus d'élaboration du consensus approuvé par l’American
National Standards Institute, qui rassemble des volontaires représentant divers points de vue et divers
intérêts pour parvenir au produit final. Les volontaires ne sont pas nécessairement des membres
de l’Institut et aucune compensation ne leur est attribuée. Bien que l’IEEE administre le processus
et établisse des règles pour favoriser l’équité au cours du processus d'élaboration du consensus,
l’IEEE n’évalue pas, ne soumet pas à essai ou ne vérifie pas de manière indépendante l’exactitude des
informations contenues dans ses normes.
L'attention est attirée sur le fait que certains des éléments du présent document peuvent faire l'objet
de droits de propriété intellectuelle ou de droits analogues. L'ISO et l’IEC ne sauraient être tenues pour
responsables de ne pas avoir identifié de tels droits de propriété et averti de leur existence. Les détails
concernant les références aux droits de propriété intellectuelle ou autres droits analogues identifiés
lors de l'élaboration du document sont indiqués dans l'Introduction et/ou dans la liste des déclarations
de brevets reçues par l'ISO (voir www.iso.org/brevets) ou dans la liste des déclarations de brevets
reçues par l'IEC (voir https://patents.iec.ch).
Les appellations commerciales éventuellement mentionnées dans le présent document sont données
pour information, par souci de commodité, à l’intention des utilisateurs et ne sauraient constituer un
engagement.
Pour une explication de la nature volontaire des normes, la signification des termes et expressions
spécifiques de l'ISO liés à l'évaluation de la conformité, ou pour toute information au sujet de
l'adhésion de l'ISO aux principes de l’Organisation mondiale du commerce (OMC) concernant les
obstacles techniques au commerce (OTC), voir www.iso.org/iso/avant-propos. Pour l'IEC, voir
www.iec.ch/understanding-standards.
ISO/IEC/IEEE 29119-1 a été élaborée par le comité technique mixte ISO/IEC JTC 1, Technologies de
l'information, sous-comité SC 7, Ingénierie du logiciel et des systèmes, en coopération avec le Systems and
Software Engineering Standards Committee de l'IEEE Computer Society, dans le cadre de l'accord de
coopération entre les Organisations Partenaires de Développement de Normes que sont l'ISO et l'IEEE.
Cette deuxième édition annule et remplace la première édition (ISO/IEC/IEEE 29119-1:2013), qui a fait
l'objet d'une révision technique.
Les principales modifications sont les suivantes:
— les termes relatifs aux tests et leurs définitions non couverts dans le présent document ont été
supprimés. Le présent document intitulé «Concepts et définitions» a donc été renommé «Concepts
généraux»;
— la couverture des concepts de test a été réduite et réorganisée;
v
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 5 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
— le concept de sous-processus de test a été supprimé en raison de sa complexité et remplacé par la
couverture supplémentaire d'instanciation des processus de test;
— le contenu attendu de la stratégie de test a été clarifié;
— un processus de conception simplifié des tests est décrit, l'origine des cas de test étant désormais
basée sur les modèles de test plutôt que sur les conditions de test;
— la couverture des indicateurs et des mesures, qui se trouvait dans une annexe, a été déplacée dans
le corps du document;
— l'annexe expliquant de quelle manière les tests s'adaptent aux différents modèles de cycle de vie a
été supprimée;
— une nouvelle annexe fournissant des exemples de la manière dont les systèmes issus de différents
domaines sont associés à certaines caractéristiques et approches de test a été ajoutée.
Une liste de toutes les parties de la série ISO/IEC/IEEE 29119 se trouve sur les sites Web de l'ISO et de
l'IEC.
Il convient que l’utilisateur adresse tout retour d’information ou toute question concernant le présent
document à l’organisme national de normalisation de son pays. Une liste exhaustive desdits organismes
se trouve à l’adresse www.iso.org/fr/members.html et www.iec.ch/national-committees.
vi
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 6 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
Introduction
L'objet de la série ISO/IEC/IEEE 29119 est de définir un ensemble de normes de tests logiciels
approuvées au plan national pouvant être utilisées par toute organisation qui entreprend toute forme
de test logiciel, quel que soit le cycle de vie.
Il est indéniable qu'il existe un grand nombre de logiciels, d'organisations logicielles et de types de
méthodologies différents. Les logiciels couvrent divers domaines, notamment les technologies de
l'information (TI), les ordinateurs personnels (PC), les technologies intégrées, mobiles et scientifiques,
ainsi que de nombreuses autres classifications. Les organisations logicielles vont des petites aux
grandes entreprises, des entreprises colocalisées aux multinationales, et des entreprises commerciales
aux services publics. Les méthodologies de développement logiciel sont tantôt orientées objets, tantôt
traditionnelles, tantôt en Agile, tantôt axées sur le DevOps. Tous ces facteurs, combinés à d'autres,
influencent les tests logiciels. La série ISO/IEC/IEEE 29119 peut faciliter la conduite de tests dans de
nombreux contextes différents.
Le présent document aide à l'utilisation des autres parties de la série ISO/IEC/IEEE 29119 en
introduisant les concepts généraux sur lesquels la série ISO/IEC/IEEE 29119 est bâtie.
Il fournit une introduction générale à la notion de tests logiciels. Il décrit le rôle des tests logiciels dans
le management de la qualité et dans le cadre des processus de vérification et de validation, et définit
leur mise en œuvre sous forme de tests statiques et dynamiques. Il explique, par ailleurs, la difficulté de
réaliser des tests exhaustifs et la nécessité de procéder par échantillonnage, et souligne l'importance de
la base de test et de l'oracle de test. Il décrit en outre les avantages de l'indépendance des tests.
Les plans de test et les stratégies de test sont décrits dans le contexte de tests basés sur les risques,
ce qui constitue l'approche recommandée pour l'élaboration d'une stratégie et la gestion des tests,
approche sous-jacente à la série ISO/IEC/IEEE 29119 et fournissant une base pour définir les priorités
et les objectifs de test. Les niveaux de test, les types de test et les techniques de conception de tests
(ainsi que les mesures correspondantes) sont décrits en tant que partie intégrante de la stratégie de
test.
Différents frameworks de test sont présentés, notamment les processus de test (et l'amélioration de ces
processus), les métriques de test, la documentation de test, la gestion de la configuration ainsi que le
support outillé.
Cette norme décrit également la conception et l'exécution des tests à partir d'un modèle de test. Elle
évalue plusieurs choix de conception et d'exécution de tests parmi les plus importants, notamment
les approches de tests basés sur des scripts et les approches de tests exploratoires, l'importance des
techniques de conception de tests pour la création de cas de test, les patterns de test, les re-tests et les
tests de régression, les tests manuels et automatisés, les tests dos-à-dos (back-to-back) et les tests A/B.
Elle présente plusieurs activités qui soutiennent directement la conception et l'exécution des tests,
notamment les environnements de test, la gestion des données de test, la communication et la
production de rapports, et la gestion des défauts et incidents.
L'Annexe A décrit brièvement certaines caractéristiques du système et suggère les approches de test
associées. Si un testeur peut identifier les caractéristiques du système qui s'appliquent au système qu'il
est en train de tester, il convient alors qu'il détermine s'il est utile d'inclure dans sa stratégie de test les
tests spécialisés répertoriés pour les caractéristiques en question.
L'Annexe B introduit plusieurs rôles de test génériques et en décrit brièvement les responsabilités.
Le modèle de processus de test sur lequel est fondée la série de normes ISO/IEC/IEEE 29119 est défini
en détail dans l'ISO/IEC/IEEE 29119-2. L'ISO/IEC/IEEE 29119-2 traite des processus de test logiciel au
niveau organisationnel, au niveau de la gestion des tests et au niveau des tests dynamiques. Le test
forme l'approche essentielle du traitement des risques dans le développement de logiciels. Le présent
document définit une approche des tests basée sur le risque.
vii
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 7 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
L'ISO/IEC/IEEE 29119-3 définit des modèles et des exemples de documentation de test produite au
cours du processus de test. Les techniques de test pouvant être utilisées au cours des tests sont définies
dans l'ISO/IEC/IEEE 29119-4.
Bien que le présent document soit informatif, l'ISO/IEC/IEEE 29119-2, l'ISO/IEC/IEEE 29119-3 et
l'ISO/IEC/IEEE 29119-4 sont normatives, ce qui signifie qu'elles stipulent des exigences obligatoires
pour quiconque souhaiterait revendiquer la conformité à ces normes. Les utilisateurs qui souhaitent
utiliser les normes, mais qui ont de bonnes raisons de ne pas en suivre chaque exigence (par exemple,
quelqu'un qui suit une approche de développement et de test en Agile) peuvent revendiquer une
conformité adaptée, à condition que le niveau d'adaptation et sa justification soient décrits et acceptés.
Les détails de conformité spécifiques sont donnés dans l'article de chaque norme applicable à la
conformité.
La série ISO/IEC/IEEE 29119 peut être utilisée de manière isolée ou dans le cadre d'un ensemble
plus vaste de normes qui couvrent d'autres aspects du cycle de vie du logiciel. Par exemple,
certains utilisateurs utiliseront l'ISO/IEC/IEEE 12207 pour définir des modèles de cycle de vie du
système logiciel qui s'appliquent à leurs produits et services (et d'autres peuvent utiliser la norme
correspondante ISO/IEC/IEEE 15288 relative à l'ingénierie des systèmes), et se référer à la série de
normes ISO/IEC/IEEE 29119 pour leurs besoins en matière de tests logiciels.
Collectivement, la série ISO/IEC/IEEE 29119 vise à donner aux parties prenantes la possibilité de gérer
et d'exécuter des tests logiciels dans n'importe quelle organisation.
viii
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 8 ----------------------
NORME INTERNATIONALE ISO/IEC/IEEE 29119-1:2022(F)
Ingénierie du logiciel et des systèmes — Essais du
logiciel —
Partie 1:
Concepts généraux
1 Domaine d'application
Le présent document définit les concepts généraux relatifs aux tests logiciels et présente les principaux
concepts employés dans la série ISO/IEC/IEEE 29119.
2 Références normatives
Le présent document ne contient aucune référence normative.
3 Termes et définitions
Pour les besoins du présent document, les termes et définitions suivants s'appliquent.
L'ISO, l'IEC et l'IEEE tiennent à jour des bases de données terminologiques destinées à être utilisées en
normalisation, consultables aux adresses suivantes:
— ISO Online browsing platform: disponible à l’adresse https:// www .iso .org/ obp
— IEC Electropedia: disponible à l’adresse https:// www .electropedia .org/
— IEEE Standards Dictionary Online: disponible à l'adresse https:// dictionary .ieee .org
NOTE Pour d'autres termes et définitions relevant du domaine de l'ingénierie des systèmes et des logiciels,
voir l'ISO/IEC/IEEE 24765, qui est régulièrement publiée sous la forme d'un «instantané» du SEVOCAB (Systems
and software Engineering Vocabulary) et est accessible publiquement à l'adresse https:// www .computer .org/
sevocab.
3.1
test A/B
test dédoublé
approche de tests (3.131) statistique qui permet aux testeurs d'identifier parmi deux systèmes ou
composants celui qui présente les meilleures performances
3.2
test d'accessibilité
type de tests (3.131) d'utilisabilité servant à mesurer le degré selon lequel des utilisateurs présentant
la plus grande diversité possible de caractéristiques et d'aptitudes peuvent faire fonctionner un
élément de test (3.107)
3.3
fonction d'activation
fonction de transfert
formule associée à un nœud dans un réseau neuronal (3.50) qui détermine
la sortie du nœud (valeur d'activation [3.4]) à partir des entrées du neurone
1
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 9 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
3.4
valeur d'activation
sortie d'une fonction d'activation (3.3) d'un nœud dans un réseau neuronal
(3.50)
3.5
résultat réel
ensemble de comportements ou de conditions d'un élément de test (3.107), ou ensemble de conditions
des données associées ou de l'environnement de test (3.95), observés à la suite de l'exécution du test (3.99)
EXEMPLE Sorties sur un écran, sorties matérielles, modifications de données, rapports et messages de
communication envoyés.
3.6
système basé sur l'IA
système comprenant un ou plusieurs composants qui met en œuvre l'IA (3.7)
3.7
intelligence artificielle
IA
capacité d'un système d'ingénierie à acquérir, traiter et appliquer des connaissances et des aptitudes
3.8
système autonome
système capable de fonctionner pendant des périodes prolongées sans
intervention humaine
3.9
autonomie
aptitude d'un système à fonctionner pendant des périodes prolongées
sans intervention humaine
3.10
test dos-à-dos
test différentiel
approche de tests (3.131) dans laquelle une version alternative du système est utilisée pour générer des
résultats attendus (3.35) en vue de les comparer avec les mêmes entrées utilisées pour le test
EXEMPLE La version alternative peut être un système qui existe déjà, un système développé par une équipe
indépendante ou un système mis en œuvre à l'aide d'un langage de programmation différent.
3.11
choix de base
valeur de base
valeur de paramètre d'entrée qui est normalement sélectionnée pour sa nature représentative ou
typique du paramètre en question
3.12
analyse des valeurs limites
technique de conception de tests (3.94) basée sur une spécification, qui consiste à exercer les valeurs aux
limites de partitions d'équivalence (3.30)
3.13
test de branches
technique de conception d'un cas de test (3.85) basée sur une structure, qui consiste à exercer des
branches du flux de contrôle (3.22) de l'élément de test (3.107)
3.14
graphe de causes à effets
représentation graphique des règles de décision (3.25) entre des causes (entrées décrites sous forme de
conditions [3.21] booléennes) et des effets (sorties décrites sous forme d'expressions booléennes)
2
© ISO/IEC 2022 – Tous droits réservés
© IEEE 2022 – Tous droits réservés
---------------------- Page: 10 ----------------------
ISO/IEC/IEEE 29119-1:2022(F)
3.15
mise en graphe de causes à effets
technique de conception de tests (3.94) basée sur une spécification, qui consiste à exercer des règles de
décision (3.25) dans un graphe de causes à effets (3.14)
3.16
classification
fonction d'apprentissage automatique (3.44) qui prédit la classe de sortie
d'une entrée donnée
3.17
classification arborescente
modèle d'arborescence hiérarchique des données d'entrée d'un programme dans lequel les sorties sont
représentées par des classifications (aspects pertinents pour les tests) et des classes (valeurs d'entrée)
distinctes
3.18
tests combinatoires
techniques de conception de tests combinatoires
catégorie de technique de conception de tests (3.94) basées sur une spécification, qui consistent à exercer
des combinaisons de paires P-V (3.56)
EXEMPLE Test par paires (3.57), tests par choix de base (3.11).
3.19
test de compatibilité
type de tests (3.131) qui évalue dans quelle mesure un élément de test (3.107) peut fonctionner de
manière satisfaisante parallèlement à d'autres produits indépendants dans un environnement partagé
(coexistence) et, si nécessaire, échanger des informations avec d'autres systèmes ou composants
(interopérabilité)
3.20
critères de clôture
conditions dans lesquelles les activités de tests (3.131) sont considérées comme clôturées
3.21
condition
expression booléenne ne contenant aucun opérateur booléen
EXEMPLE «A < B» est une condition, contrairement à «A and B».
3.22
flux de contrôle
ordre de déroulement des opérations au cours de l'exécution d'un élément de test (3.107)
3.23
décision
type d'instruction dans lequel un choix entre deux résultats possibles ou plus contrôle l'ensemble des
actions qui en résultera
Note 1 à l'article: Les décisions typiques prennent la forme de sélections simples (par exemple, if-then-else)
permettant de décider à quel moment sortir de
...
Questions, Comments and Discussion
Ask us and Technical Secretary will try to provide an answer. You can facilitate discussion about the standard in here.