Effort and Quality of Recovering Requirements-to-Code Traces: Two Exploratory Experiments

Egyed, Alexander; Graf, Florian; GrÃ¼nbacher, Paul

doi:10.1109/RE.2010.34

Effort and Quality of Recovering Requirements-to-Code Traces: Two Exploratory Experiments

by Alexander Egyed, Florian Graf, Paul GrÃ¼nbacher

Abstract:

Trace links between requirements and code are essential for many software development and maintenance activities. Despite significant advances in traceability research, creating links remains a human-intensive activity and surprisingly little is known about how humans perform basic tracing tasks. We investigate fundamental research questions regarding the effort and quality of recovering traces between requirements and code. Our paper presents two exploratory experiments conducted with 100 subjects who recovered trace links for two open source software systems in a controlled environment. In the first experiment, subjects recovered trace links between the two systems' requirements and classes of the implementation. In the second experiment, trace links were established between requirements and individual methods of the implementation. In order to assess the validity of the trace links cast by subjects, key developers of the two software systems participated in our research and provided benchmarks. Our study yields surprising observations: trace capture is surprisingly fast and can be done within minutes even for larger classes; the quality of the captured trace links, while good, does not improve with higher trace effort; and it is not harder though slightly more expensive to recover the trace links for larger, more complex classes.

Reference:

Alexander Egyed, Florian Graf, Paul GrÃ¼nbacher, "Effort and Quality of Recovering Requirements-to-Code Traces: Two Exploratory Experiments", IEEE Computer Society, pp. 221-230, 2010.

Bibtex Entry:

@Conference{DBLP:conf/re/EgyedGG10,
  Title                    = {Effort and Quality of Recovering Requirements-to-Code Traces: Two Exploratory Experiments},
  Author                   = {Alexander Egyed and Florian Graf and Paul GrÃ¼nbacher},
  Booktitle                = {Proceedings 18th IEEE Int'l Requirements Engineering Conference, Sydney, New South Wales, Australia},
  Year                     = {2010},
  Pages                    = {221-230},
  Publisher                = {IEEE Computer Society},

  Abstract                 = {Trace links between requirements and code are essential for many software development and maintenance activities. Despite significant advances in traceability research, creating links remains a human-intensive activity and surprisingly little is known about how humans perform basic tracing tasks. We investigate fundamental research questions regarding the effort and quality of recovering traces between requirements and code. Our paper presents two exploratory experiments conducted with 100 subjects who recovered trace links for two open source software systems in a controlled environment. In the first experiment, subjects recovered trace links between the two systems' requirements and classes of the implementation. In the second experiment, trace links were established between requirements and individual methods of the implementation. In order to assess the validity of the trace links cast by subjects, key developers of the two software systems participated in our research and provided benchmarks. Our study yields surprising observations: trace capture is surprisingly fast and can be done within minutes even for larger classes; the quality of the captured trace links, while good, does not improve with higher trace effort; and it is not harder though slightly more expensive to recover the trace links for larger, more complex classes.},
  Bibsource                = {DBLP, http://dblp.uni-trier.de},
  Doi                      = {10.1109/RE.2010.34},
  File                     = {Effort and Quality of Recovering Requirements-to-Code Traces:Conferences\\RE 2010 - Effort and Quality of Recovering Requirements-to-Code Traces - Two Exploratory Experiments\\Effort and Quality of Recovering Requirements-to-Code Traces.pdf:PDF},
  ISBN                     = {978-0-7695-4162-4},
  Keywords                 = {requirements, traceability},
  Owner                    = {paul},
  Timestamp                = {2015.09.12}
}