Books
Runtime Verification
"
Runtime Verification" , , , , , , , , , eds., Berlin, Heidelberg, 2010,
Link
Guide to Advanced Empirical Software Engineering
Forrest Shull, Janice Singer, Sjoberg D.I.K., "
Guide to Advanced Empirical Software Engineering", London, pp.388, 2007,
Link
Coordination Models and Languages
Arnab Ray, "
Coordination Models and Languages" , , eds., Berlin, Heidelberg, pp.125-139, 2005,
Link
Distributed Computing and Internet Technology
Arnab Ray, Bikram Sengupta, Rance Cleaveland, "
Distributed Computing and Internet Technology" , , eds., Berlin, Heidelberg, pp.273-282, 2005,
Link
Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili
Barry Boehm, Dieter H Rombach, Marvin V Zelkowitz, "
Foundations of Empirical Software Engineering: The Legacy of Victor R. Basili", pp.440, 2005,
Linkshow abstract [+]
This book captures the main scientific contributions of Victor R. Basili, who has significantly shaped the field of empirical software engineering from its very start. He was the first to claim that software engineering needed to follow the model of other physical sciences and develop an experimental paradigm. By working on this postulate, he developed concepts that today are well known and widely used, including the Goal-Question-Metric method, the Quality-Improvement paradigm, and the Experience Factory. He is one of the few software pioneers who can aver that their research results are not just scientifically acclaimed but are also used as industry standards. On the occasion of his 65th birthday, celebrated with a symposium in his honor at the International Conference on Software Engineering in St. Louis, MO, USA in May 2005, Barry Boehm, Hans Dieter Rombach, and Marvin V. Zelkowitz, each a long-time collaborator of Victor R. Basili, selected the 20 most important research papers of their friend, and arranged these according to subject field. They then invited renowned researchers to write topical introductions. The result is this commented collection of timeless cornerstones of software engineering, hitherto available only in scattered publications.
Conference Proceedings
Detecting Inconsistencies in Wrappers - A Case Study -
Henning Femmer, Dharmalingam Ganesan, Mikael Lindvall, David Mccomas, "Detecting Inconsistencies in Wrappers - A Case Study -", 2013
Investigating the Impact of Design Debt on Software Quality
Nico Zazworka, Michele Shaw, Forrest Shull, Carolyn B. Seaman, "Investigating the Impact of Design Debt on Software Quality", (Honolulu, HI), 2011
Data structure choices for on-the-fly model checking of real-time systems
P. Fontana, Rance Cleaveland, "Data structure choices for on-the-fly model checking of real-time systems", 2011
Architecture Reconstruction and Analysis of Medical Device Software
Dharmalingam Ganesan, Mikael Lindvall, Rance Cleaveland, Raoul Jetley, Paul Jones, Yi Zhang, "Architecture Reconstruction and Analysis of Medical Device Software", 2011,
Link
Architecture-based Unit Testing of the Flight software Product Line
Dharmalingam Ganesan, Mikael Lindvall, D McComas, M Bartholomew, Barbara Medina, "Architecture-based Unit Testing of the Flight software Product Line", (Jeju Island, South Korea), 2010,
Linkshow abstract [+]
This paper presents an analysis of the unit testing approach developed and used by the Core Flight Software (CFS) product line team at the NASA GSFC. The goal of the analysis is to understand, review, and recommend strategies for improving the existing unit testing infrastructure as well as to capture lessons learned and best practices that can be used by other product line teams for their unit testing. The CFS unit testing framework is designed and implemented as a set of variation points, and thus testing support is built into the product line architecture. The analysis found that the CFS unit testing approach has many practical and good solutions that are worth considering when deciding how to design the testing architecture for a product line, which are documented in this paper along with some suggested improvements
Building Empirical Support for Automated Code Smell Detection
Jan Schumacher, Nico Zazworka, Forrest Shull, Carolyn B. Seaman, Michele Shaw, "Building Empirical Support for Automated Code Smell Detection", (Bolzano, Italy), 2010
Searching for Relevant Software Change Artifacts using Semantic Networks
Mikael Lindvall, Raimund.L. Feldmann, George Karabatis, Zhiyuan Chen, Vandana P Janeja, "Searching for Relevant Software Change Artifacts using Semantic Networks", (New York, NY), 2009,
Linkshow abstract [+]
The discovery of software artifacts (files, documents, and datasets) relevant to a change request, can increase software reuse and reduce the cost of software development and maintenance. However, traditional search techniques often fail to provide the relevant documents because they do not consider relationships between software artifacts. We propose the creation of Semantic Networks which convey such relationships and assist in automatically discovering not only the requested artifacts based on a user query, but additional relevant ones that the user may not be aware of. Subsequently, we increase the accuracy of the returned artifacts by applying appropriate contexts. Experimental results show that this approach leads to better recall and precision compared to existing full-text search approaches.
Recovering Views of Inter-System Interaction Behaviors
Christopher Ackermann, Mikael Lindvall, Rance Cleaveland, "Recovering Views of Inter-System Interaction Behaviors", (Lille, France), 2009,
Linkshow abstract [+]
This paper presents an approach for recovering application-level views of the interaction behaviors between systems that communicate via networks. Rather than illustrating a single behavior, a sequence diagram is constructed that describes the characteristics of multiple combined behaviors. The approach has several properties that make it particularly suitable for analyzing heterogeneous systems. First, since the interactions are retrieved from observing the network communication, our technique can be applied to systems that are implemented in different languages and run on different platforms. Second, it does not require the availability or modification of source code. After the behaviors are extracted, we employ methods to merge multiple observed behaviors to a single sequence diagram that illustrates the overall behavior.The contributions of this paper are a technique for observing and processing the network communication to derive a model of the behavior. Furthermore, it describes a series of model transformations to construct a sequence diagram view of all observed behaviors.
Scope error detection and handling concerning software estimation models
S.A. Sarcia, Victor R. Basili, Giovanni Cantone, "Scope error detection and handling concerning software estimation models", (Lake Buena Vista, FL), 2009
Redesign for flexibility and maintainability: A case study
Christopher Ackermann, Mikael Lindvall, Greg Dennis, "Redesign for flexibility and maintainability: A case study", (Washington, DC), 2009
Developing an Approach for Analyzing and Verifying System Communication
William C Stratton, Deane E Sibol, Mikael Lindvall, Chris Ackermann, Sally Godfrey, "Developing an Approach for Analyzing and Verifying System Communication", (Big Sky, MT), 2009,
Link
Developing an approach for analyzing and verifying system communication
William C. Stratton, Deane E. Sibol, Mikael Lindvall, Chris Ackermann, Sally Godfrey, "Developing an approach for analyzing and verifying system communication", (Big Sky, MT), 2009,
Linkshow abstract [+]
Prominent characteristics of systems in the aerospace domain are that they are inherently complex, they must operate under tight resource constraints, and are often parts of a larger system of systems that must be reliable. These systems communicate with each other to exchange data and control information to together fulfill a larger task. In such a setup, the reliability of the communication channel plays a central role in the reliability of the entire system of systems and thus determines the success of fulfilling the larger task. Ensuring such a reliable communication is difficult due to several reasons: (1) the systems are developed independently by different teams at different locations, (2) the specification of the expected communication behavior is ambiguous, and (3) issues in the communication are often subtle and remain uncovered for a long time with the effect that bandwidth and other precious resources are wasted. We are proposing an approach called Dynamic Software Architecture Visualization and Evaluation (DynSAVE) to detect problems in the communication between systems by analyzing their communication behavior. The approach is divided into three main steps. The first step is the non-intrusive monitoring and recording of low level network traffic, the second step converts these raw communication records into meaningful messages, and the third step visualizes this abstracted information in such a way that issues can be detected. In this paper we discuss how the approach was applied to the Consultative Committee for Space Data Systems (CCSDS) File Delivery Protocol (CFDP), which is used for satellite communication by the JHU/APL Common Ground System. The approach has proven to be useful for understanding the communication behavior and uncovering subtle issues due to emerging system behaviors.
Evaluating the effectiveness of systems and software engineering methods, processes and tools for use in defense programs
Michael Pennotti, Richard Turner, Forrest Shull, "Evaluating the effectiveness of systems and software engineering methods, processes and tools for use in defense programs", (Vancouver, BC), 2009,
Linkshow abstract [+]
The systems engineering research center university affiliated research center (SERC-UARC) at Stevens Institute has been tasked to evaluate the effectiveness of the systems and software engineering processes, methods and tools (MPTs) used in US department of defense acquisition and development programs. This paper presents the selection and evaluation process, describes its evolution based on changing sponsor needs, and presents additional information on the characterization of MPTs for evaluation.
Gauging acceptance of software metrics: Comparing perspectives of managers and developers
Medha Umarji, Carolyn Seaman, "Gauging acceptance of software metrics: Comparing perspectives of managers and developers", (Lake Buena Vista, FL), 2009,
Linkshow abstract [+]
Metrics efforts are often impeded by factors such as poor data quality and developer resistance. To better understand and thus to address the developer perspective in a metrics program we undertook a case study at a large multi-national corporation. We identified six projects, and conducted surveys of both project managers and developers. These surveys were based on the Metrics Acceptance Model (MAM) which is a framework (i.e. a model of relationships between factors, operationalized by a survey instrument) for gauging developer opinions toward software metrics. We noticed some interesting differences between developers' and managers' perceptions of metrics. While managers were on the same page as developers when it came to factors such as ease of use of the metrics tool, they over-estimated developers' confidence to report accurate measures. Managers under-estimated developers' beliefs about the usefulness of metrics and about their fear of adverse consequences. These findings suggest that the MAM could provide useful insights to project managers to train and motivate their developers. We also found that the MAM can be an effective diagnostic tool both at an organizational and project level to identify potential impediments in metrics programs.
Coordination in large-scale software teams
Andrew Begel, Nachiappan Nagappan, Christopher Poile, Lucas Layman, "Coordination in large-scale software teams", (Vancouver, BC), 2009,
Link
Exploring language in software process elicitation: A grounded theory approach
Carlton A. Crabtree, Carolyn B. Seaman, Anthony F. Norcio, "Exploring language in software process elicitation: A grounded theory approach", (Lake Buena Vista, FL), 2009,
Link
Searching for relevant software change artifacts using semantic networks
Mikael Lindvall, Raimund L. Feldmann, George Karabatis, Zhiyuan Chen, Vandana P. Janeja, "Searching for relevant software change artifacts using semantic networks", (New York, New York, USA), 2009,
Link
Validating Automotive Control Software Using Instrumentation-Based Verification
Arnab Ray, Iris Morschhaeuser, Chris Ackermann, Rance Cleaveland, Charles Shelton, Chris Martin, "Validating Automotive Control Software Using Instrumentation-Based Verification", 2009,
Link
Using uncertainty as a model selection and comparison criterion
Salvatore Alessandro Sarcia', Victor Robert Basili, Giovanni Cantone, "Using uncertainty as a model selection and comparison criterion", (New York, New York, USA), 2009,
Link
Towards Behavioral Reflexion Models
Christopher Ackermann, Mikael Lindvall, Rance Cleaveland, "Towards Behavioral Reflexion Models", (Mysuru, Karnataka, India), 2009,
Linkshow abstract [+]
Software architecture has become essential in the struggle to manage today’s increasingly large and complex systems. Software architecture views are created to capture important system characteristics on an abstract and, thus, comprehensible level. As the system is implemented and later maintained, it often deviates from the original design specification. Such deviations can have implication for the quality of the system, such as reliability, security, and maintainability. Software architecture compliance checking approaches, such as the reflexion model technique, have been proposed to address this issue by comparing the implementation to a model of the systems’ architecture design. However, architecture compliance checking approaches focus solely on structural characteristics and ignore behavioral conformance. This is especially an issue in Systems-of-Systems. Systems-of-Systems (SoS) are decompositions of large systems, into smaller systems for the sake of flexibility. Deviations of the implementation to its behavioral design often reduce the reliability of the entire SoS. An approach is needed that supports the reasoning about behavioral conformance on architecture level.In order to address this issue, we have developed an approach for comparing the implementation of a SoS to an architecture model of its behavioral design. The approach follows the idea of reflexion models and adopts it to support the compliance checking of behaviors. In this paper, we focus on sequencing properties as they play an important role in many SoS. Sequencing deviations potentially have a severe impact on the SoS’ correctness and qualities. The desired behavioral specification is defined in UML sequence diagram notation and behaviors are extracted from the SoS implementation. The behaviors are then mapped to the model of the desired behavior and the two are compared. Finally, a reflexion model is constructed that shows the deviations between behavioral design and implementation. This paper discusses the approach and shows how it can be applied to investigate reliability issues in SoS.
Validating Automotive Control Software Using Instrumentation-Based Verification
Arnab Ray, Iris Morschhaeuser, Chris Ackermann, Rance Cleaveland, Charles Shelton, Chris Martin, "Validating Automotive Control Software Using Instrumentation-Based Verification", 2009,
Link
Why developers insert security vulnerabilities into their code
Karina Karppinen, Lyly Yonkwa, Mikael Lindvall, "Why developers insert security vulnerabilities into their code", 2009,
Linkshow abstract [+]
Modern software systems are difficult to test due to their distributed nature, and increased security complicates testing even further. Our hypothesis is that some security vulnerabilities are actually introduced due to developers’ need to facilitate testing that software requirements have been implemented correctly. If these temporary security vulnerabilities are not removed before the software is delivered, there is a great risk that they may become fielded security vulnerabilities.In this paper, we study the relationship between such security vulnerabilities and developers' need to improve the testability of an application to facilitate unit and integration testing. We trace detected vulnerabilities to characteristics of the software that made testing difficult and therefore led to testability improvements. We discuss how the need to increase testability may relate to a form of developer usability, and what the ways of dealing with the problem of security vulnerabilities as a consequence of increasing testability are.
Why Developers Insert Security Vulnerabilities into Their Code
Kaarina Karppinen, Lyly Yonkwa, Mikael Lindvall, "Why Developers Insert Security Vulnerabilities into Their Code", 2009,
Linkshow abstract [+]
Modern software systems are difficult to test due to their distributed nature, and increased security complicates testing even further. Our hypothesis is that some security vulnerabilities are actually introduced due to developers’ need to facilitate testing that software requirements have been implemented correctly. If these temporary security vulnerabilities are not removed before the software is delivered, there is a great risk that they may become fielded security vulnerabilities.In this paper, we study the relationship between such security vulnerabilities and developers' need to improve the testability of an application to facilitate unit and integration testing. We trace detected vulnerabilities to characteristics of the software that made testing difficult and therefore led to testability improvements. We discuss how the need to increase testability may relate to a form of developer usability, and what the ways of dealing with the problem of security vulnerabilities as a consequence of increasing testability are.
Verifying architectural design rules of the flight software product line
Dharmalingam Ganesan, Mikael Lindvall, Chris Ackermann, David McComas, Maureen Bartholomew, "Verifying architectural design rules of the flight software product line", 2009,
Link
The evolution and impact of code smells: A case study of two open source systems
Steffen Olbrich, Daniela S. Cruzes, Victor Basili, Nico Zazworka, "The evolution and impact of code smells: A case study of two open source systems", (Lake Buena Vista, FL), 2009,
Linkshow abstract [+]
Code smells are design flaws in object-oriented designs that may lead to maintainability issues in the further evolution of the software system. This study focuses on the evolution of code smells within a system and their impact on the change behavior (change frequency and size). The study investigates two code smells, God Class and Shotgun Surgery, by analyzing the historical data over several years of development of two large scale open source systems. The detection of code smells in the evolution of those systems was performed by the application of an automated approach using detection strategies. The results show that we can identify different phases in the evolution of code smells during the system development and that code smell infected components exhibit a different change behavior. This information is useful for the identification of risk areas within a software system that need refactoring to assure a future positive evolution.
Software Engineering Education for Bioinformatics
Medha Umarji, Carolyn Seaman, A. Gunes Koru, Hongfang Liu, "Software Engineering Education for Bioinformatics", (Hyderabad, India), 2009,
Linkshow abstract [+]
As software engineering educators, it is important for us to realize the increasing domain-specificity of software, and incorporate these changes in our design of teaching material. Bioinformatics software is an example of immensely complex and critical scientific software and this domain provides an excellent illustration of the role of computing in the life sciences. To study bioinformatics from a software engineering standpoint, we conducted an exploratory survey of bioinformatics developers. The survey had a range of questions about people, processes and products. We learned that practices like extreme programming, requirements engineering and documentation. As software engineering educators, we realized that the survey results had important implications for the education of bioinformatics professionals. We also investigated the current status of software engineering education in bioinformatics, by examining the curricula of more than fifty bioinformatics programs and the contents of over fifteen textbooks. We observed that there was no mention of the role and importance of software engineering practices essential for creating dependable software systems. Based on our findings and existing literature we present a set of recommendations for improving software engineering education in bioinformatics.
Architecture-Centric Reliability Analysis of the NASA cFE/CFS
Dharmalingam Ganesan, Mikael Lindvall, D McComas, M Bartholomew, Mike Blau, Glenn Cammarata, "Architecture-Centric Reliability Analysis of the NASA cFE/CFS", 2009
Tool supported detection and judgment of nonconformance in process execution
Nico Zazworka, Victor R. Basili, Forrest Shull, "Tool supported detection and judgment of nonconformance in process execution", (Lake Buena Vista, FL), 2009
Experience Report on the Effect of Software Development Characteristics on Change Distribution
A Gupta, Forrest Shull, Daniela Cruzes, Chris Ackermann, H Ronneberg, E Landre, "Experience Report on the Effect of Software Development Characteristics on Change Distribution", 2008,
Linkshow abstract [+]
This paper reports on an industrial case study in a large Norwegian Oil and Gas company (StatoilHydro ASA) involving a reusable Java-class framework and an application that uses that framework. We analyzed software changes from three releases of the framework and the application. On the basis of our analysis of the data, we found that perfective and corrective changes account for the majority of changes in both the reusable framework and the non-reusable application. Although adaptive changes are more frequent and has longer active time in the reusable framework, it went through less refactoring compared to the non-reusable application. For the non-reusable application we saw preventive changes as more frequent and with longer active time. We also found that designing for reuse seems to lead to fewer changes, as well as we saw a positive effect on doing refactoring.
Going Beyond the Numbers--The Usefulness of Qualitative Data on Software Development Projects
Linda Esker, Carolyn Seaman, Forrest Shull, Kathleen Dangle, "Going Beyond the Numbers--The Usefulness of Qualitative Data on Software Development Projects", 2008
show abstract [+]
Software development metrics are usually quantitative and analyzed using traditional methods of computation on numbers. Although qualitative data has long been used for the social sciences, they do not play a large role in software development metrics and analysis. However, there is a lot of insight to be obtained from qualitative data on software development projects. In our presentation, we will discuss our experiences applying a qualitative approach to software project analysis. We will investigate setting up a classification of the data by topics, understanding how the topics relate to goals, and then aggregating results to enable a better understanding of the issues. Qualitative analysis provides a new perspective when examining trends and anomalies and can lead to improvements for training, management, and a program's culture.
The Influence of Organizational Structure on Software Quality: An Empirical Case Study
N Nagappan, Murphy, B., Victor R Basili, "The Influence of Organizational Structure on Software Quality: An Empirical Case Study", (New York, New York), 2008,
Linkshow abstract [+]
Often software systems are developed by organizations consisting of many teams of individuals working together. Brooks states in the Mythical Man Month book that product quality is strongly affected by organization structure. Unfortunately there has been little empirical evidence to date to substantiate this assertion. In this paper we present a metric scheme to quantify organizational complexity, in relation to the product development process to identify if the metrics impact failure-proneness. In our case study, the organizational metrics when applied to data from Windows Vista were statistically significant predictors of failure-proneness. The precision and recall measures for identifying failure-prone binaries, using the organizational metrics, was significantly higher than using traditional metrics like churn, complexity, coverage, dependencies, and pre-release bug measures that have been used to date to predict failure-proneness. Our results provide empirical evidence that the organizational metrics are related to, and are effective predictors of failure-proneness.
Using Sequence Diagrams to Detect Communication Problems between Systems
Mikael Lindvall, Chris Ackermann, William C Stratton, Deane E Sibol, Arnab Ray, Lyly Yonkwa, Jan Kresser, Sally Godfrey, Jens Knodel, "Using Sequence Diagrams to Detect Communication Problems between Systems", (Big Sky, MT), 2008,
Linkshow abstract [+]
Many software systems are evolving complex system of systems (SoS) for which inter-system communication is both mission-critical and error-prone. Such communication problems ideally would be detected before deployment. In a NASA-supported Software Assurance Research Program (SARP) project, we are researching a new approach addressing such problems. In this paper, we show that problems in the communication between two systems can be detected by using sequence diagrams to model the planned communication and by comparing the planned sequence to the actual sequence. We identify different kinds of problems that can be addressed by modeling the planned sequence using different level of abstractions.
A Methodology for Using Measures to Assess Software Safety Risk From an Independent Testing Perspective
Victor R Basili, Kathleen Dangle, Linda Esker, "A Methodology for Using Measures to Assess Software Safety Risk From an Independent Testing Perspective", 2008
show abstract [+]
Software contributes an ever-increasing level of functionality and control in today's systems. This increased use of software can dramatically increase the complexity and time needed to evaluate the safety of a system. The independent safety test team who assures the system is safe plays an important role in system safety by evaluating the safety characteristics of the software through a thorough analysis and disciplined test process. The result of these activities is assurance that safety releases for software-intensive systems are credible.
Model-Based Design Verification: A Monitor Based Approach
Chris Ackermann, Arnab Ray, Rance Cleaveland, Juergen Heit Shelton, Charles, Martin, Chris, "Model-Based Design Verification: A Monitor Based Approach", 2008,
Linkshow abstract [+]
This paper assesses the utility of an automated functional verification methodology in the design of a body electronics application. The use of design models as detailed specifications for software is gaining steady acceptance in the automotive industry because of the advantages it offers in terms of rapid system prototyping and automatic code generation. This paper argues that such modeling effort can also be used to provide support for design verification, so that errors may be pinpointed before they find their way into code, when they become much harder and more expensive to isolate, diagnose and repair.
The Role of MPI in Development Time: A Case Study
Lorin Hochstein, Forrest Shull, L Reid, "The Role of MPI in Development Time: A Case Study", 2008
show abstract [+]
There is widespread belief in the computer science community that MPI is a difficult and time-intensive approach to developing parallel software. Nevertheless, MPI remains the dominant programming model for HPC systems, and many projects have made effective use of it. It remains unknown how much impact the use of MPI truly has on the productivity of computational scientists. In this paper, we examine a mature, ongoing HPC project, the Flash Center at the University of Chicago, to understand how MPI is used and to estimate the time that programmers spend on MPI-related issues during development. Our analysis is based on an examination of the source code, version control history, and regression testing history of the software. Based on our study, we estimate that about 20% of the development effort is related to MPI. This implies a maximum productivity improvement of 25% for switching to an alternate parallel programming model.
Detecting Security Vulnerabilities with Software Architecture Analysis Tools
Karina Karppinen, Mikael Lindvall, Lyly Yonkwa, "Detecting Security Vulnerabilities with Software Architecture Analysis Tools", (Lillehammer, Norway), 2008,
Link
Advanced Search Tools for Learning Software Organizations based on Semantic Networks
Mikael Lindvall, Zhiyuan Chen, "Advanced Search Tools for Learning Software Organizations based on Semantic Networks", 2008
Adopting Curvilinear Component Analysis to Improve Software Cost Estimation Accuracy: Model, Application Strategy, and an Experimental Verification
S A Sarcia, G Cantone, Victor R Basili, "Adopting Curvilinear Component Analysis to Improve Software Cost Estimation Accuracy: Model, Application Strategy, and an Experimental Verification", 2008,
Linkshow abstract [+]
Cost estimation is a critical issue for software organizations. Good estimates can help us make more informed decisions (controlling and planning software risks), if they are reliable (correct) and valid (stable). In this study, we apply a variable reduction technique (based on auto-associative feed--forward neural networks – called Curvilinear component analysis) to log-linear regression functions calibrated with ordinary least squares. Based on a COCOMO 81 data set, we show that Curvilinear component analysis can improve the estimation model accuracy by turning the initial input variables into an equivalent and more compact representation. We show that, the models obtained by applying Curvilinear component analysis are more parsimonious, correct, and reliable.
Defect Categorization: Making Use of a Decade of Widely Varying Historical Data
Carolyn Seaman, Forrest Shull, Myrna Regardie, D Elbert, Raimund.L. Feldmann, Y Guo, Sally Godfrey, "Defect Categorization: Making Use of a Decade of Widely Varying Historical Data", 2008,
Linkshow abstract [+]
This paper describes our experience in aggregating a number of historical datasets containing inspection defect data using different categorization schemes. Our goal was to make use of the historical data by creating models to guide future development projects. We describe our approach to reconciling the different choices used in the historical datasets to categorize defects, and the challenges we faced. We also present a set of recommendations for others involved in classifying defects.
An Analysis Framework for Inter-System Interaction Behavior
Chris Ackermann, Deane E Sibol, William C Stratton, Mikael Lindvall, Sally Godfrey, "An Analysis Framework for Inter-System Interaction Behavior", 2008
show abstract [+]
Systems often collaborate to form a system-of-systems (SoS) and together fulfill some larger task. Correctness and performance issues in the interaction between participating systems are frequent occurrences and decrease the reliability of the entire SoS. We are currently developing an analysis framework to automatically compare a model of the desired interaction behavior (specification) to a model that is retrieved from the system execution and detect deviations between the two. The specification, the observed interaction behavior, and the evaluation result are presented in behavioral diagrams to be analyzed by the user.
Analyzing the Core Flight Software with SAVE
Dharmalingam Ganesan, Mikael Lindvall, M Bartholomew, D McComas, G Cammarata, "Analyzing the Core Flight Software with SAVE", 2008
show abstract [+]
This viewgraph presentation describes the SAVE tool and its application to Core Flight Software (CFS). The contents include: 1) Fraunhofer-a short intro; 2) Context of this Collaboration; 3) CFS-Core Flight Software; 4) The SAVE Tool; 5) Applying SAVE to CFS -A few example analyses; and 6) Goals.
Analyzing and detecting problems in Systems of Systems
Mikael Lindvall, Chris Ackermann, William C Stratton, Deane E Sibol, Sally Godfrey, "Analyzing and detecting problems in Systems of Systems", (Washington, D. C.), 2008
show abstract [+]
Many software systems are evolving complex system of systems (SoS) for which inter-system communication is mission-critical. Evidence indicates that transmission failures and performance issues are not uncommon occurrences. In a NASA-supported Software Assurance Research Program (SARP) project, we are researching a new approach addressing such problems. In this paper, we are presenting an approach for analyzing inter-system communications with the goal to uncover both transmission errors and performance problems. Our approach consists of a visualization and an evaluation component. While the visualization of the observed communication aims to facilitate understanding, the evaluation component automatically checks the conformance of an observed communication (actual) to a desired one (planned). The actual and the planned are represented as sequence diagrams. The evaluation algorithm checks the conformance of the actual to the planned diagram. We have applied our approach to the communication of aerospace systems and were successful in detecting and resolving even subtle and long existing transmission problems.
Model-Based Validation & Verification Integrated with SW Architecture Analysis: A Feasibility Study
Iris Morschhaeuser, Mikael Lindvall, "Model-Based Validation & Verification Integrated with SW Architecture Analysis: A Feasibility Study", 2007
A Snapshot of the State of Practice in Software Development for Medical Devices
C Denger, Raimund.L. Feldmann, M Host, Christin Lindholm, Forrest Shull, "A Snapshot of the State of Practice in Software Development for Medical Devices", (Madrid, Spain), 2007,
Linkshow abstract [+]
The medical device industry is a constantly growing domain which makes use of more and more software products. Given the importance to this industry of dependable software components, rigorous software engineering techniques would seem to have an important role to play. However, in a recent survey of the industry we found a lower than expected rate of adoption of certain sound software engineering practices. To ensure and improve the quality of developed software products, whether they are stand-alone applications or embedded in complex systems, both the medical device industry as well as software engineers have to take action. Our survey is a first step in this direction and may help in identifying appropriate programs and future research topics.
Using Context Distance Measurement to Analyze Results across Studies
Daniela Cruzes, M Mendonca, Victor R Basili, Forrest Shull, M Jino, "Using Context Distance Measurement to Analyze Results across Studies", (Madrid, Spain), 2007,
Linkshow abstract [+]
Providing robust decision support for software engineering (SE) requires the collection of data across multiple contexts so that one can begin to elicit the context variables that can influence the results of applying a technology. However, the task of comparing contexts is complex due to the large number of variables involved. This works extends a previous one in which we proposed a practical and rigorous process for identifying evidence and context information from SE papers. The current work proposes a specific template to collect context information from SE papers and an interactive approach to compare context information about these studies. It uses visualization and clustering algorithms to help the exploration of similarities and differences among empirical studies. This paper presents this approach and a feasibility study in which the approach is applied to cluster a set of papers that were independently grouped by experts.
A Survey of Software Engineering Techniques in Medical Device Development
Raimund.L. Feldmann, Forrest Shull, Christian Denger, Martin Hoest, Christin Lindholm, "A Survey of Software Engineering Techniques in Medical Device Development", (Los Alamitos, CA), 2007,
Linkshow abstract [+]
A wide variety of the functions provided by today's medical devices relies heavily on soft ware. Most of these capabilities could not be offered without the underlying integrated software solutions. As a result, the medical device industry has become highly interdisciplinary. Medical device manufacturers are finding an increasing need to incorporate the research ideas and results from traditionally disconnected research areas such as medicine, software and system engineering, and mechanical engineering. In 2006, we conducted a survey with more than 100 companies from Europe and the USA to shine some light on the current status of the integration of software engineering technologies into the medical device domain. The initial results of this survey are presented in this paper. Both software engineers and the medical device industry can use these findings to better understand current challenges and future directions, to achieve a better integration of the fields.
Management By Data: The GQM+Strategies for Mapping Business Strategy to Software Development
Victor R Basili, J Muench, Jens Heidrich, Mikael Lindvall, Myrna Regardie, Carolyn Seaman, Adam Trendowicz, "Management By Data: The GQM+Strategies for Mapping Business Strategy to Software Development", 2007
Bridging the Gap Between Business Strategy and Software Measurement Tutorial
Myrna Regardie, Carolyn Seaman, "Bridging the Gap Between Business Strategy and Software Measurement Tutorial", 2007
show abstract [+]
In software-intensive organizations, an organizational management system will not guarantee organizational success unless the business strategy can be translated into a set of operational software goals. The Goal Question Metric (GQM) approach has proven itself useful in a variety of industrial settings to support quantitative software project management. However, it does not address linking software measurement goals to higher-level goals of the organization in which the software is being developed. This linkage is important, as it helps to justify software measurement efforts and allows measurement data to contribute to higher-level decisions.
Assessing the Quality Impact of Design Inspections
Chris Ackermann, Forrest Shull, R Carbon, C Denger, Mikael Lindvall, "Assessing the Quality Impact of Design Inspections", (Madrid, Spain), 2007,
Linkshow abstract [+]
Inspections are widely used and studies have found them to be effective in uncovering defects. However, there is less data available regarding the impact of inspections on different defect types and almost no data quantifying the link between inspections and desired end product qualities. This paper addresses this issue by investigating whether design inspection checklists can be tailored so as to effectively target certain defect types without impairing the overall defect detection rate. The results show that the design inspection approach used here does uncover useful design quality issues and that the checklists can be effectively tailored for some types of defects.
Software architecture-driven detection of security vulnerabilities
Karina Karppinen, Mikael Lindvall, "Software architecture-driven detection of security vulnerabilities", 2007
Technology Infusion of the SAVE Tool into the Common Ground Software Development Process for NASA Missions at JHU/APL
William C Stratton, Deane E Sibol, Mikael Lindvall, Patricia Costa, "Technology Infusion of the SAVE Tool into the Common Ground Software Development Process for NASA Missions at JHU/APL", (Big Sky, MT), 2007,
Linkshow abstract [+]
The JHU/APL Space Department develops Missions Operations Center (MOC) system software for all JHU/APL-supported NASA missions using a shared architecture called Common Ground. The architecture is 10 years old and difficult to maintain for current missions and to evolve for reuse in future missions. The Software Architecture Visualization and Evaluation (SAVE) tool and process developed by Fraunhofer Center -Maryland has been applied to preserve the Common Ground architectural goals in order to avoid further maintenance and evolution problems. This report summarizes JHU/APL's experiences in using the SAVE tool and process to capture the originally planned architecture, to compare the actual Common Ground software to the plan, to create a new target architecture, and to guide ongoing development to bring the planned and actual architectures into alignment.
Bridging the Gap between Business Strategy and Software Development
Victor R Basili, J Muench, Jens Heidrich, Mikael Lindvall, Myrna Regardie, Carolyn Seaman, Adam Trendowicz, "Bridging the Gap between Business Strategy and Software Development", (Montreal, Canada), 2007,
Link
Automated Information Extraction from Empirical Software Engineering Literature: Is That Possible?
Daniela Cruzes, M Mendonca, Victor R Basili, Forrest Shull, M Jino, "Automated Information Extraction from Empirical Software Engineering Literature: Is That Possible?", (Madrid, Spain), 2007,
Linkshow abstract [+]
The number of scientific publications is constantly increasing, and the results published on Empirical Software Engineering are growing even faster. Some software engineering publishers have begun to collaborate with research groups to make available repositories of software engineering empirical data. However, these initiatives are limited due to data ownership and privacy issues. As a result, many researchers in the area have adopted systematic reviews as a mean to extract empirical evidence from published material. Systematic reviews are labor intensive and costly. In this paper, we argue that the use of Information Extraction Tools can support systematic reviews and significantly speed up the creation of repositories of SE empirical evidence.
The SAVE Tool and Process Applied to Ground Software Development at JHU/APL:An Experience Report on Technology Infusion
William C Stratton, Deane E Sibol, Mikael Lindvall, Patricia Costa, "The SAVE Tool and Process Applied to Ground Software Development at JHU/APL:An Experience Report on Technology Infusion", (Columbia, MD), 2007,
Linkshow abstract [+]
The JHU/APL Space Department develops Missions Operations Center (MOC) system software for all JHU/APL-supported NASA missions using a shared software architecture called Common Ground. The software architecture is 10 years old and difficult to maintain for current missions and to evolve for reuse in future missions. The Software Architecture Visualization and Evaluation (SAVE) tool and process developed by Fraunhofer Center - Maryland has been applied to the Common Ground software in order to avoid further maintenance and evolution problems. This report summarizes JHU/APL's experiences in using the SAVE tool and process to capture the originally planned architecture, to compare the actual Common Ground software to the planned architecture, to create a new target architecture, and to guide ongoing development to bring the planned and actual architectures into alignment.
Gaining Early Insight into Software Safety: Measures of Potential Problems and Risks
Victor R Basili, Kathleen Dangle, Linda Esker, Frank Marotta, Ioana Rus, "Gaining Early Insight into Software Safety: Measures of Potential Problems and Risks", 2007
show abstract [+]
Evaluation of the safety of a software system is a complex problem, usually done during test at the end of the system's development life cycle. While we cannot measure the safety of the software system during its development, it is possible to gain visibility into software safety considerations, thereby reducing safety risks. There are several levels at which safety visibility can be gained. We propose that visibility is achieved by the identification of safety-related software requirements, their traceability, the identification and quantification of safety-related defects, and the extent to which safety-related failures are detectable by software. We present a set of software-related safety metrics tailored to the needs of a large complex system of systems program. These metrics can support detection and analysis of software-related safety issues. For defining these metrics we leverage existing data sources (e.g., hazard tracking database, requirements management tools, etc.) and processes (e.g., safety analysis processes). The purpose of this presentation is to 1. Address the visibility into the software safety process, as the system is being developed 2. Define concepts such as software safety requirements and software safety hazards, so they can be made visible and measured 3. Discuss multiple levels at which we can measure and analyze the software safety process, from the initial breathalyzer to deeper levels, depending on the maturity of the organization's measurement capability Propose a set of metrics associated to each of these levels
GQM+Strategies - Aligning Business Strategies with Software Measurement
Victor R Basili, Jens Heidrich, Mikael Lindvall, J Muench, Myrna Regardie, Adam Trendowicz, "GQM+Strategies - Aligning Business Strategies with Software Measurement", (Madrid, Spain), 2007
show abstract [+]
GQM^+ Strategies is a measurement approach that builds on the well-tested GQM approach to planning and implementing software measurement. Although GQM has proven itself useful in a variety of industrial settings, one recognized weakness is the difficulty for GQM users to link software measurement goals to higher-level goals of the organization in which the software is being developed. This linkage is important, as it helps to justify software measurement efforts and allows measurement data to contribute to higher-level decisions. GQM^+ Strategies provides mechanisms for explicitly linking software measurement goals, to higher-level goals for the software organization, and further to goals and strategies at the level of the entire business.
Characterizing Software Architecture Changes: An Initial Study
Byron Williams, J Carver, "Characterizing Software Architecture Changes: An Initial Study", (Madrid, Spain), 2007
show abstract [+]
With today?s ever increasing demands on software, developers must produce software that can be changed without the risk of degrading the software architecture. Degraded software architecture is problematic because it makes the system more prone to defects and increases the cost of making future changes. The effects of making changes to software can be difficult to measure. One way to address software changes is to characterize their causes and effects. This paper introduces an initial architecture change characterization scheme created to assist developers in measuring the impact of a change on the architecture of the system. It also presents an initial study conducted to gain insight into the validity of the scheme. The results of this study indicated a favorable view of the viability of the scheme by the subjects, and the scheme increased the ability of novice developers to assess and adequately estimate change effort.
Extracting Information from Experimental Software Engineering Papers
Daniela Cruzes, M Mendonca, Victor R Basili, Forrest Shull, M Jino, "Extracting Information from Experimental Software Engineering Papers", (Iquique, Chile), 2007,
Linkshow abstract [+]
Experiments have been conducted to investigate analysis, design, implementation, testing, maintenance, quality assurance and reuse techniques, but, a body of evidence has not yet been built that enables a project manager to know with confidence what software processes produce what product characteristics and under what conditions. This paper extends an approach we proposed earlier to extract information from papers so that systematically analyzing results from several papers is possible. It also describes an in-vitro experiment we did with graduate students to validate the approach. The results show that the approach is feasible and can be taught to less experienced researchers.
Guidance Navigation and Control/Flight Software Framework
D McComas, "Guidance Navigation and Control/Flight Software Framework", 2007
Flight Software Architectural Modeling with SAVE
Mark Reid, "Flight Software Architectural Modeling with SAVE", 2007
Predicting Return on Investement for Product Line Generations
Dharmalingam Ganesan, Dirk Muthig, Kentaro Yoshimura, "Predicting Return on Investement for Product Line Generations", (Kaiserslautern, Germany), 2006,
Linkshow abstract [+]
The decision of an organization to introduce product line engineering depends on a sound and careful analysis of risks and return on investment. The latter is computed by an economic model, which relies on high quality input and must reflect the envisioned migration strategy sufficiently. To facilitate risk analysis, this paper applies Monte-Carlo simulation to an existing product line economic model. Additionally, the model is extended by the support of product line generations that is, considering the degeneration of product line infrastructures and taking reinvestment into an existing product line into account. The practical application of the model is demonstrated by an industrial case study
Comparing Code Reading and Testing Criteria: A Replication of Experimental Studies
J C Maldonado, S Fabbri, M Mendonca, E Doria, L A F Martimiano, J Carver, Forrest Shull, Victor R Basili, "Comparing Code Reading and Testing Criteria: A Replication of Experimental Studies", 2006
Understanding change requests in order to predict software impact
Chris Ackermann, Mikael Lindvall, "Understanding change requests in order to predict software impact", (Washington, DC), 2006
A Software Architectural Approach to Security by Design
Arnab Ray, Rance Cleaveland, "A Software Architectural Approach to Security by Design", 2006,
Link
Integrating Product Line Engineering and Agile Methods: Flexible Design Up-front vs. Incremental Design
R Carbon, Mikael Lindvall, Dirk Muthig, Patricia Costa, "Integrating Product Line Engineering and Agile Methods: Flexible Design Up-front vs. Incremental Design", (Baltimore, Maryland, USA), 2006,
Link
Static Evaluation of Software Architectures
Jens Knodel, Mikael Lindvall, Dirk Muthig, Matthias Naab, "Static Evaluation of Software Architectures", (Bari, Italy), 2006,
Linkshow abstract [+]
The software architecture is one of the most crucial artifacts within the lifecycle of a software system. Decisions made at the architectural level directly enable, facilitate, hamper, or interfere with the achievement of business goals, functional and quality requirements. Architecture evaluations play an important role in the development and evolution of software systems since they determine how adequate the architecture is for its intended usage. This paper summarizes our practical experience with using architecture evaluations and gives an overview on when and how static architecture evaluations contribute to architecture development. We identify ten distinct purposes and needs for static architecture evaluations and illustrate them using a set of industrial and academic case studies. In particular, we show how subsequent steps in architecture development are influenced by the results from architecture evaluations.
Building Decision Support in an Imperfect World
Raimund.L. Feldmann, Forrest Shull, Michele Shaw, "Building Decision Support in an Imperfect World", 2006
Getting the Most Out of Your Measurement Data: Approaches for Using Software Metrics
Linda Esker, Dave Zubrow, Kathleen Dangle, "Getting the Most Out of Your Measurement Data: Approaches for Using Software Metrics", 2006
show abstract [+]
Among the challenges facing the Army's Future Combat Systems program is the roll up, interpretation, and analysis of software measurement data that is reported by a large number of suppliers provided by partners, government organizations, and subcontractors. Data must accommodate the needs of the hierarchy responsible for managing the development of this incredibly large and complex network-centric system of systems. Data must inform management of relevant status and reveal insights into upcoming issues and questions in ways that are appropriate and understandable. Techniques for how to interpret the data and gather follow-up information are needed to ensure that the various levels of the organization are actually using the data. Many of these challenges are common to a single organization or on a single system; however, the magnitude of these issues is greater and requires some novel solutions.
Application of Design for Verification with Concurrency Controllers to Air Traffic Control Software
Aysu Betin-Can, Tevfik Bultan, Mikael Lindvall, Benjamin Lux, Stefan Topp, "Application of Design for Verification with Concurrency Controllers to Air Traffic Control Software", 2005,
Link
Establishing a Multi-Level, Multi-Organizational Measurement Program
Kathleen Dangle, Michael Edwards, Patricia E Larsen, Dave Zubrow, "Establishing a Multi-Level, Multi-Organizational Measurement Program", 2005
Metrics of software architecture changes based on structural distance
T Nakamura, Victor R. Basili, "Metrics of software architecture changes based on structural distance", (Como, Italy), 2005
Equipping the Reflexion Method with Automated Clustering
A Christl, R Koschke, M-A Storey, "Equipping the Reflexion Method with Automated Clustering", (Pittsburgh, Pennsylvania), 2005,
Linkshow abstract [+]
A significant aspect in applying the Reflexion Method is the mapping of components found in the source code onto the conceptual components defined in the hypothesized architecture. To date, this mapping is established manually, which requires a lot of work for large software systems. In this paper, we present a new approach, in which clustering techniques are applied to support the user in the mapping activity. The result is a semi-automated mapping technique that accommodates the automatic clustering of the source model with the user?s hypothesized knowledge about the system?s architecture. This paper describes also a case study in which our semi-automated mapping technique has been applied successfully to extend a partial map of a real-world software application.
Towards Reusable Measurement Patterns
Mikael Lindvall, Paolo Donzelli, Sima Asgari, Victor R Basili, "Towards Reusable Measurement Patterns", (Como, Italy), 2005,
Linkshow abstract [+]
Software measurement programs can help organizations make better decisions regarding their software projects. However, creating and establishing software measurement programs can be both costly and difficult. This paper addresses the problem by focusing on reusability of metrics for software measurement programs through the identification of measurement patterns. We illustrate our work with identifying measurement patterns by providing an extensive and detailed measurement example that is broken down into interdependent building blocks and activities.
An Empirical Approach to Best Practice Identification and Selection: The US Department of Defense Acquisition Best Practices Clearinghouse
Forrest Shull, Richard Turner, "An Empirical Approach to Best Practice Identification and Selection: The US Department of Defense Acquisition Best Practices Clearinghouse", (Noosa Heads, Australia), 2005
show abstract [+]
Recommendations concerning best practices are ubiquitous in academic literature as well as audit reports and institutional policy. However, information on the effectiveness of recommended practices, or the context within which that effectiveness has been observed, is not readily available. In the rare occasions this information is available, it is not packaged in such a way as to support practitioners. Most organizations and projects do not possess sufficient resources to implement even a fraction of the practices espoused, so often choose to adopt none. There are also few if any tools to support practice selection. This paper describes an approach to identify, analyze, synthesize, and package empirical information about practice effectiveness, and evaluates the knowledge gained from the development of a prototype tool designed to implement that approach.
Empirical Study Design in the Area of High Performance Computing (HPC)
Forrest Shull, J Carver, Lorin Hochstein, Victor R Basili, "Empirical Study Design in the Area of High Performance Computing (HPC)", (Noosa Heads, Australia), 2005,
Linkshow abstract [+]
The development of high-performance computing (HPC) programs is crucial to progress in many fields of scientific endeavor. We have run initial studies of the productivity of HPC developers and of techniques for improving that productivity, which have not previously been the subject of significant study. Because of key differences between development for HPC and for more conventional software engineering applications, this work has required the tailoring of experimental designs and protocols. A major contribution of our work is to begin to quantify the code development process in a specialized area that has previously not been extensively studied. Specifically, we present an analysis of the domain of high-performance computing for the aspects that would impact experimental design; show how those aspects are reflected in experimental design for this specific area; and demonstrate how we are using such experimental designs to build up a body of knowledge specific to the domain. Results to date build confidence in our approach by showing that there are no significant differences across studies comparing subjects with similar experience tackling similar problems, while there are significant differences in performance and effort among the different parallel models applied.
Static Evaluation of Software Architectures - A Short Summary
Jens Knodel, Mikael Lindvall, Dirk Muthig, "Static Evaluation of Software Architectures - A Short Summary", (Pittsburgh, PA), 2005,
Linkshow abstract [+]
The software architecture is one of the most crucial artifacts within the lifecycle of a software system. Decisions made at the architectural level directly enable, facilitate, hamper, or interfere with the achievement of business goals as well as meeting functional and quality requirements. Software architectures are also essential for the success of product line engineering. In this work, we summarize how, from our practical experience, static architecture evaluation contributes to architecture development. We also describe the different purposes of architectural evaluations.
Using Visualization to understand dependability: A tool support for requirements analysis (to appear)
Paolo Donzelli, Daniel Hirschbach, Victor R Basili, "Using Visualization to understand dependability: A tool support for requirements analysis (to appear)", 2005,
Linkshow abstract [+]
Dealing with dependability requirements is a complex task for stakeholders and analysts as many different aspects of a system must be taken into account at the same time: services characteristics and quality properties, failure modes and tolerable failure rates, reactions and recovery time in case of failure, and so on. Visualization helps cope with this complexity. In this paper, we build upon a practical framework for eliciting and modeling dependability requirements to show how graphical data representation can facilitate requirements analysis during the requirements elicitation and definition process. An Air Traffic Control System, adopted as a Testbed within the NASA High Dependability Computing Project, is used as a case study.
HPC Programmer Productivity: A Case Study of Novice HPC Programmers
Lorin Hochstein, J Carver, Forrest Shull, Sima Asgari, Victor R Basili, J K Hollingsworth, Marvin V Zelkowitz, "HPC Programmer Productivity: A Case Study of Novice HPC Programmers", (Seattle, WA), 2005
show abstract [+]
In developing High-Performance Computing (HPC) software, time to solution is an important metric. This metric is comprised of two main components: The human effort required developing the software, plus the amount of machine time required to execute it. To date, little empirical work has been done to study the first component: the human effort required and the effects of approaches and practices that may be used to reduce it. In this paper, we describe a series of studies that address this problem. We instrumented the development process used in multiple HPC classroom environments. We analyzed data within and across such studies, varying factors such as the parallel programming model used and the application being developed, to understand their impact on the development process.
Combining self-reported and automatic data to improve effort measurement
Lorin Hochstein, Victor R Basili, Marvin V Zelkowitz, J K Hollingsworth, J Carver, "Combining self-reported and automatic data to improve effort measurement", 2005
Generating Testable Hypotheses from Tacit Knowledge for High Performance Computing
Sima Asgari, Lorin Hochstein, Victor R Basili, J Carver, J K Hollingsworth, Forrest Shull, Marvin V Zelkowitz, "Generating Testable Hypotheses from Tacit Knowledge for High Performance Computing", 2005,
Linkshow abstract [+]
In this research, we are developing our understanding of how the high performance computing community develops effective parallel implementations of programs by collecting the folklore within the community. We use this folklore as the basis for a series of experiments, which we expect, will validate or negate these assumptions.
Preventing race condition attacks on file-systems
Prem Uppuluri, Uday Joshi, Arnab Ray, "Preventing race condition attacks on file-systems", (New York, New York, USA), 2005,
Link
Experimenting with Software Architecture Flexibility Using an Implementation of the Tactical Separation Assisted Flight Environment
Bettina Anders, Joerg Fellman, Mikael Lindvall, Ioana Rus, "Experimenting with Software Architecture Flexibility Using an Implementation of the Tactical Separation Assisted Flight Environment", (Greenbelt, MD), 2005,
Linkshow abstract [+]
At NASA, more and more functionality is implemented in software. Change requests tend to arrive late, making software flexibility an important aspect. However, such changes present a high risk and are more likely to introduce defects and to violate flexibility properties of the software, causing its "degeneration." In this paper, we present some preliminary results from a recent study of software architecture flexibility. The study was conducted as an experiment with students using the TSAFE experimental testbed. The main conclusion is that constructs for flexibility are not automatically understood, used, and followed by new developers. Better ways to describe the intention with certain constructs in combination with assigning responsibility for making sure that the implementation adheres to the intended architecture and its design rationale could provide a successful approach for avoiding degeneration of built-in flexibility
Architecture Reading Techniques: A Feasibility Study
J Carver, Krystle Lemon, "Architecture Reading Techniques: A Feasibility Study", (Mississippi, USA), 2005,
Link
Establishing Software Measurement on a HUGE Program: The Case of FCS
Michael Edwards, Kathleen Dangle, Patricia E Larsen, Dave Zubrow, Sue Hermanson, "Establishing Software Measurement on a HUGE Program: The Case of FCS", 2005
DOD Acquisition Best Practices Clearinghouse
Kathleen Dangle, Deborah Fuller, "DOD Acquisition Best Practices Clearinghouse", 2005
Agile Methods in Software Engineering Education
Christian Bunse, Raimund.L. Feldmann, Jorg Dorr, "Agile Methods in Software Engineering Education", (Garmisch-Partenkirchen, Germany), 2004
show abstract [+]
Agile methods such as extreme programming (XP) are becoming increasingly important for the rapid development of software applications. However, there is a tradeoff in using agile methods. Often they lack in providing a systematic guidance (i.e., a sound description of roles, artifacts, and activities), and thus, require disciplined and experienced developers. Are the promised benefits of agile methods still valid if they are applied by novice (student) developers? To gain some experience, we performed a study on teaching students agile software development with XP. Students performed a small software development project at the University of Kaiserslautern to collect some lessons learned. One result is that although agile approaches are easy to learn and quickly produce results, they are not the best starting point in training software development. The quality of the resulting system, at least in our experience, heavily depends on the discipline of the developers and their background and experience in software development.
DOD Best Practices Clearinghouse
Kathleen Dangle, Richard Turner, "DOD Best Practices Clearinghouse", 2004
Challenges in Measuring HPCS Learner Productivity in an Age of Ubiquitous Computing: The HPCS Program
Sima Asgari, Victor R Basili, J Carver, Lorin Hochstein, J K Hollingsworth, Forrest Shull, Marvin V Zelkowitz, "Challenges in Measuring HPCS Learner Productivity in an Age of Ubiquitous Computing: The HPCS Program", (Edinburgh, Scotland), 2004
show abstract [+]
Collecting development data automatically is difficult in this era of ubiquitous home computing. This paper describes our efforts in the High Productivity Computing Systems project to better calculate effort data among a set of student programming exercises. 1.
Knowledge Management as an Integral Part of Program Assessment
Kathleen Dangle, "Knowledge Management as an Integral Part of Program Assessment", 2004
Empirical-based Estimation of the Effect on Software Dependability of a Technique for Architecture Conformance Verification
Sima Asgari, Victor R Basili, Patricia Costa, Paolo Donzelli, Lorin Hochstein, Mikael Lindvall, Ioana Rus, Forrest Shull, Roseanne Tesoriero Tvedt, Marvin V Zelkowitz, "Empirical-based Estimation of the Effect on Software Dependability of a Technique for Architecture Conformance Verification", (Edinburgh, Scotland), 2004,
Linkshow abstract [+]
The High Dependability Computing Program (HDCP) project is a NASA initiative for increasing dependability of software-based systems. It researches achieving high dependability by introducing new technologies. We focus on the evaluation of the effectiveness of technologies with respect to dependability. We employ empirical evaluation methods along with evaluation testbeds. In this paper, our technology evaluation approach is described. A testbed representative of air traffic control applications is used. An experiment to evaluate a technology to for identification of architectural violations is presented.
Innovative Strategies for Implementing the CMMI
Richard Bechtold, Kathleen Dangle, Patricia E Larsen, "Innovative Strategies for Implementing the CMMI", 2004
Building a Useful Best Practices Clearinghouse
Kathleen Dangle, "Building a Useful Best Practices Clearinghouse", 2004
Studying Code Development for High Performance Computing: The HPCS Program
J Carver, Sima Asgari, Victor R Basili, Lorin Hochstein, J K Hollingsworth, Forrest Shull, Marvin V Zelkowitz, "Studying Code Development for High Performance Computing: The HPCS Program", 2004
Understanding the Impact of Assumptions on Experimental Validity
J Carver, J VanVoorhis, Victor R Basili, "Understanding the Impact of Assumptions on Experimental Validity", 2004
Reusing Knowledge on Software Quality for Developing Measurement Programs
O Jaufman, B Freimut, Ioana Rus, "Reusing Knowledge on Software Quality for Developing Measurement Programs", 2004
show abstract [+]
In order develop high quality software, companies must understand what constitutes quality for their products and stakeholders and then manage and engineer quality accordingly. To support this, the measurement of software quality is essential. Various measurement approaches and quality models have been proposed for this, such as the ISO9126 standard, the models proposed by Boehm and McCall, or the GQM approach. The problem however is, that the application of these approaches implicitly assumes that the measurement personnel has experience in defining and measuring software quality, an assumption that is not true for many software companies. Therefore we propose to address this problem by supporting quality measurement with reuse of previously acquired knowledge. We gathered and organized quality measurement knowledge in a knowledge repository and designed a quality definition process that allows elicitation of quality requirements from multiple stakeholders.
Software Best Practices Clearinghouse: Promoting Adoption and Effective Implementation
Kathleen Dangle, Tom McGibbon, Richard Turner, "Software Best Practices Clearinghouse: Promoting Adoption and Effective Implementation", 2004
Journals
Assurance cases: their use today and the challenges ahead.
Arnab Ray, "Assurance cases: their use today and the challenges ahead.", 46, (3):195, 198-200, 2012,
Link
An analysis of unit tests of a flight software product line
Dharmalingam Ganesan, Mikael Lindvall, David McComas, Maureen Bartholomew, Steve Slegel, Barbara Medina, Rene Krikhaar, Chris Verhoef, Lisa P. Montgomery, "An analysis of unit tests of a flight software product line", March 2012, 2012,
Link
Assurance cases: their use today and the challenges ahead.
Arnab Ray, "Assurance cases: their use today and the challenges ahead.", 46, (3):195, 198-200, 2012,
Link
Generic safety requirements for developing safe insulin pump software.
Yi Zhang, Raoul Jetley, Paul L Jones, Arnab Ray, "Generic safety requirements for developing safe insulin pump software.", 5, (6):1403-19, 2011,
Linkshow abstract [+]
The authors previously introduced a highly abstract generic insulin infusion pump (GIIP) model that identified common features and hazards shared by most insulin pumps on the market. The aim of this article is to extend our previous work on the GIIP model by articulating safety requirements that address the identified GIIP hazards. These safety requirements can be validated by manufacturers, and may ultimately serve as a safety reference for insulin pump software. Together, these two publications can serve as a basis for discussing insulin pump safety in the diabetes community.
Generic safety requirements for developing safe insulin pump software.
Yi Zhang, Raoul Jetley, Paul L Jones, Arnab Ray, "Generic safety requirements for developing safe insulin pump software.", 5, (6):1403-19, 2011,
Linkshow abstract [+]
The authors previously introduced a highly abstract generic insulin infusion pump (GIIP) model that identified common features and hazards shared by most insulin pumps on the market. The aim of this article is to extend our previous work on the GIIP model by articulating safety requirements that address the identified GIIP hazards. These safety requirements can be validated by manufacturers, and may ultimately serve as a safety reference for insulin pump software. Together, these two publications can serve as a basis for discussing insulin pump safety in the diabetes community.
Architectural Analysis of Systems Based on the Publisher-Subscriber Style
Dharmalingam Ganesan, Mikael Lindvall, Lamont Ruley, Robert Wiegand, Vuong Ly, Tina Tsui, "Architectural Analysis of Systems Based on the Publisher-Subscriber Style":173-182, 2010
Connecting Research and Practice: An Experience Report on Research Infusion with Software Architecture Visualization and Evaluation (SAVE)
Mikael Lindvall, William C Stratton, Deane E Sibol, Chris Ackermann, Mark Reid, Dharmalingam Ganesan, D McComas, M Bartholomew, Sally Godfrey, "Connecting Research and Practice: An Experience Report on Research Infusion with Software Architecture Visualization and Evaluation (SAVE)":1-23, 2010,
Link
Automatic requirement extraction from test cases
Chris Ackermann, Rance Cleaveland, Samuel Huang, Arnab Ray, Charles Shelton, Elizabeth Latronico, "Automatic requirement extraction from test cases":1-15, 2010,
Link
Automatic requirement extraction from test cases
Chris Ackermann, Rance Cleaveland, Samuel Huang, Arnab Ray, Charles Shelton, Elizabeth Latronico, "Automatic requirement extraction from test cases":1-15, 2010,
Link
Architectural Analysis of Systems Based on the Publisher-Subscriber Style
Dharmalingam Ganesan, Mikael Lindvall, Lamont Ruley, Robert Wiegand, Vuong Ly, Tina Tsui, "Architectural Analysis of Systems Based on the Publisher-Subscriber Style":173-182, 2010,
Link
Model-based engineering for medical-device software.
Arnab Ray, Raoul Jetley, Paul L Jones, Yi Zhang, "Model-based engineering for medical-device software.", 44, (6):507-18, 2010,
Linkshow abstract [+]
This paper demonstrates the benefits of adopting model-based design techniques for engineering medical device software. By using a patient-controlled analgesic (PCA) infusion pump as a candidate medical device, the authors show how using models to capture design information allows for i) fast and efficient construction of executable device prototypes ii) creation of a standard, reusable baseline software architecture for a particular device family, iii) formal verification of the design against safety requirements, and iv) creation of a safety framework that reduces verification costs for future versions of the device software. 1.
Model-based engineering for medical-device software.
Arnab Ray, Raoul Jetley, Paul L Jones, Yi Zhang, "Model-based engineering for medical-device software.", 44, (6):507-18, 2010,
Linkshow abstract [+]
This paper demonstrates the benefits of adopting model-based design techniques for engineering medical device software. By using a patient-controlled analgesic (PCA) infusion pump as a candidate medical device, the authors show how using models to capture design information allows for i) fast and efficient construction of executable device prototypes ii) creation of a standard, reusable baseline software architecture for a particular device family, iii) formal verification of the design against safety requirements, and iv) creation of a safety framework that reduces verification costs for future versions of the device software. 1.
Linking Software Development and Business Strategy Through Measurement
Victor R Basili, Mikael Lindvall, Myrna Regardie, Carolyn Seaman, Jens Heidrich, Jurgen Munch, Dieter Rombach, Adam Trendowicz, "Linking Software Development and Business Strategy Through Measurement", 43, (4):57-65, 2010,
Linkshow abstract [+]
The GQM+Strategies approach extends the goal/question/metric paradigm for measuring the success or failure of goals and strategies, adding enterprise-wide support for determining action on the basis of measurement results. An organization can thus integrate its measurement program across all levels.
An examination of change profiles in reusable and non-reusable software systems
A Gupta, Daniela Cruzes, Forrest Shull, Reidar Conradi, H Ronneberg, E Landre, "An examination of change profiles in reusable and non-reusable software systems", 22, (5):359-380, 2010,
Link
What Do We Know about Knowledge Management? Practical Implications for Software Engineering
Torgeir Dingsoyr, Finn Olav Bjornson, Forrest Shull, "What Do We Know about Knowledge Management? Practical Implications for Software Engineering", 26, (3):100-103, 2009,
Linkshow abstract [+]
There have been many claims about knowledge management's benefits in software engineering, such as decreased time and cost for development, increased quality, and better decision-making abilities. Although we can find some success stories illustrating these claims, particularly on aspects related to the systems and engineering schools, more research is necessary to explore the intersection between each school and the software engineering field. Researchers should continue to emphasize the need for a broad focus across multiple KM schools to suceed in improving KM's practical application in software engineering.
Engineering high confidence medical device software
Arnab Ray, Raoul Jetley, Paul Jones, "Engineering high confidence medical device software", 6, (2):1-7, 2009,
Link
What Do We Know about Agile Software Development?
Tore Dyba, Torgeir Dingsoyr, "What Do We Know about Agile Software Development?", 26, (5):6-9, 2009,
Linkshow abstract [+]
Agile software development has had a huge impact on how software is developed worldwide. We can view agile methods such as Extreme Programming (XP) and Scrum as a reaction to plan-based or traditional methods, which emphasize a "rationalized, engineering-based approach, incorporating extensive planning, codified processes, and rigorous reuse. In contrast, agile methods address the challenge of an unpredictable world, emphasizing the value competent people and their relationships bring to software development. To clarify the effectiveness of agile methods, we reviewed the agile development literature and conducted a systematic study of what we know empirically about its benefits and limitations.
Engineering high confidence medical device software
Arnab Ray, Raoul Jetley, Paul Jones, "Engineering high confidence medical device software", 6, (2):1-7, 2009,
Link
Measuring Developers: Aligning Perspectives and Other Best Practices
Medha Umarji, Forrest Shull, "Measuring Developers: Aligning Perspectives and Other Best Practices", 26, (6):92-94, 2009,
Linkshow abstract [+]
The paper discusses the software metrics programs. Software metrics programs might rank among the all-time touchiest subjects in software development. Done well, a measurement program can prove an effective tool for keeping on top of development effort-especially for large, distributed projects. It can help developers feel that they have a fair and objective way of communicating their progress and getting resources allocated where they're needed most. An effective metrics program must address both products and processes. Product-related metrics can be tied to customer satisfaction. They often represent objectively verifiable properties such as the number of bugs or defects found in a product. Process metrics, include measures of effort and effectiveness and can give quick feedback about the status of defect containment, productivity, and other desirable properties at many points during development. However, instituting these metrics poses different challenges that the authors discussed and developed some best practice solutions. These include design measures with multiple stakeholders, build on measures that come naturally out of existing processes, and transparency.
Maturing Software Engineering Knowledge through Classifications: A Case Study on Unit Testing Techniques
S. Vegas, N. Juristo, V.R. Basili, "Maturing Software Engineering Knowledge through Classifications: A Case Study on Unit Testing Techniques", 35, (4):551-565, 2009,
Linkshow abstract [+]
Classification makes a significant contribution to advancing knowledge in both science and engineering. It is a way of investigating the relationships between the objects to be classified and identifies gaps in knowledge. Classification in engineering also has a practical application; it supports object selection. They can help mature Software Engineering knowledge, as classifications constitute an organized structure of knowledge items. Till date, there have been few attempts at classifying in Software Engineering. In this research, we examine how useful classifications in Software Engineering are for advancing knowledge by trying to classify testing techniques. The paper presents a preliminary classification of a set of unit testing techniques. To obtain this classification, we enacted a generic process for developing useful Software Engineering classifications. The proposed classification has been proven useful for maturing knowledge about testing techniques, and therefore, SE, as it helps to: 1) provide a systematic description of the techniques, 2) understand testing techniques by studying the relationships among techniques (measured in terms of differences and similarities), 3) identify potentially useful techniques that do not yet exist by analyzing gaps in the classification, and 4) support practitioners in testing technique selection by matching technique characteristics to project characteristics.
To Game or Not to Game?
Christiane Gresse von Wangenheim, Forrest Shull, "To Game or Not to Game?", 26, (2):92-94, 2009,
Link
Linking Software Development and Business Strategy Through Measurement
Victor R Basili, Jens Heidrich, Mikael Lindvall, J Muench, Myrna Regardie, Dieter H Rombach, Carolyn Seaman, Adam Trendowicz, "Linking Software Development and Business Strategy Through Measurement", 43, (4):57 - 65, 2009,
Linkshow abstract [+]
The GQM+Strategies approach extends the goal/question/metric paradigm for measuring the success or failure of goals and strategies, adding enterprise-wide support for determining action on the basis of measurement results. An organization can thus integrate its measurement program across all levels.
Architecture compliance checking at run-time
Dharmalingam Ganesan, Thorsten Keuler, Yutaro Nishimura, "Architecture compliance checking at run-time", 51, (11):1586-1600, 2009,
Linkshow abstract [+]
In this paper, we report on our experiences with architecture compliance checking – the process of checking whether the planned or specified software architecture is obeyed by the running system – of an OSGi-based, dynamically evolving application in the office domain. To that end, we first show how to dynamically instrument a running system in the context of OSGi in order to collect run-time traces. Second, we explain how to bridge the abstraction gap between run-time traces and software architectures, through the construction of hierarchical Colored Petri nets (CP-nets). In addition, we demonstrate how to design reusable hierarchical CP-nets. In an industry example, we were able to extract views that helped us to identify a number of architecturally relevant issues (e.g., architectural style violations, behavior violations) that would not have been detected otherwise, and could have caused serious problems like system malfunctioning or unauthorized access to sensitive data. Finally, we package valuable experiences and lessons learned from this endeavor.
A Look at 25 Years of Data
Natalia Juristo, Ana Moreno, Sira Vegas, Forrest Shull, "A Look at 25 Years of Data", 26, (1):15-17, 2009,
Linkshow abstract [+]
Is 25 years enough time to build up a coherent body of knowledge that can help point to useful principles? As a testbed for helping us answer this question, software testing techniques are a good place to start. Few software practices are as important as testing, and testing techniques are amenable to measurement and reasoning about their effectiveness. Because they're aimed at removing faults, measuring the number and type of such removed faults seems like a natural part of applying these techniques. To make sense of this data, Universidad Politecnica de Madrid researchers have spent some time worrying about how to put 25 years' worth of work together usefully.
An update to experimental models for validating computer technology
Marvin V. Zelkowitz, "An update to experimental models for validating computer technology", 82, (3):373-376, 2009,
Linkshow abstract [+]
In 1998 a survey was published on the extent to which software engineering papers validate the claims made in those papers. The survey looked at publications in 1985, 1990 and 1995. This current paper updates that survey with data from 2000 to 2005. The basic conclusion is that the situation is improving. One earlier complaint that access to data repositories was difficult is becoming less prevalent and the percentage of papers including validation is increasing.
Measures and Risk Indicators for Early Insight Into Software Safety
Victor R Basili, Kathleen Dangle, Linda Esker, Ioana Rus, Frank Marotta, "Measures and Risk Indicators for Early Insight Into Software Safety", 21, (No. 10, Fault Tolerant Systems):4-8, 2008,
Linkshow abstract [+]
Software contributes an ever-increasing level of functionality and control in today's systems. This increased use of software can dramatically increase the complexity and time needed to evaluate the safety of a system. Although the actual safety of the system cannot be verified during its development, measures can reveal early insights into potential safety problems and risks. An approach for developing early software safety measures is presented here. The approach and the example software measures presented are based on experience working with the safety engineering group on a large Department of Defense program.
Bridging the Software Architecture Gap
Mikael Lindvall, Dirk Muthig, "Bridging the Software Architecture Gap", 41, (6):98-101, 2008,
Linkshow abstract [+]
There's a growing gap between the state of the art and the state of the practice in software architecture. The state of the art revolves around advanced architecture description and modeling languages, while in practice, the software architect - if there is one - has neither the power nor the tools to create a sound software architecture that is truly useful throughout the entire software life cycle. The SAVE tool helps developers align a system with the planned architecture.
Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer
Colin Campbell, Nikolai Tillmann, Wolfgang Grieskamp, Wolfram Schulte, Lev Nachmanson, "Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer", 4949:39-76, 2008
Inspecting the History of Inspections: An Example of Evidence-Based Technology Diffusion
Forrest Shull, Carolyn Seaman, "Inspecting the History of Inspections: An Example of Evidence-Based Technology Diffusion", 24, (7):88-90, 2008,
Linkshow abstract [+]
If we're going to have a column about evidence in software engineering, we're going to need to talk about inspections sooner or later. Inspections are among the most mature and perhaps best-studied practices in software engineering. In short, software inspection was one of those rare software engineering innovations that had the ability to effect real process change.
The ASC-Alliance Projects: A Case Study of Large-Scale Parallel Scientific Code Development
Lorin Hochstein, Victor R Basili, "The ASC-Alliance Projects: A Case Study of Large-Scale Parallel Scientific Code Development", 41, (3):50-58, 2008,
Linkshow abstract [+]
Computational scientists face many challenges when developing software that runs on large-scale parallel machines. However, software-engineering researchers haven't studied their software development processes in much detail. To better understand the nature of software development in this context, the authors examined five large-scale computational science software projects operated at the five ASC-Alliance centers.
A Pilot Study to Compare Programming Effort for Two Parallel Programming Models
Lorin Hochstein, Victor R Basili, U Vishkin, J Gilbert, "A Pilot Study to Compare Programming Effort for Two Parallel Programming Models", 81, (11):1920-1930, 2008,
Linkshow abstract [+]
Writing software for the current generation of parallel systems requires significant programmer effort, and the community is seeking alternatives that reduce effort while still achieving good performance.
Understanding the High Performance Computing Community: A Software Engineer's Perspective
Victor R Basili, J Carver, Daniela Cruzes, Lorin Hochstein, J K Hollingsworth, Forrest Shull, Marvin V Zelkowitz, "Understanding the High Performance Computing Community: A Software Engineer's Perspective", 25, (4):29-36, 2008,
Linkshow abstract [+]
Computational scientists developing software for HPC systems face unique software engineering issues. Attempts to transfer SE technologies to this domain must take these issues into account.
Measures and Risk Indicators for Early Insight into Software Safety
Victor Basili, Frank Marotta, Kathleen Dangle, Linda Esker, Ioana Rus, "Measures and Risk Indicators for Early Insight into Software Safety", (October):4-8, 2008
show abstract [+]
Software contributes an ever-increasing level of functionality and control in today's systems. This increased use of software can dramatically increase the complexity and time needed to evaluate the safety of a system. Although the actual system safety cannot be verified during its development, measures can reveal early insights into potential safety problems and risks. An approach for developing early software safety measures is presented in this article. The approach and the example software measures presented are based on experience working with the safety engineering group on a large Department of Defense program.
The Role of Replications in Empirical Software Engineering
J Carver, Natalie Juristo, Forrest Shull, S Vegas, "The Role of Replications in Empirical Software Engineering", 13, (2):211-218, 2008,
Link
Executable Specifications for Real-Time Distributed Systems
Arnab Ray, Rance Cleaveland, "Executable Specifications for Real-Time Distributed Systems", 203, (4):3-17, 2008,
Link
A Practical Approach for Quality-Driven Inspections
C Denger, Forrest Shull, "A Practical Approach for Quality-Driven Inspections", 24, (2):79-86, 2007,
Linkshow abstract [+]
Software inspection is a rigorous process for validating software work products that's both efficient and cost effective. However, this process presents challenges that might keep software developers from continuing to implement inspections. From our experience, major reasons for this are poor or missing customization of inspections for given context characteristics and insufficient stakeholder involvement. The TAQtIC (Tailoring Approach for Quality-Driven Inspections) inspection approach lets organizations implement inspections in a sustainable way in a given organizational context. Practitioners can use TAQtIC's underlying concepts to customize inspections for their environment. Experiences from projects using this approach demonstrate how organizations can tailor inspections of their software products.
Decision Support for Best Practices: Lessons Learned in Bridging the Gap between Research and Applied Practice
Raimund.L. Feldmann, Forrest Shull, Michele Shaw, "Decision Support for Best Practices: Lessons Learned in Bridging the Gap between Research and Applied Practice", 14, (1):234-247, 2007,
Linkshow abstract [+]
Today, everyone is looking at best practices for developing a system or making the right choice in acquiring system components. If the right best practices are applied, they help to avoid common problems and improve quality, cost, or both. However, finding and selecting an appropriate best practice is not always an easy endeavor. In most cases guidance, based on sound experience, is missing; often the best practice is too new, still under study, or the existing experiences do not fit the user's context. This article reports on a program that tries to bridge the gap between rigorous empirical research and practical needs for guiding practitioners in selecting appropriate best practices.
The Quality of Knowledge: Knowledge Patterns and Knowledge Refactorings
Joerg Rech, Bjoern Decker, Eric Ras, Andreas Jedlitschka, Raimund.L. Feldmann, "The Quality of Knowledge: Knowledge Patterns and Knowledge Refactorings", 3, (3):74-103, 2007,
Linkshow abstract [+]
Knowledge management is a relatively young discipline. Nevertheless, it has accumulated a valuable body-of-knowledge in the structuring of knowledge and in the design of socio-technical knowledge management systems. However, concepts to describe common, recurring patterns of how to describe, structure, interrelate, group, or manage knowledge elements are still missing. In this paper, we introduce the concepts “knowledge pattern” and “knowledge anti-pattern” to describe best and worst practices in knowledge management, “knowledge refactoring” to improve or change knowledge anti-patterns, and “quality of knowledge” to describe desirable characteristics of knowledge in knowledge management systems. The concepts are transferred from software engineering to the field of knowledge management based on our experience from several knowledge management projects.
Skoll: A process and infrastructure for distributed continuous quality assurance
A Porter, A M Yilmaz, A Memon, D C Schmidth, B Natarajan, "Skoll: A process and infrastructure for distributed continuous quality assurance", 33, (8):510-525, 2007,
Linkshow abstract [+]
Software engineers increasingly emphasize agility and flexibility in their designs and development approaches. They increasingly use distributed development teams, rely on component assembly and deployment rather than green field code writing, rapidly evolve the system through incremental development and frequent updating, and use flexible product designs supporting extensive end-user customization. While agility and flexibility have many benefits, they also create an enormous number of potential system configurations built from rapidly changing component implementations. Since today's quality assurance (QA) techniques do not scale to handle highly configurable systems, we are developing and validating novel software QA processes and tools that leverage the extensive computing resources of user and developer communities in a distributed, continuous manner to improve software quality significantly. This paper provides several contributions to the study of distributed, continuous QA (DCQA). First, it shows the structure and functionality of Skoll, which is an environment that defines a generic around-the-world, around-the-clock QA process and several sophisticated tools that support this process. Second, it describes several novel QA processes built using the Skoll environment. Third, it presents two studies using Skoll: one involving user testing of the Mozilla browser and another involving continuous build, integration, and testing of the ACE+TAO communication software package. The results of our studies suggest that the Skoll environment can manage and control distributed continuous QA processes more effectively than conventional QA processes. For example, our DCQA processes rapidly identified problems that had taken the ACE+TAO developers much longer to find and several of which they had not found. Moreover, the automatic analysis of QA results provided developers information that enabled them to quickly find the root causes of problems.
Protocols in the use of Empirical Software Engineering Artifacts
Victor R Basili, Marvin V Zelkowitz, Sjoberg D.I.K., Johnson Philip, Cowling A.J., "Protocols in the use of Empirical Software Engineering Artifacts", 12, (1):107-119, 2007,
Linkshow abstract [+]
If empirical software engineering is to grow as a valid scientific endeavor, the ability to acquire, use, share, and compare data collected from a variety of sources must be encouraged. This is necessary to validate the formal models being developed within computer science. However, within the empirical software engineering community this has not been easily accomplished. This paper analyses experiences from a number of projects, and defines the issues, which include the following: (1) How should data, testbeds, and artifacts be shared? (2) What limits should be placed on who can use them and how? How does one limit potential misuse? (3) What is the appropriate way to credit the organization and individual that spent the effort collecting the data, developing the testbed, and building the artifact? (4) Once shared, who owns the evolved asset? As a solution to these issues, the paper proposes a framework for an empirical software engineering artifact agreement. Such an agreement is intended to address the needs for both creator and user of such artifacts and should foster a market in making available and using such artifacts. If this framework for sharing software engineering artifacts is commonly accepted, it should encourage artifact owners to make the artifacts accessible to others (gaining credit is more likely and misuse is less likely). It may be easier for other researchers to request artifacts since there will be a well-defined protocol for how to deal with relevant matters.
Eliminating synchronization faults in air traffic control software via design for verification with concurrency controllers
Aysu Betin-Can, Tevfik Bultan, Mikael Lindvall, Benjamin Lux, Stefan Topp, "Eliminating synchronization faults in air traffic control software via design for verification with concurrency controllers", 14, (2):129-178, 2007,
Linkshow abstract [+]
The increasing level of automation in critical infrastructures requires development of effective ways for finding faults in safety critical software components. Synchronization in concurrent components is especially prone to errors and, due to difficulty of exploring all thread interleavings, it is difficult to find synchronization faults. In this paper we present an experimental study demonstrating the effectiveness of model checking techniques in finding synchronization faults in safety critical software when they are combined with a design for verification approach. We based our experiments on an automated air traffic control software component called the Tactical Separation Assisted Flight Environment (TSAFE). We first reengineered TSAFE using the concurrency controller design pattern. The concurrency controller design pattern enables a modular verification strategy by decoupling the behaviors of the concurrency controllers from the behaviors of the threads that use them using interfaces specified as finite state machines. The behavior of a concurrency controller is verified with respect to arbitrary numbers of threads using the infinite state model checking techniques implemented in the Action Language Verifier (ALV). The threads which use the controller classes are checked for interface violations using the finite state model checking techniques implemented in the Java Path Finder (JPF). We present techniques for thread isolation which enables us to analyze each thread in the program separately during interface verification. We conducted two sets of experiments using these verification techniques. First, we created 40 faulty versions of TSAFE using manual fault seeding. During this exercise we also developed a classification of faults that can be found using the presented design for verification approach. Next, we generated another 100 faulty versions of TSAFE using randomly seeded faults that were created automatically based on this fault classification. We used both infinite and finite state verification techniques for finding the seeded faults. The results of our experiments demonstrate the effectiveness of the presented design for verification approach in eliminating synchronization faults.
Experimenting with Software Testbeds for Evaluating New Technologies
Mikael Lindvall, Ioana Rus, Paolo Donzelli, A Memon, Marvin V Zelkowitz, Aysu Betin-Can, Tevfik Bultan, Chris Ackermann, Bettina Anders, Sima Asgari, Victor R Basili, Joerg Fellman, Daniel Hirschbach, Lorin Hochstein, Forrest Shull, Roseanne Tesoriero Tvedt, Dan Pech, "Experimenting with Software Testbeds for Evaluating New Technologies", 12, (4):417-444, 2007,
Linkshow abstract [+]
The evolution of a new technology depends upon a good theoretical basis for developing the technology, as well as upon its experimental validation. In order to provide for this experimentation, we have investigated the creation of a software testbed and the feasibility of using the same testbed for experimenting with a broad set of technologies. The testbed is a set of programs, data, and supporting documentation that allows researchers to test their new technology on a standard software platform. An important component of this testbed is the Unified Model of Dependability (UMD), which was used to elicit dependability requirements for the testbed software. With a collection of seeded faults and known issues of the target system, we are able to determine if a new technology is adept at uncovering defects or providing other aids proposed by its developers. In this paper, we present the Tactical Separation Assisted Flight Environment (TSAFE) testbed environment for which we modeled and evaluated dependability requirements and defined faults to be seeded for experimentation. We describe two completed experiments that we conducted on the testbed. The first experiment studies a technology that identifies architectural violations and evaluates its ability to detect the violations. The second experiment studies model checking as part of design for verification. We conclude by describing ongoing experimental work studying testing, using the same testbed. Our conclusion is that even though these three experiments are very different in terms of the studied technology, using and re-using the same testbed is beneficial and cost effective.
Getting Requirements Right: The Perspective-Based Reading Technique and the Rational Unified Process
P Costa, Forrest Shull, W Melo, "Getting Requirements Right: The Perspective-Based Reading Technique and the Rational Unified Process", 2006,
Linkshow abstract [+]
This article presents a method for requirements inspection that incorporates an extended version of the Perspective-Based Reading (PBR) technique in combination with the Requirements discipline portion of the Rational Unified Process (RUP). Extensive examples of the proposed approach are provided, along with case study data to illustrate its cost and benefits.
Experiments to Understand HPC Time to Development
Lorin Hochstein, Victor R Basili, Sima Asgari, Marvin V Zelkowitz, J K Hollingsworth, Forrest Shull, J Carver, Voelp M., Zazaworka Nico, Johnson Philip, "Experiments to Understand HPC Time to Development", 2, (4A):24-32, 2006,
Linkshow abstract [+]
In order to understand how high performance computing (HPC) programs are developed, a series of experiments, using students in graduate level HPC classes, have been conducted at many universities in the US. In this paper we discuss the general process of conducting those experiments, give some of the early results of those experiments, and describe a web-based process we are developing that will allow us to run additional experiments at other universities and laboratories that will be easier to conduct and generate results that more accurately reflect the process of building HPC programs.
Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness
J C Maldonado, J Carver, Forrest Shull, S Fabbri, E Doria, L A F Martimiano, M Mendonca, Victor R Basili, "Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness", 11, (1):119-142, 2006,
Linkshow abstract [+]
This paper describes a replication conducted to compare the effectiveness of inspectors using Perspective Based Reading (PBR) to the effectiveness of inspectors using a checklist. The goal of this replication was to better understand the complementary aspects of the PBR perspectives. To this end, a brief discussion of the original study is provided as well as a more detailed description of the replication. A detailed statistical analysis is then provided along with analysis of the PBR perspectives. For the individual PBR perspectives, we saw an interesting dichotomy: In the original study there was little overlap among the sets of defects found by each of the three perspectives, while in the replication two of the three perspectives found similar sets of defects on one of the two documents used in the study. Interestingly this document was the only case where the users of PBR were not more effective than the users of a checklist. This result leads to a new hypothesis that the complementary aspect of the PBR perspectives is the characteristic that provides the benefit over other defect detection techniques.
Observations about Software Development for High End Computing
J Carver, Lorin Hochstein, Kendall Richard, T Nakamura, Marvin V Zelkowitz, Victor R Basili, Post Douglass, "Observations about Software Development for High End Computing", 2, (4A):33-38, 2006,
Linkshow abstract [+]
This paper describes observations about software development for high end computing that we have made from several environments. We conducted a series of case studies of different types of codes, from academic codes to codes from governmental agencies. Based on those studies, we have formed a series of observations, some common and some different across these environments. This paper discusses each of those observations along with the supporting information from the different environments.
Victor R. Basili's Contributions to Software Quality
Forrest Shull, Carolyn Seaman, Marvin V Zelkowitz, "Victor R. Basili's Contributions to Software Quality", 23, (1):16-18, 2006,
Linkshow abstract [+]
This article deals with Victor R. Basili's contributions to software quality. Basili's contributions cover three broad areas: research in the 1970s and early 1980s on software measurement and the Goal Question Metric (GQM) model, research in the 1980s and 1990s on these measurement ideas' maturation into a software engineering model of empirical studies, including the development of the Quality Improvement Paradigm (QIP) and the influence of the NASA Goddard Space Flight Center Software Engineering Laboratory, and research since 1990 in the Experience Factory as a model for creating learning organizations for continuous software process improvement. Some of Basili's most important contributions are in measuring software development processes and products and gifted the community with an invaluable tool: the GQM approach. The GQM approach is based on the assumption that for an organization to measure its products and processes usefully, it must first specify goals for itself and its projects
Finding and Fixing Problems Early: A Perspective-Based Approach to Requirements and Design Inspections
J Carver, Forrest Shull, Ioana Rus, "Finding and Fixing Problems Early: A Perspective-Based Approach to Requirements and Design Inspections", 19, (12):25-28, 2006
show abstract [+]
Viewing security vulnerabilities as a specific type of software defect allows proven software engineering techniques for finding and fixing them to be used early in the development of the product. Finding and fixing these problems early (i.e. at the requirements or design phase) will reduce the overall risk and cost of the product. This article describes the application of a previously successful early life cycle software inspection approach (perspective-based reading [PBR] ) to the problem of software security. Excerpts from this tailored approach are provided along with guidance on it use.
Can Observational Techniques Help Novices Overcome the Software Inspection Learning Curve?
J Carver, Forrest Shull, Victor R Basili, "Can Observational Techniques Help Novices Overcome the Software Inspection Learning Curve?", 11, (4):523-539, 2006,
Linkshow abstract [+]
When learning a new software engineering technique, there is a learning curve that must be overcome. In particular, when studies are conducted in a classroom setting, researchers need a method for quickly accelerating the experience of novice subjects to allow the results to be more applicable in industrial settings. In this paper, we propose and test a method to enable novices to gain process experience to allow them to more quickly overcome the learning curve. The method we evaluate allows an inspector to gain experience with the inspection process by observing an inspection performed by someone else. The results of the study show that the proposed method for gaining experience appears to be useful in some limited cases, that is, for low experienced subjects who were inspecting a requirements document from a domain in which they had low knowledge. Based on the results of this study, we are able to propose some new related hypotheses to be tested in future studies.
A Practical Framework for Eliciting and Modeling System Dependability Requirements: Experience from the NASA High Dependability Computing Project
Paolo Donzelli, Victor R Basili, "A Practical Framework for Eliciting and Modeling System Dependability Requirements: Experience from the NASA High Dependability Computing Project", 79, (1):107-119, 2006,
Link
Evaluating COTS Component Dependability: The case of real-time Java virtual machiine for spacecraft software
Paolo Donzelli, Marvin V Zelkowitz, Victor R Basili, Dan Allard, Kenneth N Meyer, "Evaluating COTS Component Dependability: The case of real-time Java virtual machiine for spacecraft software", 22, (4):46-53, 2005,
Linkshow abstract [+]
The software industry has increasingly expanded its adoption of COTS components for complex, mission-critical applications. Using COTS products can shorten development and deployment time because they let system developers focus on creating domain-specific services. Selecting the right COTS component, however, is no easy task. We present a practical process that developers can use to empirically evaluate component dependability in their context. Our approach uses the unified model of dependability, a requirements engineering approach specially devised to capture dependability in context. This model clearly specifies the measurable characteristics the component must have to be dependable in a specific context. The model then serves as a reference, providing guidance on effectively designing experiments to compare similar components and interpret collected data. The process can be applied to any specific context and COTS component. Here, we describe our comparison of the dependability of real-time Java virtual machines (RT JVM) in the spacecraft software context.
Software Process Improvement in Small Organizations: A Case Study
Kathleen Dangle, Patricia E Larsen, Michele Shaw, Marvin V Zelkowitz, "Software Process Improvement in Small Organizations: A Case Study", 22, (6):68-75, 2005,
Linkshow abstract [+]
The Capability Maturity Model for software has become a major force in software development process improvement. We looked at the role of process improvement in the context of a small organization. Although the Capability Maturity Model integration is replacing the CMM, we focused on the older CMM. We had to look at more than which CMM key process areas apply to small businesses. Our overall goal was to institute good software development practices, we used the language of the CMM to gain the management's cooperation. Applying these practices is essential to managing growth, yet undertaking this effort without prior experience could impede a small company's innovative nature. This case study's purpose was to investigate the success factors of a software process improvement effort for a small software development organization.
Combating Architectural Degeneration: A Survey
Lorin Hochstein, Mikael Lindvall, "Combating Architectural Degeneration: A Survey", 47, (10):644-656, 2005,
Linkshow abstract [+]
As software systems evolve over time, they invariably undergo changes that can lead to a degeneration of the architecture. Left unchecked, degeneration may reach a level where a complete redesign is necessary, a task that requires significant effort. In this paper, we present a survey of technologies developed by researchers that can be used to combat degeneration, that is, technologies that can be employed in identifying, treating and researching degeneration. We also discuss the various causes of degeneration and how it can be prevented.
Productivity measures for high performance computers
Marvin V Zelkowitz, Victor R Basili, Sima Asgari, Lorin Hochstein, J K Hollingsworth, T Nakamura, "Productivity measures for high performance computers", 2005,
Link
Introducing the Department of Defense Acquisition Best Practice Clearinghouse
Kathleen Dangle, Laura Dwinnell, John Hickok, Richard Turner, "Introducing the Department of Defense Acquisition Best Practice Clearinghouse", 18, (5):4-5, 2005,
Link
An Evolutionary Testbed for Software Technology Evaluation
Mikael Lindvall, Ioana Rus, Forrest Shull, Marvin V Zelkowitz, Paolo Donzelli, A Memon, Victor R Basili, Patricia Costa, Roseanne Tesoriero Tvedt, Lorin Hochstein, Sima Asgari, Chris Ackermann, Dan Pech, "An Evolutionary Testbed for Software Technology Evaluation", 1, (1):3-11, 2005,
Link
An Algebraic Theory Of Boundary Crossing Transitions
Arnab Ray, Rance Cleaveland, Arne Skou, "An Algebraic Theory Of Boundary Crossing Transitions", 115:69-88, 2005,
Link
Simulating Families of Studies to Build Confidence in Defect Hypotheses
Forrest Shull, Daniela Cruzes, Victor R Basili, M Mendonca, "Simulating Families of Studies to Build Confidence in Defect Hypotheses", 47, (15):1019-1032, 2005,
Linkshow abstract [+]
While it is clear that there are many sources of variation from one development context to another, it is not clear a priori what specific variables will influence the effectiveness of a process in a given context. For this reason, we argue that knowledge about software process must be built from families of studies, in which related studies are run within similar contexts as well as very different ones. Previous papers have discussed how to design related studies so as to document as precisely as possible the values of likely context variables and be able to compare with those observed in new studies. While such a planned approach is important, we argue that an opportunistic approach is also practical. The approach would combine results from multiple individual studies after the fact, enabling recommendations to be made about process effectiveness in context. In this paper, we describe two processes with which we have been working to build empirical knowledge about software development processes: one is a manual and informal approach, which relies on identifying common beliefs or ‘folklore’ to identify useful hypotheses and a manual analysis of the information in papers to investigate whether there is support for those hypotheses; the other is a formal approach based around encoding the information in papers into a structured hypothesis base that can then be searched to organize hypotheses and their associated support. We test these processes by applying them to build knowledge in the area of defect folklore (i.e. commonly accepted heuristics about software defects and their behavior). We show that the formal methodology can produce useful and feasible results, especially when it is compared to the results output from the more manual, expert-based approach. The formalized approach, by relying on a reusable hypothesis base, is repeatable and also capable of producing a more thorough basis of support for hypotheses, including results from papers or articles that may have been overlooked or not considered by the experts.
Formal Modeling Of Middleware-based Distributed Systems
Arnab Ray, Rance Cleaveland, "Formal Modeling Of Middleware-based Distributed Systems", 108:21-37, 2004,
Link
Agile Software Development in Large Organizations
Mikael Lindvall, Dirk Muthig, Aldo Dagnino, Christina Wallin, Mikael Stupperich, David Kiefer May, John, Tuomo K�hk�nen, "Agile Software Development in Large Organizations", 37, (12):26-34, 2004,
Linkshow abstract [+]
Developers need evidence that a new technology works in a certain context before they promote and deploy it on a larger scale. This need looms greater in large organizations because of their complexity and the need to integrate new technologies and processes with existing ones. To further evaluate agile methods and their underlying software development practices, several Software Experience Center member companies initiated a series of activities to discover if agile practices match their organizations' needs. Based on the experiences of these organizations, researchers concluded that agile practices match the needs of large organizations, but integrating new practices with existing processes and quality systems that govern the conduct of software development requires further tailoring. The challenge here lies not in applying agile practices to a project, but in efficiently integrating the agile project into its environment.
Defect evolution in a product line environment
Marvin V Zelkowitz, Ioana Rus, "Defect evolution in a product line environment", 70, (1-2):143-154, 2004,
Linkshow abstract [+]
One mechanism used for monitoring the development of the Space Shuttle flight control software, in order to minimize any risks to the missions, is the independent verification and validation (IV&V) process. Using data provided by both the Shuttle software developer and the IV&V contractor, in this paper we describe the overall IV&V process as used on the Space Shuttle program and provide an analysis of the use of metrics to document and control this process over multiple releases of this software. Our findings reaffirm the value of IV&V, show the impact of IV&V on multiple releases of a large complex software system, and indicate that some of the traditional measures of defect detection and repair are not applicable in a multiple-release environment such as this one.
Susceptibility matrix: A new aid to software auditing
K Jiwnani, Marvin V Zelkowitz, "Susceptibility matrix: A new aid to software auditing", 2, (2):16-21, 2004
A Unified Model of Dependability: Capturing Dependability in Context
Victor R Basili, Paolo Donzelli, Sima Asgari, "A Unified Model of Dependability: Capturing Dependability in Context", 21, (6):19 - 25, 2004,
Linkshow abstract [+]
In contemporary societies, individuals and organizations increasingly depend on services delivered by sophisticated software-intensive systems to achieve personal and business goals. So, a system must have engineered and guaranteed dependability, regardless of continuous, rapid, and unpredictable technological and context changes. The International Federation for Information Processing Working Group defines dependability as "the trustworthiness" of a computing system, which allows reliance to be justifiably placed on the services it deliver.
Knowledge-sharing issues in experimental software engineering
Forrest Shull, M Mendonca, Victor R Basili, J Carver, J C Maldonado, S Fabbri, G H Travassos, Maria Ferreira, "Knowledge-sharing issues in experimental software engineering", 9, (1):111-137, 2004,
Linkshow abstract [+]
Recently the awareness of the importance of replicating studies has been growing in the empirical software engineering community. The results of any one study cannot simply be extrapolated to all environments because there are many uncontrollable sources of variation between different environments. In our work, we have reasoned that the availability of laboratory packages for experiments can encourage better replications and complementary studies. However, even with effectively specified laboratory packages, transfer of experimental know-how can still be difficult. In this paper, we discuss the collaboration structures we have been using in the Readers Project, a bilateral project supported by the Brazilian and American national science agencies that is investigating replications and transfer of experimental know-how issues. In particular, we discuss how these structures map to the Nonaka–Takeuchi knowledge sharing model, a well-known paradigm used in the knowledge management literature. We describe an instantiation of the Nonaka–Takeuchi Model for software engineering experimentation, establishing a framework for discussing knowledge sharing issues related to experimental software engineering. We use two replications to illustrate some of the knowledge sharing issues we have faced and discuss the mechanisms we are using to tackle those issues in Readers Project.
Reports
Automatic Test Case Generation and Parallel Test Case Execution at NASA
Dharmalingam Ganesan, Mikael Lindvall, "Automatic Test Case Generation and Parallel Test Case Execution at NASA", 2011
Evaluation of Systems Engineering Methods, Processes and Tools on Department of Defense and Intelligence Community Programs
R. Turner, F. Shull, B. Boehm, A. Carrigy, L. Clarke, P. Componation, C. Dagli, J. Lane, L. Layman, A. Miller, S. O’Brien, L. Osterweil, D. Sabados, S. Wise, "Evaluation of Systems Engineering Methods, Processes and Tools on Department of Defense and Intelligence Community Programs", 2009
show abstract [+]
Final Technical Report
An Approach to Improving Parametric Estimation Models in the Case of Violation Assumptions Based Upon Risk Analysis
S.A. Sarcia, V.R. Basili, G. Cantone, "An Approach to Improving Parametric Estimation Models in the Case of Violation Assumptions Based Upon Risk Analysis", 2009,
Link
Evaluating Software Design in Various Implementations of TSAFE
Thomas Asel, Mikael Lindvall, Chris Ackermann, "Evaluating Software Design in Various Implementations of TSAFE", 2006
An Analysis of Software Flexibility in Seven Different Implementations of a Command Line Interpreter
Bettina Anders, Joerg Fellman, Mikael Lindvall, Ioana Rus, "An Analysis of Software Flexibility in Seven Different Implementations of a Command Line Interpreter", 2006
Inspections and Pair Programming - competing or complementary? Experiences from an Expert eWorkshop
R Carbon, M Ciolkowski, C Denger, Mikael Lindvall, Forrest Shull, Patricia Costa, Dieter H Rombach, Victor R Basili, "Inspections and Pair Programming - competing or complementary? Experiences from an Expert eWorkshop", 2005
show abstract [+]
The common goal of the two practices Pair Programming and Inspections is to produce high quality software. Even though they have a common goal, their approaches are different, and they are typically used in different situations. Pair programming is typically applied as a part of agile development methodologies, such as Extreme Programming, whereas Inspections are often applied in plan-driven or CMM-based methodologies. In order to gain a better understanding of these two practices and their strengths and weaknesses, we facilitated an expert eWorkshop. Our goals were to compare the two practices as well as to understand in which situations the two practices can be best applied. Our eWorkshop discussion highlighted several differences in the benefits that can be expected from the practices (along dimensions such as objectivity of review and achievable level of quality), indicating that the practices can be considered complementary in order to achieve the full range of effects.
A Pilot Study to Evaluate Development Effort for High Performance Computing
Victor R Basili, Sima Asgari, J Carver, Lorin Hochstein, J K Hollingsworth, Forrest Shull, Marvin V Zelkowitz, "A Pilot Study to Evaluate Development Effort for High Performance Computing", 2004
show abstract [+]
The ability to write programs that execute efficiently on modern parallel computers has not been fully studied. In a DARPA-sponsored project, we are looking at measuring the development time for programs written for high performance computers (HPC). To attack this relatively novel measurement problem, our goal is to initially measure such development time in student programming to evaluate our own experimental protocols. Based on these results, we will generate a set of feasible experimental methods that can then be applied with more confidence to professional expert programmers.
Data Analysis of the First Two PBR Replications in Readers Project
S Fabbri, J C Maldonado, E N Hohn, L A F Martimiano, M Mendonca, Forrest Shull, J Carver, Victor R Basili, "Data Analysis of the First Two PBR Replications in Readers Project", 2004
Evaluation of Software Architectures with Eclipse
Paul Miodonski, Thomas, Jens Knodel, Mikael Lindvall, Dirk Muthig, "Evaluation of Software Architectures with Eclipse", 2004,
Linkshow abstract [+]
The thesis deals with software architecture process of evaluation, with the goal to evaluate it according to given constraints (e.g., to determine the quality of the architecture, assure that requirements are addressed, fulfill business goals). The evaluation of the software architecture can be a part of the reengineering process, but where an existing system is being examined and altered, it has to be part of the process. The lack of the software architecture force to reconstruct the architecture of the existing system and if the software architecture exists, it hat to be verified against the implementation of the system. The intention of the diploma thesis is to develop a tool that closes the gap between pre- and post-implementation phase architectural evaluation. The objectives are to evaluate architectural rules and principles while developers are im- plementing the system. The ongoing evaluation accompanying all development activities has the following benefits: - immediately raise of awareness - enabling to decide about architectural problems when they occur and when still in the development context - ability to communicate architectural flaws directly when they appear and flowback of information from developers to the architects The tool name is SAVE. This is an acronym for Software Architecture Visualization and Evaluation. The tool is an Eclipse plug-in which evaluates the given software architecture (High Level Model) with the source code (Source Code Model) and points out the differences between these two artifacts. The results of the comparison are three sets: - Convergences - Divergences - Absences The goal of the comparison between the High Level Model (HLM) and the Source Code Model (SCM) is to achieve higher quality of both, the software architecture of the system and the implementation. Higher consistency between these two models gives the development team more control about the system. SAVE gives the developer the control of the consistency by comparing these two models on-the-fly in the background and pointing out the violations. To save resources this is achieved by computing only the delta between HLM and SCM, this means only the changed code and its effects on the models are recomputed. The work achieves in detail the following goals: 1. ongoing software architecture evaluation during implementation shown to the user to indicate the effects of the delta. 2. software architecture evaluation during implementation computation, which is only preformed once; the result of the computation is presented in a graphical manner to the user. 3. smooth IDE integration to give the developer most possible comfort using the tool all functions are integrated in one IDE; additional achievements are: - Language independence - the idea is to extract information out of the code in form of XMI (XML Metadata Interchange) and perform the analysis on this data in order to achieve language independence; the diploma thesis focuses on the Java language - Support for the component browser - the plug-in supports the component browser plug-in (another tool developed at IESE to support the management of components present in a system) in order to import the High Level Model and the mapping. The component browser is an external application/Eclipse plug-in - XMI Repository - every kind of model will be stored in XMI format, this format enables the postprocessing of results (i.e., evaluation results can be analyzed by other tools. 4. traceability (Code Navigation)
Book Sections
Chapter 6 - Functional and Nonfunctional Design Verification for Embedded Software Systems
Arnab Ray, Christopher Ackermann, Rance Cleaveland, Charles Shelton, Chris Martin, , Chapter 6 - Functional and Nonfunctional Design Verification for Embedded Software SystemsMarvin V Zelkowitz B T - Advances in Computers, eds., pp. 277-321, 2011,
Link
Functional and Nonfunctional Design Verification for Embedded Software Systems
Arnab Ray, Christopher Ackermann, Rance Cleaveland, Charles Shelton, Chris Martin, , Functional and Nonfunctional Design Verification for Embedded Software SystemsMarvin V Zelkowitz B T - Advances in Computers, eds., pp. 277-321, 2011,
Link
Evaluating the Modifiability of Software Architectural Designs
Omolade Saliu, Guenther Ruhe, Mikael Lindvall, Chris Ackermann, , Evaluating the Modifiability of Software Architectural Designs, pp. 243-297, 2009,
Linkshow abstract [+]
In this chapter, we propose an architectural design evaluation technique called EBEAM (Expert-Based Evaluation of Architecture for Modifiability) that assists experts in articulating their knowledge of architectural designs and expressing the knowledge in measurable terms. EBEAM supports the evaluation of different architectural design versions for modifiability. In addition, EBEAM supports relative comparison between these design versions and the target design. We develop EBEAM as a generalized technique that is reusable for evaluating other architectural design attributes, apart from modifiability. We discuss EBEAM in detail and report on two case studies that investigate its applicability, and one study that validates the results of the evaluations made using EBEAM
Using Semantic Networks and Context in Search for Relevant Software Engineering Artifacts
George Karabatis, Zhiyuan Chen, Vandana P Janeja, Tania Lobo, Monish Advani, Mikael Lindvall, Raimund.L. Feldmann, , Using Semantic Networks and Context in Search for Relevant Software Engineering Artifacts, pp. 74-104, 2009
show abstract [+]
The discovery of relevant software artifacts can increase software reuse and reduce the cost of software development and maintenance. Furthermore, change requests, which are a leading cause of project failures, can be better classified and handled when all relevant artifacts are available to the decision makers. However, traditional full-text and similarity search techniques often fail to provide the full set of relevant documents because they do not take into consideration existing relationships between software artifacts. We propose a metadata approach with semantic networks which convey such relationships. Our approach reveals additional relevant artifacts that the user might have not been aware of. We also apply contextual information to filter out results unrelated to the user contexts, thus, improving the precision of the search results. Experimental results show that the combination of semantic networks and context significantly improve the precision and recall of the search results.
Knowledge Patterns and Knowledge Refactorings for Increasing the Quality of Knowledge
Joerg Rech, Raimund.L. Feldmann, Eric Ras, Andreas Jedlitschka, Bjoern Decker, , Knowledge Patterns and Knowledge Refactorings for Increasing the Quality of KnowledgeMurray E.Jennex, eds., pp. 281, 2008
show abstract [+]
Knowledge management is a relatively young discipline. It has accumulated a valuable body-of-knowledge on how to structure and represent knowledge, or how to design socio-technical knowledge management systems. A wide variety of approaches and systems exist that are often not interoperable, and hence, prevent an easy exchange of the gathered knowledge. Industry standards, which have been accepted and are in widespread use are missing, as well as general concepts to describe common, recurring patterns of how to describe, structure, interrelate, group, or manage knowledge elements. In this chapter, we introduce the concepts "knowledge pattern" and "knowledge anti-pattern" to describe best and worst practices in knowledge management, "knowledge refactoring" to improve or change knowledge antipatterns, and "quality of knowledge" to describe desirable characteristics of knowledge in knowledge management systems. The concepts are transferred from software engineering to the field ofknowledge management based on our experience from several knowledge management projects.
An Environment for Conducting Families of Software Engineering Experiments
Lorin Hochstein, T Nakamura, Forrest Shull, Zazaworka Nico, Victor R Basili, Marvin V Zelkowitz, , An Environment for Conducting Families of Software Engineering Experiments, New York, pp. 175-200, 2008,
Linkshow abstract [+]
The classroom is a valuable resource for conducting software engineering experiments. However, coordinating a family of experiments in classroom environments presents a number of challenges to researchers. Understanding how to run such experiments, developing procedures to collect accurate data, and collecting data that is consistent across multiple studies are major problems. This paper describes an environment, the Experiment Manager that simplifies the process of collecting, managing, and sanitizing data from classroom experiments, while minimizing disruption to natural subject behavior. We have successfully used this environment to study the impact of parallel programming languages in the high-performance computing domain on programmer productivity at multiple universities across the United States.
Authorization, Access Control and Information Flow
Arnab Ray, , Authorization, Access Control and Information Flow, pp. 213-223, 2008
Building Theories from Multiple Evidence Sources
Forrest Shull, Raimund.L. Feldmann, , Building Theories from Multiple Evidence SourcesForrest Shull, Janice Singer, Sjoberg D.I.K., eds., London, pp. 337-364, 2008
show abstract [+]
As emphasized in other chapters of this book, useful results in empirical software engineering require a variety of data to be collected through different studies – focusing on a single context or single metric rarely tells a useful story. But, in each study, the requirements of the local context are liable to impose different constraints on study design, the metrics to be collected, and other factors. Thus, even when all the studies focus on the same phenomenon (say, software quality), such studies can validly collect a number of different measures that are not at all compatible (say, number of defects required to be fixed during development, number of problem reports received from the customer, total amount of effort that needed to be spent on rework). Can anything be done to build a useful body of knowledge from these disparate pieces? This chapter addresses strategies that have been applied to date to draw conclusions from across such varied but valid data sets. Key approaches are compared and the data to which they are best suited are identified. Our analysis together with associated lessons learned provide decision support for readers interested in choosing and using such approaches to build up useful theories.
The Role of Controlled Experiments in Software Engineering Research
Victor R Basili, , The Role of Controlled Experiments in Software Engineering Research V.Basili, eds., pp. 33-37, 2007,
Linkshow abstract [+]
Empirical studies play an important role in the evolution of the software engineering discipline. They allow us to build a body of knowledge in software engineering that has been supported by observation and empirical evidence. These studies allow us to test out theories, identify important variables and to build models that can be supported by empirical evidence.
Evolving Defect 'Folklore': A Cross-Study Analysis of Software Defect Behavior
Victor R Basili, Forrest Shull, , Evolving Defect 'Folklore': A Cross-Study Analysis of Software Defect Behavior, Berlin, Germany, pp. 1-9, 2005,
Linkshow abstract [+]
Answering “macro-process” research issues – which require understanding how development processes fit or do not fit in different organizational systems and environments – requires families of related studies. While there are many sources of variation between development contexts, it is not clear a priori what specific variables influence the effectiveness of a process in a given context. These variables can only be discovered opportunistically, by comparing process effects from different environments and analyzing points of difference. In this paper, we illustrate this approach and the conclusions that can be drawn by presenting a family of studies on the subject of software defects and their behaviors – a key phenomenon for understanding macro-process issues. Specifically, we identify common “folklore,” i.e. widely accepted heuristics concerning how defects behave, and then build up a body of knowledge from empirical studies to refine the heuristics with information concerning the conditions under which they do and do not hold.
Impact Analysis
Per Jonsson, Mikael Lindvall, , Impact AnalysisA Aurum, C Wohlin, eds., Heidelberg, Germany, pp. 116-142, 2005
Evaluating Software Architectures
Roseanne Tesoriero Tvedt, Patricia Costa, Mikael Lindvall, , Evaluating Software Architectures, 2004
show abstract [+]
As software systems become increasingly complex, the need to investigate and evaluate them at high levels of abstraction becomes more important. When systems are very complex, evaluating the system from an architectural level is necessary in order to understand the structure and interrelationships among the components of the system. There are several existing approaches available for software architecture evaluation. Some of these techniques, pre-implementation software architectural evaluations, are performed before the system is implemented. Others, implementation-oriented software architectural evaluations, are performed after a version of the system has been implemented. This chapter briefly describes the concepts of software architecture and software architectural evaluations, describes a new process for software architectural evaluation, provides results from two case studies where this process was applied, and presents areas for future work.
The Roots of Agility
David Cohen, Mikael Lindvall, Patricia Costa, , The Roots of Agility, pp. 53-68, 2004
Impact Analysis in Software Engineering
Mikael Lindvall, , Impact Analysis in Software Engineering, pp. 127-209, 2004
An introduction to agile methods
David Cohen, Mikael Lindvall, Patricia Costa, , An introduction to agile methods, pp. 1-66, 2004
show abstract [+]
Agile Methods are creating a buzz in the software development community, drawing their fair share of advocates and opponents. While some people consider agile methods the best thing that has happened to software development in recent years, other people view them as a backlash to software engineering and compare them to hacking. The aim of this chapter is to introduce the reader to agile methods allowing him/her to judge whether or not agile methods could be useful in modern software development. The chapter discusses the history behind agile methods as well as the agile manifesto, a statement from the leaders of the agile movement. It looks at what it means to be agile, discusses the role of management, describes and compares some of the more popular agile methods, provides a guide for deciding where an agile approach is applicable, and lists common criticisms. It summarizes empirical studies, anecdotal reports, and lessons learned from applying agile methods and concludes with an analysis of various agile methods. The target audiences for this chapter include practitioners, who will be interested in the discussion of the different methods and their applications, researchers who may want to focus on the empirical studies and lessons learned, and educators looking to teach and learn more about agile methods.
Theses
Directed Random Testing
C Pacheco, "Directed Random Testing", 2009,
Link