Misalignments of software quality models.
详细信息
- 作者：Rahman ; Mohammad Foyzur.
- 学历：Doctor
- 年：2013
- 毕业院校：University of California
- Department：Computer Science.
- ISBN：9781303792526
- CBH：3614263
- Country：USA
- 语种：English
- FileSize：2905844
- Pages：219
文摘
Software defects are expensive,costing the US economy $60 billion annually [1]. However,quality control techniques are expensive and are further constrained by the availability of resources. Therefore,a large body of research focuses on efficient quality control activities. Defect prediction and hypothesis testing (collectively known as software quality models) are important parts of the ongoing quality assurance research. In recent years widespread availability of powerful statistical and machine learning techniques resulted in countless defect prediction as well as hypothesis testing models. However,easy obtainability of such models may not translate to successful deployment and interpretation. In this thesis,we study various caveats in planning,building and execution of models； as well as the resulting misalignments of such models with their intended purpose. Model building starts with the collection of data such as defect history and different hypothesized factors related to defect introduction. Performance of the resulting model is constrained by the quality of the collected data. Data quality issues may therefore raise serious concerns about the validity and utility of the resulting models. In our prior work [2] we reported the presence of data quality issues in defect history data. In this thesis we further focus on empirically assessing the impact of data quality on the resulting models. Model performance is also determined by our ability to properly measure different properties of software entities. Models are learned functions that try to describe the relationship between the defect proneness of the entities and their measured properties. Entity properties are traditionally known as metrics. Not all metrics are equally useful or suitable to build appropriate models with specific performance criteria. Therefore,in this work we also study the suitability of metrics under different model performance criteria. Gathering properties of entities requires determining the proper granularity of these entities. For software quality models,different granularities of interests include modules,files,functions or even group of lines. Historically,researchers worked on defective files or modules. While such models are useful for testing hypotheses or predicting defects at the granularities that they were learned,attempts to use the same models to infer properties at a different granularity may result in ecological fallacy [3]. Therefore,we need to carefully consider the correct granularity for quality models. Given that developers contribute at the line level,file or module level models may only be useful for the aggregate interpretation of the relationship between metrics and quality. We therefore carefully study quality factors at the line level granularity for two popular process metrics,namely ownership and experience and compare their impact on quality with that of the traditional coarse grained models. We also study at the line level the impact of copied code (also known as code clones) on software quality in the absence of proper refactoring. Once models are trained,selection of the correct model involves proper model comparison. The comparison technique must align with the expected usage of the chosen model. We study the comparison process to aid practitioners in correctly comparing and selecting their desired models. Under a carefully defined set of objectives,we investigate the suitability of traditional comparison metrics such as information retrieval performance and cost-sensitive comparison techniques to select the desired models. Our findings reveal several sources of misalignments during planning,execution,selection and interpretation of software quality models. We show that traditional concern about data quality issues is compounded by sample size. In fact,depending on the desired model characteristics,sample size can be much more important than data quality issues. We also unearth a searching under the lamppost phenomenon for choosing predictive metrics for software quality models,and provide recommendations for the correct set of metrics that tailor to different model deployment objectives. Our work also raises awareness about the importance of fine grained analysis for building software quality models by contrasting the resultant outcome with coarse grained analysis. Finally,our investigation of the appropriateness of model comparison techniques reveals the misalignment of expectation and selection of models. Specifically,the incorrect choice of model selection techniques may often select less stellar models while rejecting valuable ones.