Raphael Saraiva, Allysson Allex Araújo, Altino Dantas, Italo Yeltsin and Jerffeson Souza
Optimization in Software Engineering Group (GOES.UECE) | State University of Ceara - Brazil
Background: The Release Planning (RP) is one of the most complex and relevant activities in the iterative and incremental software development, because addresses all decisions related to the selection and assignment of requirements to releases. There are a number of existing approaches which are based on the belief that the RP can be formalized as an optimization problem. In this context, Search Based Software Engineering (SBSE) proposes to apply search techniques to resolve complex problems of software engineering. Since the RP is an ill-defined problem with a large focus on human intuition, the Decision Maker (DM) preferences and engagement poses a key factor to the resolution process. Thus, we emphasize the importance at gathering the human preferences and, consequently, guide the search process to solutions which are also subjectively valuable to the DM.
Methods: In this paper, we explore a multi-objective approach which considers the human preferences as another objective to be maximized, as well as maximizing the overall clients satisfaction and minimizing the project risk. Basically, the DM defines a set of preferences about the requirements allocation which are stored in a Preferences Base responsible for influencing the search process. The performed empirical study was divided in two different experiments, respectively named as (a) Artificial Experiment and (b) Participant-based Experiment. Basically, the first one aims to analyze the approach using different search-based algorithms (NSGA-II, MOCell, IBEA and SPEA-II), over artificial and real-world instances, while the second one aims at evaluating the use of the proposal in a scenario composed of real human evaluations.
Results: The Artificial Experiment points out that NSGA-II obtained overall superiority in two of the three datasets investigated, positioning itself as superior search technique for smaller scenarios, while IBEA showed to be better for large ones. Regarding the Participant-based Experiment, it was found that two thirds of the participants evaluated the preference-based solution better than the non preference-based one.
Conclusions: Through the achieved results, we may assume that is feasible to investigate the approach in a real-world scenario. In addition, we made available a prototype tool for the release planning able to incorporate the human preferences about the requirements allocation into the final optimized solution.
Keywords: Release Planning, Genetic Algorithm, Human Preferences, Search Based Software Engineering.
Three instances were used in this experiment, respectively named as dataset-1, dataset-2 and dataset-3. Both dataset-1 and dataset-2 are based on real-world. In Dataset-3, we generated him artificially with 600 requirements and 5 clients.
For the artificial experiment (RQ1) a simple version of the algorithms was used to evaluate the behavior of multiobjective metaheuristics:
For the experiment with human participants (RQ2 and RQ3) a version of the graphical interface algorithm was used to evaluate the behavior of the approach in a real context:
Three research questions were designed to assess and analyze the behavior of our approach. Due to the space limitations of the article, some information regarding RQ1 and RQ2 could not be presented. Therefore, it is shown below some extra data for a better understanding of the approach
RQ1: Which search-based techniques, among the evaluated ones, produce better solutions?
The results produced by all scenario algorithms for each instance and scenario presented in the article.
The table below shows for dataset-1 the average result achieved by each algorithm in the Hyervolume (HV), Spread (SP) and Generational Distance (GD) metrics for the lowPrefs and highPrefs
The table below shows for dataset-2 the average result achieved by each algorithm in the Hyervolume (HV), Spread (SP) and Generational Distance (GD) metrics for the lowPrefs and highPrefs
The table below shows for dataset-3 the average result achieved by each algorithm in the Hyervolume (HV), Spread (SP) and Generational Distance (GD) metrics for the lowPrefs and highPrefs
RQ2: What is the subjective benefit when considering the DM’s preferences as an objective tobe optimized?
In order to evaluate the subjective benefit, we analyzed the subjective evaluation obtained during the second and third stages of the experiment with Participants. The table below shows the results obtained for each participant in terms of value of the objectives and subjective evaluation attributed to the final solution presented.
|Participants||Second Stage||Third Stage|
|No-Interactive value||Satisfaction||Risk||Interactive value||Satisfaction||Risk||Preferences||Nº of Attended Preferences|