SUPPORTING PAGE
Software Release Planning Based on Interactive Optimization Through the Formalization of Decision Maker Preferences
Altino Dantas, Italo Yeltsin, Allysson Allex Araújo and Jerffeson Souza
● GOES.UECE ●
Optimization in software Engineering Group From State University of Ceará

Abstract

Search Based Software Engineering proposes to solve Software Engineering problems by applying optimization techniques. On the incremental development, deciding which requirements must be implemented in each release of the software poses as complex task due to several conflicting aspects. Different search-based approaches have been proposed to deal with the release planning problem, however, most of them do not consider the human expertise during he generation of the solutions. Including the Decision Maker (DM) during the resolution process may be useful because it enables to incorporate his/her tacit assessments and avoids any resistance or lack of reliability in the final results. Thus, we propose an interactive approach for Release Planning based on Interactive Optimization through the formalization of 8 types of preferences related to allocate requirements in the releases. In addition, we conducted an empirical study with automatic and participant-based experiments that indicated the proposal is able to satisfy the DM preferences and prioritize the most important ones. Overall, the approach was able to increase DM satisfaction, on average, from 44% to 82% when compared to the non-interactive version.

Palavras-chave: Release Planning, Interactive Optimization, Preferences.

Instances

Four instances were used in the empirical study. Two of them, “I_1” and “I_2” available here are based on information from a word processor and a support decision tool real projects, respectively. In addition, other two artificial instances “I_3” and “I_4” were generated by random way.

Instance Name Amount Files
Clients (M) Requirements (N) Releases (P) Instance Ramdon preferences
I_1 (Word Processor) 4 50 5 Download Download
I_2 (ReleasePlanner) 9 25 8 Download Download
I_3 (Dataset-3) 3 100 6 Download Download
I_4 (Dataset-4) 6 150 8 Download Download

Results

Mouse over to see RQ text and click to the its results.

RQ1: What is the proposal capacity in satisfying the DM's preferences?

RQ1a: What is impact of satisfying DM's preferences?

RQ1b: Is the proposal able to prioritize the most important preferences?

RQ2: What is the relation between number of preferences in the Preferences Base and the percentage of satisfied preferences by the proposal?

RQ3: Using the proposal how satisfied the DM would be in regarding to his/her preferences contemplation?

RQ4: Given a interactive solution generated through the proposal, what is the relation between the DM satisfaction feeling and the percentage of satisfied more important preferences?

RESULTSRQ1

DP=10% Word Processor ReleasePlanner Dataset-3 Dataset-4
μ PS Â12 PS Â12 PS Â12 PS Â12
0.0 0.513 - - 0.211 - - 0.493 - - 0.529 - -
0.1 0.900 0.96 0.667 1 0.797 0.98 0.767 0.96
0.2 1.000 1 0.711 1 0.887 1 0.844 1
0.3 1.000 1 - 0.867 1 0.887 1 - 0.873 1
0.4 1.000 1 - 0.978 1 0.920 1 0.869 1
0.5 1.000 1 - 1.000 1 0.903 1 0.902 1
0.6 1.000 1 - 1.000 1 - 0.907 1 0.904 1
0.7 1.000 1 - 1.000 1 - 0.930 1 0.904 1 -
0.8 1.000 1 - 1.000 1 - 0.930 1 - 0.942 1
0.9 1.000 1 - 0.989 1 0.917 1 0.924 1
1.0 1.000 1 - 1.000 1 0.930 1 0.918 1

Table - Average results of SP, Â12 values between each configuration μ and setting μ = 0, and statistical difference of variations on μ for each instance using different PD.

Figure - Average percentage of satisfied preferences of all instances, considering μ variation and different levels of Preferences Density.

RESULTSRQ1a

DP=10% Word Processor ReleasePlanner Dataset-3 Dataset-4
μ GP PP Â12 GP PP Â12 GP PP Â12 GP PP Â12
0.1 0.75 0.015 0.71 2.16 0.018 0.89 0.61 0.002 0.50 0.45 0.000 0.52
0.2 0.95 0.024 0.81 2.37 0.023 0.92 0.80 0.000 0.50 0.60 -0.006 0.45
0.3 0.95 0.020 0.78 3.11 0.028 0.96 0.80 0.003 0.53 0.65 0.004 0.56
0.4 0.95 0.019 0.79 3.63 0.034 0.98 0.86 0.005 0.60 0.64 -0.007 0.44
0.5 0.95 0.017 0.76 3.74 0.034 1 0.83 0.011 0.68 0.71 -0.001 0.51
0.6 0.95 0.021 0.79 3.74 0.038 1 0.84 0.005 0.60 0.71 0.001 0.54
0.7 0.95 0.024 0.85 3.74 0.038 1 0.89 0.010 0.63 0.71 0.002 0.53
0.8 0.95 0.020 0.82 3.74 0.036 1 0.89 0.005 0.57 0.78 0.010 0.63
0.9 0.95 0.022 0.8 3.68 0.039 1 0.86 0.012 0.66 0.75 0.003 0.55
1.0 0.95 0.024 0.83 3.74 0.036 1 0.89 0.004 0.53 0.74 0.002 0.52

Table - Average values of PG and PP obtained for all instances, μ and PD variations, and statistical difference between consecutive \textit{PP} values from same instance, and Â12 results between each μ and μ = 0 taking score function results

Figure - Comparison between PG and PP, considering the average of the results for the all instances with each PD and configuration of μ.

RESULTSRQ1b

Preferences Density = 10% Word Processor ReleasePlanner Dataset-3 Dataset-4
μ PS NS Â12 PS NS Â12 PS NS Â12 PS NS Â12
0.1 0.90 0.93 0.57 0.67 0.94 1 0.80 0.91 0.9 0.77 0.88 0.86
0.2 1.00 1.00 0.50 0.71 0.95 0.88 0.89 0.96 0.88 0.84 0.95 0.9
0.3 1.00 1.00 0.50 0.87 0.98 0.58 0.89 0.97 0.97 0.87 0.96 0.88
0.4 1.00 1.00 0.50 0.98 1.00 0.50 0.92 0.98 0.79 0.87 0.96 0.93
0.5 1.00 1.00 0.50 1.00 1.00 0.50 0.90 0.97 0.9 0.90 0.98 0.97
0.6 1.00 1.00 0.50 1.00 1.00 0.50 0.91 0.97 0.88 0.90 0.97 0.88
0.7 1.00 1.00 0.50 1.00 1.00 0.50 0.93 0.98 0.74 0.90 0.97 0.88
0.8 1.00 1.00 0.50 1.00 1.00 0.50 0.93 0.98 0.74 0.94 0.98 0.69
0.9 1.00 1.00 0.50 0.99 1.00 0.50 0.92 0.98 0.82 0.92 0.98 0.85
1.0 1.00 1.00 0.50 1.00 1.00 0.50 0.93 0.98 0.74 0.92 0.97 0.78

Tabela - Average of \textit{SP} and \textit{SL} results and values of Â12 between they, for each instance, μ variation and number of preferences to be considered.

Figure - Comparison between SP and SL, considering the average of the results for all instances with each DP and configuration of μ.

RESULTSRQ2

µ = 0.1
DP 10 20 30 40 50 60 70 80 90 100
Word Processor 0.90 0.89 0.74 0.65 0.57 0.53 0.54 0.50 0.46 0.44
ReleasePlanner 0.67 0.80 0.72 0.59 0.52 0.56 0.58 0.58 0.55 0.51
Dataset-3 0.80 0.62 0.58 0.62 0.59 0.59 0.57 0.56 0.53 0.51
Dataset-4 0.77 0.63 0.58 0.56 0.55 0.51 0.49 0.48 0.47 0.47

Table: Average of SP results and indication of statistical difference between the values obtained for each instance and each DP variation, considering three different μ configurations.

Figure - Satisfaction Level obtained for each instance considering the quantity of preferences with three different μ configurations.

RESULTSRQ3

PARTICIPANTS #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Average Standard Deviation
Number of Interactions 8 109 15 47 23 54 20 31 20.00 25 35.20 29.48
Time (min) 16.14 45.40 26.93 19.57 11.81 25.41 21.50 19.88 16.87 26.88 23.04 9.26
Final Number of Preferences 3 4 8 11 4 10 33 20 20.00 5.00 11.80 9.70
PS (Si) 1.00 1.00 0.88 0.91 1.00 0.90 0.78 1.00 0.85 1.00 0.93 0.08
PS (S) 0.33 0.75 0.25 0.55 0.50 0.30 0.27 0.30 0.25 0.60 0.41 0.18
SL (Si) 1.00 1.00 0.90 0.88 1.00 0.90 0.76 1.00 0.86 1.00 0.93 0.08
SL (S) 0.33 0.75 0.27 0.52 0.43 0.31 0.28 0.28 0.23 0.61 0.40 0.17
PP 0.07 0.02 0.04 0.04 0.01 0.09 0.07 0.13 0.03 0.006 0.05 0.04
Subjective Evaluation (Si) 0.93 0.87 0.88 0.75 0.93 0.84 0.75 0.88 0.50 0.87 0.82 0.13
Subjective Evaluation (S) 0.65 0.81 0.50 0.25 0.62 0.58 0.62 0.13 0.25 0.00 0.44 0.27

Table – Results of the test for each one of the 10 participants.

RESULTSRQ4

PARTICIPANTS #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 Média
SL (Si) 1.00 1.00 0.90 0.88 1.00 0.90 0.76 1.00 0.86 1.00 0.93±0.08
Subjective Evaluation (Si) 0.93 0.87 0.88 0.75 0.93 0.84 0.75 0.88 0.50 0.87 0.82±0.13
SL(Si)-SE(Si) 0.07 0.13 0.02 0.13 0.07 0.06 0.02 0.12 0.36 0.13 0.11±-0.05

Table - Results of SL and Subjective Evaluation as well as difference between them.

Figure - Relation between Satisfaction Level (SL) and Subjective Evaluation.

SOURCECODE

The search technique used as optimization process was an Interactive Genetic Algorithm adapted from the mono-objective GA present in jMetal framework. Below you can download the source code of the entire project, written in Java, or the .jar file


Download source code Download .jar file

Screen of the tool developed for Participant-based Experiment.

Contact

E-mail: altino.dantas@uece.br

Acknowledgment

All members of Optimization in Software Engineering Group from State University of Ceará - GOES.UECE