Context: UML diagrams are the de facto standard for analysing, communicating and designing software systems, as well as automated code generation. However there is a certain degree of difficulty in understanding a system represented by means of UML diagrams.
Object: Our previous research demonstrates a significant improvement in understanding the structural aspects of a system represented as a UML class diagram when using a feedback-inclusive prototype of a model. This paper extends our previous work with an empirical validation study for the effectiveness of the feedback-inclusive rapid prototyping (FIRP) method, on the comprehension of system dynamics represented as multiple interacting UML statecharts. Because models often combine structural and behavioural views that are highly intertwined, we additionally evaluate the effectiveness of the proposed method with respect to comprehension of the between-view consistency.
Method: The FIRP environment was built following the principles of Design Science Research in Information Systems. This study targets the empirical validation of the effectiveness of the proposed technique using an experimental study method. Two experiments were conducted with the participation of 65 final-year master students in the context of different modelling courses from different study programs at KU Leuven using a two-group factorial experimental design. The effectiveness of the FIRP method was measured by comparing students’ performance between the cycles with and without the use of the method, using the understandability (comprehension test results) as the dependent variable and the use of FIRP as the independent variable. Effects from unknown variables were neutralized by means of randomized group compositions. The effectiveness of FIRP was additionally assessed with respect to personal characteristics (age, gender, self-efficacy) and user acceptance (perceived ease of use, perceived utility, preference, satisfaction).
Results: The findings reveal a significant positive impact of the use of the prototyping technique on students’ comprehension of system dynamics represented as multiple interacting statecharts.
Conclusions: The findings provide empirical support for the advantage of the use of FIRP over manual inspection of interacting statecharts. The findings also suggest that the method is suitable for training system's analysis and modelling skills when UML statecharts are involved.