Experimentation on the trustworthiness of Open Source Software

Publication Type:



The QualiPSo Project (IST-FP6-034763/2005-2010) (2011)


Empirical studies will be carried out in industrial environments. Therefore,
experiments will need to be carefully planned, designed, and executed, to
minimize the risk of having incomplete or misleading information. Clearly, the
second iteration of this task in the second round of experiments will benefit from
the experience gathered in the first round of experiments. The empirical studies
will be as little invasive as possible for the industrial environments studied to
disturb the observed environment as little as possible and also maximize the
chances that data are actually collected from the industrial environment. To this
end, the automated tools built in WP5.5 will be used. At any rate, questionnaires
and interview may also be used to collect additional pieces of information that
would not be possible to retrieve from the raw data. The collected information
will be organized and stored in repositories. In the second iteration of this Task,
some measures used in the first round of experiments may be deleted, while
others may be added, based on the results of the first round of experiments.
This task will clearly provide inputs to the tool building WP 5.5 and will rely on
the tools to be carried out effectively and efficiently. In addition to data on
trustworthiness, data on the cost-effectiveness of and practicality of the
approach will be collected, to assess the overall impact that the approach may
have on industrial environments.
The goal of the task is to assess the effectiveness of the approach outlined in
Activity A5. In particular, the trustworthiness factors identified in WP5.3, the test
approaches, suites and benchmarks identified in WP5.4, and the tools
developed, customized and integrated in WP5.5 are experimented with in Task
The main result of the experimentation generates the data concerning the
trustworthiness of the OSS products examined during the experimentation.
These data are an input to Task 5.6.2, which analyzes them to find out whether
the factors identified were actually influential on the trustworthiness of the OSS
products and artefacts, and –if so– derives quantitative model that represent
such dependency.
Other results of the task are feedbacks concerning the methods, models,
techniques and tools being defined.
The main instrument for the experimentation is represented by empirical studies
and measurement.
According to the indications from WP5.3, the experimentation addresses two
aspects of trustworthiness: the perception of trustworthiness by users and the
contribution to trustworthiness from the qualities of the software products. The
former is assessed by collecting evaluations from users (both from industry and
public administrations); the second is measured. QualiPSo • 034763 • wd5.6.1 • version 3.0, dated 31/01/2011 • Page 4 of 41
Users evaluations are collected by means of questionnaires and interviews.
The measurements of the OSS product are performed using the tools identified,
produced, or customized in WP5.5. The collected information is stored in
The main results obtained are:
• The definition of a GQM plan that is fully operational and can be used to
support the trustworthiness measurement and analysis process.
• The data reporting the users’ subjective perceptions of the trustworthiness of
OSS product.
• A great deal of measures –all properly stored in a measure repository–
concerning various features of OSS products:
o Static code measures
o Dynamic code measures.
o Measures about the product versioning and configuration.
o Measures about the licensing information provided with OSS