ISERN Co-Chairs

Prof. Markku Oivo

University of Oulu, Finland

Prof. Dag Sjøberg

Simula Research Laboratory and University of Oslo, Norway

ISERN Meeting Registration

List of Attendees

Monday, Setember 18th

Opening

S1: New participant introductions

(Chair: Markku Oivo)

S2: Empirical Research

(Chair: Dieter Rombach)

Context: Previous ISERN sessions at ISERNs have focused on Guidelines for Documenting Empirical Results, but these guidelines have not been accepted and applied broadly. At the same time we still face problems getting empirical results published in journals within the larger software engineering community ”

Goal of the session: Identify guidelines for getting empirical studies published. Input are the existing guidelines from previous ISERN 2005, the discussion that happened at a recent Dagstuhl Workshop on Empirical Software Engineering (2006), and an invited presentation by Basili & Elbaum (Better Empirical Science for Software Engineering) at ICSE 2006 in Shanghai. In light of this input we will then conduct group work aimed at applying the guidelines from ISERN 2005 to already published or pre-published studies. These experiences will be fed into a rework of these guidelines.

 Session format:

- Introduction on the goals of the session

- Short summaries of the Dagstuhl and ICSE Feedback.

- Group work on applying the guidelines to example studies

- Discussion of findings from group work & rework of guidelines

- (Summarize contributions from the session for ISERN experience factory) – post meeting

Reading Materials:

“Slides (Basili & Elbaum) from ICSE 2006”

(MORE later)

Coffee Break

S2: Empirical Research

(Chair: Dieter Rombach)

CONTINUED

Lunch

S3: Data ownership

(Chair: Marv Zelkowitz)

Context: Previous ISERN sessions in 2004 and 2005 discussed issues concerning data ownership and how experimental software engineering data can be shared. Since ISERN 2005 a series of protocols have been developed for understanding data ownership issues. Both creators of empirical data and users of empirical data need a set of guidelines on the use of this data. No one set of rules works in all situations. Properties such as data ownership, privacy concerns, transfer to a third party, publication rights, and costs for using or maintaining the data all need to be spelled out in order to create a market in empirical software engineering data.

Goal of the session: In this laboratory session, attendees will apply the data ownership protocols to test their applicability in a variety of settings. The goal is to uncover any remaining attributes still undefined by the protocols.

Session format:

- Introduction on the goals of the session.

- Brief overview of the data ownership attributes.

- Group will divide into 5 or 6 subgroups. Each subgroup will apply the data ownership attributes to a different data set

- Each subgroup will give a short presentation to the larger group on how effective the data ownership attributes were to describing data usage rules.

- General discussion on the effectiveness of the data ownership rules to the community at large.

Reading Materials:

“Protocols in the use of Empirical Software Engineering Artifacts” by M. Zelkowitz, V. Basili, D. Sjøberg, P. Johnson and T. Cowling.

S4: Experimentation and Decision making in Software Engineering: How they are related?

(Chair: Andreas Jedlitschka)

Context: Experimentation is always a means to better understand, control and improve software processes, products and technologies. Decisions are the crystallization point is being successful in software projects. This session is addressing the relationship between experimentation and decision-making and how synergies can be made.

Emphasis is made to look at this relationship form both a research and industry perspective.

Goal of the session:

-   Identify the relationship between experimentation and decision-making from an industrial and research perspective.

-   Provide examples how questions of decision-making have helped to run more focused experiments and how results of experimentation have helped to make better decisions.

-   Add to the guidelines for empirical research.

Session format:

- Introduction and goals of the session

- Panel discussion with initial position statements of

Natalia Juristo (Univ Politecnica de Madrid)

Nachi Nagappan (Microsoft)

Forrest Shull (Fraunhofer-Center Maryland)

Claes Wohlin (Blekinge Institute of Technology)

- Open discussion

- Summary and outlook

Reading Material:

The topic is based and linked to some former and two forthcoming ISERN sessions:

- 2003 session titled: Empirical Study of Decisions in Software Engineering and Project Management (organized by Stefan Biffl)

- 2004/05: Guidelines for empirical research (addressed decisions to some extend).

- 2006: link to the sessions

S6: Good practices for empirical research with industry (Jyrki Kontio) and

S8: Value-Based Empirical Research (Stefan Biffl)

 Paper “Software Engineering Decision Support and Empirical Investigations – A Proposed Marriage”, WSESE 2003

(b) Reading material

(c) Slides

Coffee Break

Wrap-up of parallel sessions

S5: ISERN Experience Factory

(Chair: Marcus Ciolkowski)

Reading Material: ISERN Experience Factory set of slides

Closing and planning for day 2

Tuesday, Setember 19th

S6: Good practices for empirical research with industry

(Chair: Jyrki Kontio)

Context: Cooperation with industry is a central element in empirical software engineering research. Due to time and cost constraints, it is challenging to attract industry to cooperate in empirical research. The empirical software engineering community will benefit from sharing good practices in establishing and mainintaing such cooperation.

Goal of the session: identify and refine good strategies and practices in industry university cooperation in empirical software engineering.

Session format:

- conclusions from a panel at CSEET (http://www.sbl.tkk.fi/CSEET06panel/) are presented

- participants are divided into teams to evaluate and refine the proposed good practices

- results are summarized and reviewed

- ownership of writing a "Good practice guide on empirical research with industry" is allocated

S7: Generalization from empirical studies

(Chair: Dag Sjøberg)

Context: Previous ISERN sessions at ISERN 2004 (“Empirical theory”) and 2005  on “Generalization from empirical studies”,

Goal of the session: Discuss the complex issues of generalising from empirical studies and hopefully agree on certain strategies or directions for future work

 Session format:

- Introduction on the challenges and goals of the session and short description of the topics to be discussed in each of the three discussion groups (15 min)

- Group work on respectively theory (moderator: Tore Dybå), statistical generalization (moderator: Dag Sjøberg) and aggregation of evidence (moderator: Marcus Ciolkowski) (45 min)

- Summarize discussion from the group work in the session (30 min)

Reading Materials:

“Generalization Session ISERN06 Preliminary Slides”

 M. Jørgensen and D. I.K. Sjøberg. Generalization and Theory-Building in Software Engineering Research, In: Empirical Assessment in Software Engineering (EASE2004). IEE Proceedings, pages 29–36, 2004.

Previous ISERN session slides:

 ISERN 2004 Empirical theory

 ISERN 2005: Generalization from empirical studies

S8: “Value-Based Empirical Research” (with industry focus)

(Chair: Stefan Biffl)

Context: Previous ISERN sessions at ISERNs 2003 to 2005  on “Need-driven Empirical Research”, “Value of empirical work” , and “Value-based Empirical Studies”

Goal of the session: Identify typical industry stakeholders and their expectations of value contributions from empirical studies; look at recent empirical studies and their value contributions. From the group work and discussion researchers in the audience should become more aware on the potential value contribution of their empirical studies and on risks to consider when planning an empirical study targeting industry stakeholders.

Session format:

- Introduction on the goals of the session

- Short issue summaries of senior researchers on their experience with empirical research with industry stakeholders and typical value propositions.

- Group work on identifying stakeholders in empirical research and value contribution (with industry focus)

- Discussion of findings from group work; simplifiers and complicators in aligning empirical research with the industrial stakeholders’ value propositions

- Summarize contributions from the session for ISERN experience factory and a roadmap (conclusions, guidelines, lessons learned).

Reading Materials:

“Value-Based Empirical Research Session ISERN06 Preliminary Slides ”]

"Value-based Software Engineering" (VBSE) book chapter 6 (key elements of VBSE, get started with section 6.9)

- Previous ISERN session slides :

“Need-driven Empirical Research” ISERN 2003

“Value of empirical work” ISERN 2004

“Value-based Empirical Studies” ISERN 2005 

Wrap-up of parallel sessions

S9: ISERN business

(Dieter Rombach)

This session addresses issues related to future activities (e.g., next meetings, cooperation projects) and further improvements of ISERN. Concrete proposals are a presentation by next year's organizers of the meeting in Spain, the updating of our interrelation matrix, and a critical review of the 2006 meeting. In addition, the decisions of the Steering Committee will be presented.  Additional issues may be suggested by ISERN members.

S10: Open Forum

ISERN 2006 Adjourned with refreshments (Happy hour)