International Symposium on Empirical Software Engineering and Measurement

ESEM Best Paper Awards

This award is presented at each ESEM, ACM/IEEE International Symposium on Empirical Software Engineering and Measurement.


    Best Full Paper: Susanne Braun, Stefan Deßloch, Eberhard Wolff, Frank Elberzhager, Andreas Jedlitschka: Tackling Consistency-related Design Challenges of Distributed Data-Intensive Systems: An Action Research Study

    Candidate Best Papers Invited to the Special Issue of EMSE:

    Nasif Imtiaz, Seaver Thorn and Laurie Williams. A Comparative Study of Vulnerability Reporting by Software Composition Analysis Tools

    Elizabeth Bjarnason, Franz Lang and Alexander Mjöberg. A Model of Software Prototyping based on a Systematic Map

    Danilo Caivano, Pietro Cassieri, Simone Romano and Giuseppe Scanniello. An Exploratory Study on Dead Methods in Open-source Java Desktop Applications

    Arif Nurwidyantoro, Mojtaba Shahin, Michel Chaudron, Waqar Hussain, Harsha Perera, Rifat Ara Shams and Jon Whittle. Towards a Human Values Dashboard for Software Development: An Exploratory Study

    Best Emerging Results Paper: Zedong Peng, Upulee Kanewala, Nan Niu.Contextual Understanding and Improvement of Metamorphic Testing in Scientific Software Development

    Best Full Paper: Marvin Muñoz Barón, Marvin Wyrich and Stefan Wagner. An Empirical Validation of Cognitive Complexity as a Measure of Source Code Understandability.

    Best Emerging Results Paper: Gouri Deshpande and Guenther Ruhe. Title: Beyond Accuracy: ROI-driven Data Analytics of Empirical Data.

    Best Industry Paper: Jannik Fischbach, Henning Femmer, Daniel Mendéz, Davide Fucci and Andreas Vogelsang. Title: What Makes Agile Test Artifacts Useful? An Activity-Based Quality Model from a Practitioners' Perspective.

    Best Full Paper:

    Best Emerging Results Paper:

    Best Industry Paper:

    Best Full Paper: Adrian Santos Parrilla and Natalia Juristo. Comparing Techniques for Aggregating Interrelated Replications in Software Engineering.

    Highly Commended Full Paper: Bowen Xu, Amirreza Shirani, David Lo and Mohammad Amin Alipour. Prediction of Relatedness in Stack Over?ow: Deep Learning vs. SVM.

    Best Emerging Results paper (short paper 1): Gema Rodriguez, Andy Zaidman, Alexander Serebrenik, Gregorio Robles and Jesus M. Gonzalez-Barahona. What if a Bug has a Different Origin? Making Sense of Bugs Without an Explicit Bug Introducing Change .

    Best Industry paper: Junji Shimagaki, Yasutaka Kamei, Abram Hindlefor and Naoyasu Ubayashi. Automatic Topic Classi?cation of Test Cases Using Text Mining at Sony Mobile.

    Best Full Paper: Amanda Lee and Jeffrey Carver. An empirical analysis of FLOSS repositories to compare One-Time Contributors to Core and Periphery Developers.

    Highly Commended Full Paper: Brian Malloy and James Power. Quantifying the Transition from Python 2 to 3: An Empirical Study of Python Applications.

    Best Full Paper: Rana Alkadhi, Jan Ole Johanssen, Emitza Guzman and Bernd Bruegge. REACT: An Approach for Capturing Rationale in Chat Messages.

    Best Full Paper: Wilson Rosa, Raymond Madachy, Bradford Clark and Barry Boehm. Early Phase Cost Models for Agile Software Processes in the US DoD.

    Best Full Paper: Davide Fucci, Giuseppe Scanniello, Simone Romano, Martin Shepperd, Boyce Sigweni, Fernando Uyaguari, Burak Turhan, Natalia Juristo and Markku Oivo. An External Replication on the Effects of Test-driven Development Using Blind Analysis.

    Best Short Paper: Katia Romero Felizardo, Emilia Mendes, Marcos Kalinowski, Erica Ferreira Souza and Nandamudi Vijaykumar. Using Forward Snowballing to update Systematic Reviews in Software Engineering

    Best Industry Paper: Ricardo Britto, Darja Smite and Lars-Ola Damm. Experiences from Measuring Learning Potential and Performance in Large-Scale Distributed Software Development.

    Best Full Paper: Keheliya Gallaba, Ali Mesbah, and Ivan Beschastnikh. Don't Call Us, We'll Call You: Characterizing Callbacks in JavaScript.

    Best Full Paper: Leandro Minku, Federica Sarro, Emilia Mendes, and Filomena Ferrucci. How to Make Best Use of Cross-Company Data for Web Effort Estimation?

    Best Short Paper: - Diomidis Spinellis, Panagiotis Louridas, and Maria Kechagia. An Exploratory Study on the Evolution of C Programming in the Unix Operating System.

    Best Full Paper: Ming Fang and Munawar Hafiz. "Discovering Buffer Overflow Vulnerabilities In The Wild: An Empirical Study" .

    Best Short Paper: Nauman Bin Ali and Kai Petersen. "Evaluating strategies for study selection in systematic literature studies" .

    Best Short Paper: Rafael Mello, Pedro Silva, Per Runeson and Guilherme Travassos. "Towards a Framework to Support Large Scale Sampling in Software Engineering Surveys".

    Best Industry Track Paper: Nils Brede Moe, Darja Smite, Aivars Sablis, Anne-Lie Börjesson and Pia Andréasson. "Networking in a Large-Scale Distributed Agile Project" .

    Best Full Paper: Sebastian Nanz, Scott West, Kaue Soares Da Silveira and Bertrand Meyer. Benchmarking Usability and Performance of Multicore Languages.

    Best Short Paper: Jeffrey Stuckman, Kent Wills and James Purtilo. Evaluating software product metrics with synthetic defect data.

    Best Industry Track Paper: Gul Calikli, Ayse Bener, Turgay Aytac and Ovunc Bozcan. Towards a Metric Suite Proposal to Quantify Confirmation Biases of Developers.

    Best Full Paper: Experimental Assessment of Software Metrics using Automated Refactoring; Mel Ó Cinnéide, Laurence Tratt, Mark Harman, Steven Counsell and Iman Hemati Moghadam.

    Best Short Paper: Handling Categorical Variables in Effort Estimation; Masateru Tsunoda, Sousuke Amasaki and Akito Monden

2011 The following technical papers were selected as the "top 5" papers at the conference, on the strength of the reviews received from the PC. The winners were invited to send an extended version of their conference paper for consideration for a special issue of IST Journal. In no particular order, our selected papers are:
  • An Empirical Investigation of Systematic Reviews in Software Engineering; Zhang and Ali Babar
  • End-User Programmers and their Communities: An Artifact-Based Analysis; Stolee, Elbaum and Sarma
  • Design of an Empirical Study for Comparing the Usability of Concurrent Programming Languages; Nanz, Torshizi, Pedroni and Meyer
  • One Technique is Not Enough: A Comparison of Vulnerability Discovery Techniques; Austin and Williams
  • An Empirical Study on the Use of Team Building Criteria in Software Projects; Da Silva, França, Gouveia, Monteiro, Cardozo and Suassun

IEEE Software sponsored an award for the best paper in the industry experience track. The criteria were that the selected paper should be well-written and of high-quality, but also be able to influence the state of the practice. The winner is expected to be an example of the kind of practical, experiential, yet rigorous papers that are of interest to IEEE Software readers. The winning paper was selected by a review committee that consisted of IEEE Software Editor in Chief Forrest Shull, the Industry Experience track chair Brian Robinson, and additional members drawn from both the magazine's editorial board and the experience track's program committee: Mark Grechanik, Maurizio Morisio, and Helen Sharp.
The winning paper was:

  • "Scrum + Engineering Practices: Experiences of Three Microsoft Teams" by Laurie Williams, Gabe Brown, Adam Meltzer, Nachiappan Nagappan

2010 Best Full Papers:
  • Samireh Jalali, Cigdem Gencel and Darja Smite. Trust Dynamics in Global Software Engineering
  • Jingyue Li, Nils B. Moe and Tore Dybå. Transition from a Plan-Driven Process to Scrum - A Longitudinal Case Study on Software Quality
  • Nico Zazworka, Kai Stapel, Eric Knauss, Forrest Shull, Victor Basili and Kurt Schneider. Are Developers Complying with the Process: An XP Study

Best Short Paper:

Stephen MacDonell and Martin Shepperd. Data Accumulation and Software Effort Prediction

2009 Juristo, N. and Vegas, S.: Using Differences among Replications of Software Engineering Experiments to Gain Knowledge
2008 Cataldo, M, Herbsleb, J., Carley, K.: Socio-Technical Congruence: A Framework for Assessing the Impact of Technical and Work Dependencies on Software Development Productivity.
2007 Layman, L.; Williams, L.; Amant, R.S.: Toward Reducing Fault Fix Time: Understanding Developer Behavior for the Design of Automated Fault Detection Tools


Kamei, Y.; Monden, A.; Matsumoto S.; Kakimoto, T. and Matsumoto, K.:.The Effects of Over and Under Sampling on Fault-prone Module Detection

International Symposium on Empirical Software Engineering and Measurement