EAQSE: Candidate software engineering questions

What’s on your mind? Whether you are a software engineering practitioner or a researcher, tell EAQSE what you feel are important questions worthuy of empirical study.

Submit a Software Engineering question that matters to you by replying to this post!

11 thoughts on “EAQSE: Candidate software engineering questions”

  1. This is a first list of important software engineering questions which:
    A. Are important for practicing professionals!
    B. Could immensely benefit from answers, even partial, even limited, but based on objective, empirical studies!

    They come from the two blog posts starting at http://bit.ly/2DRatKk.

    1. What are the respective values of upfront design and refactoring? How best can we combine these approaches?

    2. Specification and testing are complementary techniques. Specifications are in principles superior to testing in general, but testing remains necessary. What combination of specification and testing works best?

    3. What is the best commit/release technique, and in particular should we use RTC (Review Then Commit, as with Apache originally then Google) or CTR (Commit To Review, as with Apache later)?

    4. What measure of code properties best correlates with effort? Many fancy metrics have appeared in the literature over the years, but there is still a nagging feeling among many practitioners that for all its obvious limitations the elementary SLOC metrics (Source Lines Of Code) still remains the least bad.

    5. When can a manager decide to stop testing?

    6. Is test coverage a good measure of test quality?

    Please add your own!

    — Bertrand Meyer

  2. Here are some (influenced, as you might have guessed, from my current focus on Model-Based Systems Engineering).
    Note that the numbers do not imply any particular importance or order and start from 7 for referencing purpose in the follow-up of the workshop.

    7. As in Systems’ modeling, the definition of the environment is almost as critical as the definition of the expected system itself, how can the V&V and testing approaches can be used to assess the quality of the environment definition ?

    8. How can we enforce relationships and traceability between Requirements (as in “regular” development approaches) with User Stories (as in Agile or BDD approaches) ?

    9. How can we reduce the gap between the artifacts manipulated by the engineers to develop the system (models, components, objects, …) and the one we would like the final user to manipulate to control, redefine, tune, etc. the system (intents, goals, services, …) ?

    — JMB

    1. We compiled a list of 145 questions that Microsoft software engineers would like data scientists to investigate about software, about software processes and practices, and about software engineers.

      The paper is available here (with a top-ten list):
      https://www.microsoft.com/en-us/research/publication/analyze-this-145-questions-for-data-scientists-in-software-engineering-2/

      The list of 145 questions is in the appendix:
      https://www.microsoft.com/en-us/research/publication/appendix-to-analyze-this-145-questions-for-data-scientists-in-software-engineering/

      1. Monika Gupta and her co-authors have performed a similar survey/interview study of industry practitioners asking them about problems encountered in software development process management that the practitioners would like process mining team to solve. They have identified 30 such problems. Here is the top 5:

        1 Identify BOTTLENECKS and inefficiencies causing delay to take remedial actions and
        have better estimation in future.
        2 Enable early detection and PREVENTION OF DEFECTS instead of fixing them during
        the later stage by understanding patterns of escaped defects.
        3 Avoid putting efforts on LESS SIGNIFICANT ACTIVITIES by identifying redundant
        or unnecessary steps of process.
        4 Automatic ADAPTION OF PROCESS according to different project specifications that
        is, design process based on knowledge of similar successful projects instead of selecting
        process only on the basis of experience.
        5 Inspect REOPENED issues to identify the root cause and recommend verification for
        future issues based on learning from issues reopened in the past.

        The paper has been published in MSR 2015.

        Monika Gupta, Ashish Sureka, Srinivas Padmanabhuni, Allahbaksh M. Asadullah:
        Identifying Software Process Management Challenges: Survey of Practitioners in a Large Global IT Company. MSR 2015: 346-356

  3. The unspoken Grand Challenge of software engineering in industry is the hiring problem. One aspect of this is:

    Which assessment techniques have the highest predictive power in determining whether a candidate for a software engineering position will be successful if hired into the company?

  4. It all comes down to: How can we produce better software faster? This can be subdivided into dimensions of better (quality, user experience, etc) and faster (developer velocity, time to market, etc). There’s a variety of subquestions that are worthy of study, but they can’t be answered in a vacuum; there must be available data and methods for answering those questions with sufficiently credible transferable evidence (if not generalizability).

  5. I’m especially interested in questions that have to do with applying rigorous techniques to the software development process, and in particular the role of some kind of formal specifications:

    — What forms of specifications are more approachable (reading and writing) by practitioners?

    — What features (syntax, simplicity, similarity to the implementation language, tool support, …) of a specification language have the most impact on its ease of usage?

    — What kind of software (libraries, system software, concurrent code, …) benefits the most from having specifications?

    — For specifications, what is the three-way trade-off between completeness, bug-catching effectiveness, and writability by non-experts?

    — How can we quantify the cost-effectiveness of writing specifications vs. writing (unit) tests?

  6. Looking at officially sanctioned V&V practices, we can see hundreds of methods for “improving” software (see sections 2+3 of http://menzies.us/pdf/07ivv.pdf). Which actually work? What least cost subset of them should be applied to some current software system? I tried answering those questions, at NASA mid last decade (see above paper) but i had to really… reach… to get answers (and in the end, did not like the methodology). But can we do better now?

  7. Users have different goals for their systems. How can we tune our recommendations for different projects to these goals?

    (HINT: my answer is multi-objective optimization… but that’s just my view)

  8. There is much talk in software about evidence-based reasoning (EBR). Usually, it cites as justification, its use in medicine.

    But if we look into recent papers in the medical literature (*), we can see papers that i would describe as “post-EBR”. that is, not all medical conditions are covered by prior medical trials (so no evidence). and conducting trials for everything to get that evidence is impractically expensive (so those trials will never exist). and the the particulars of the current problem might contain specifics that nullify the results of the trials (so if those trials do exist, then they won’t be relevant).

    (*) https://www.futuremedicine.com/doi/10.2217/cer.13.65

    one proposed solution assumes databases of many, many treatment reports (so not formal studies , more like records of person-by-person interaction; think issue reports from Github from hundreds of millions of “patients” which is this discussion would be “project”). some instance based reasoning scheme then looks at your problem, then pulls relevant records, from which you synthesize a particular treatment for your particular case

    all of which begs the question: should we seek “models” that answer big questions? or should we seek “methods” that can find particular answers to our particular problems?

Leave a Reply

Your email address will not be published. Required fields are marked *