Understanding the challenges of developing sustainable RS

(*) Christina von Flach (IC-UFBA)

Daniela Feitosa (PGCOMP-UFBA)

Joenio Costa (LISIS-France)

Campo Grande, 25/9/2023

Context (1)

The lack of sustainability of Research Software, affecting reliability and reproducibility of studies.

Context (2)

The lack of awareness and knowledge among researchers about the best practices for developing sustainable RS.

Objective

Identify the current practices, challenges and opportunities related to the development and socio-technical sustainability of Research Software in academic research groups in the Federal University of Bahia (UFBA).

Workflow

4-steps workflow for each interaction

Talk

40-minute talk to introduce concepts about open science, reproducibility, Research Software, sustainability, and software development.

Interview

Research Questions (1)

RQ1: What Open Science practices do researchers use?

RQ2: What do researchers think about Open Science?

Research Questions (2)

RQ3: What do researchers think about Sustainable Research Software?

RQ4: What are the barriers that prevent the adoption of practices that increase Research Software sustainability?

Research Questions (3)

RQ5: What are the supporting factors for adopting practices that help Research Software be sustainable?

RQ6: Is it beneficial for researchers to make their software sustainable?

Participants

A convenient sample of participants was selected due to general lack of availability and narrow time limit of researchers with such profile.

Interview Guide

• Semistructured interviews
• Developed based on the research questions
• Encourages an organic exchange of ideas
• Includes directives that provide information about what is expected from each question and follow-up questions

Data Analysis

• Qualitative coding procedure combining two strategies (1) a concept-driven way; (2) a data-driven way
• Four main categories: Challenges, Opportunities, Open Science Practices, Open Science Knowledge
• Full transcription
• Coding labels defined based on the data and our understanding of the subject

Margrit Schreier. 2012. Qualitative Content Analysis in Practice.

Pilot Interview Study

Context

• 1 participant: a senior researcher in Applied Physics with over 30 years of research experience
• Open-ended interview following the interview guide
• The interview was recorded

After the interview

• The interview was fully transcribed
• The transcripts were reviewed and relevant codes were assigned to specific segments
• In each iteration the coding structure was being refined to develop a coding framework.

Coding Framework

• 32 coding labels were identified during the coding process

• The coding structure defined after the analysis of the segments were organized in four categories

Coding Framework - Challenges

Factors and challenges related to adopting sustainable RS practices:

• Institution Support

• Concerns about opennes

• Developers’ expertise

Coding Framework - Opportunities

Factors that support the adoption of practices to help RS to be more sustainable:

• Positive reaction to practices

• Addressing errors

• Collaboration

Coding Framework - Open Science Practices

Practices related to Open Science currently applied by the research group:

• Public code
• Versioning
• Licensing
• Reproducibility
• Documentation
• Software Reuse
• Testing and Validation
• Credits to Software Collaborators

Coding Framework - Open Science Knowledge

The categories of knowledge the participant has about practices for sustainable RS:

• General knowledge
• Licensing knowledge
• Software Contribution Skills
• Automated testing

Results

RQ3: What does the researcher think about Sustainable Research Software?

• He admires the efforts to make the software more sustainable
• The researcher understands the importance of replication
• He strongly endorses sustainable RS for the research group
• He thinks the research groups need guidance and support in the early stages of adopting sustainable practices

RQ4: What are the barriers that prevent the adoption of practices to help RS be sustainable?

• The lack of an institutional support
• Protection of their intellectual contributions, and the fear of compromising the integrity and credibility of their research if shared prematurely
• The lack of familiarity of RS developers with best practices in code development

RQ5: What are the supporting factors for adopting practices that help RS be sustainable?

• Recognition of the benefits and importance of adopting sustainability practices
• Promote reproducibility, contributing to scientific knowledge’s overall reliability and robustnes
• Opportunity for other experts in the field to collaborate on software development
• Extend the impact beyond the research community

RQ6: Is it beneficial for the researchers to make their software sustainable?

• The software made available to everyone can help others researchers
• The possibility of replication can validate the results accuracy and reliability
• The automated tests can save time and effort
• Making RS sustainable will collaborate to advance scientific knowledge

Discussion

Limitations

• The related findings are limited to a specific time, place, and condition and do not imply any generalization.
• Future interviews should include researchers from different study fields

Positive Feedback

• The semistructured interview format with researchers is preferred over the survey
  • It allows discussions
  • Provides an opportunity to reflect on their own practices
  • The discussions contribute to researchers’ understanding of the subject
  • A tool for promoting awareness, knowledge sharing, and behavior change

Improvement Opportunities (1)

• The length could be an issue. The duration should not be more than 40 minutes
• It should include more questions related to the support needed from the institution
• Enhance understanding of the concepts by incorporating clear definitions and examples

Improvement Opportunities (2)

• Sharing relevant materials containing key concepts before the interview
• Institutions should invest resources in addressing software research, open science policies, practices, training and scientific infrastructures

Conclusion

Interviews offer a rich and interactive space for discussions

A talk, prior to the interview, with the fundamental concepts is important.

The finding helped us structuring a coding framework.

By embracing the strengths and addressing the limitations identified, we will be able to understand the current practices adopted, the challenges and supporting factors to develop sustainable RS.

Understanding the challenges of developing sustainable RS

(*) Christina von Flach (IC-UFBA): flach@ufba.br

Daniela Feitosa (PGCOMP-UFBA)

Joenio Costa (LISIS-França)

Campo Grande, 25/9/2023