At SolDevelo, we are committed to prioritizing security and performance improvements. Our organization’s primary goal is to propose and implement enhancements in these crucial areas for our existing projects.

Initially, SolDevelo has outlined a plan to assess multiple open-source projects, including openIMIS, openLMIS, OpenBoxes, and ODK. Such criteria carry significant weight, especially when contemplating the potential risks inherent in deploying software in production environments, particularly in the context of commercializing open-source projects.

To kickstart this initiative, our company has already taken steps to deploy the essential tools required for gathering metrics related to code quality. SonarCloud is our initial choice due to its versatility, but for other tests like SCA we will most likely go with other software. The establishment of static code analysis within project repositories serves as an excellent foundational step in our journey towards creating and maintaining high-quality code.

In this article, we will discuss points related to:

• Code analysis and the power of using it.
• Current state of openIMIS in terms of code analysis and testing tools.
• The idea of the Centralized Sonar Pipeline and its setup.
• Goals of our incoming activities.

Introduction

About us

SolDevelo is a dynamic software development and information technology outsourcing company focused on delivering high-quality software and innovative solutions. An experienced team of developers, customer-oriented service, and a passion for creating the highest quality products using the latest technology are the undeniable advantages of the company.

Our company’s mission is: “We deliver high-quality solutions for modern world challenges. As IT professionals, together with our partners, we build a digital future.”

ISO 9001 confirms our dedication to the highest quality and ISO 27001 shows that we treat security extremely seriously. Over 70% of our team members are certified Scrum Professionals, over 35% are Oracle Certified Professionals and 100% of our quality assurance team has ISTQB certificates.

openIMIS and its significance

openIMIS (open-source Insurance Management Information System) is designed to assist in managing health insurance schemes. The main objective of this initiative is to increase the availability of efficient and effective technology adapted to the conditions prevailing in developing countries.

The software can be used to manage:

• Membership/patient registry
• Benefit package (services and prices)
• Contracted providers
• Service claims and reimbursement
• Client feedback
• Monitoring and reporting, etc.

SolDevelo has been engaged in the development of openIMIS for many years, and we’re committed to improving its security and performance. Furthermore, we’ve been recently involved in extending openIMIS by incorporating a social protection scheme and integrating it with the existing healthcare layer. 

Read our other articles to learn more about the project, and the upcoming implementation of openIMIS in Moldova. 

When it comes to the technical side of the system, the modular version of openIMIS is designed to be flexible and scalable. It makes it easy to put together both backend and frontend modules. At its core, this modular structure is made up of various business modules, each with a specific role in the openIMIS system. These modules are carefully designed to handle specific functions, making the system more organized and easier to maintain. 

Broken into smaller, modular parts, openIMIS becomes more adaptable and development becomes simpler. Users can easily customize the system to suit their specific needs while benefiting from a high level of reusability across the entire platform. This modular approach increases flexibility, empowering users to configure openIMIS according to their exact requirements, all within a robust and unified framework.

Why is code analysis important?

Code analysis, as powered by SonarCube and its cloud based version SonarCloud, holds importance for several reasons:

• Identifying and preventing code issues: This code analytics tool excels at automatically detecting and highlighting various code issues, be it bugs, vulnerabilities, or code smells. This invaluable capability empowers developers to intercept potential problems early in the development cycle, thereby minimizing the chances of issues reaching the production stage.
• Enhancing code quality: Continuous code analysis fosters the overall enhancement of code quality within development teams. This encompasses upholding consistent coding standards, optimizing code for performance, and adhering to industry best practices.
• Security fortification: Code analysis tools also serve as guardians against security vulnerabilities and code weaknesses. By pinpointing these vulnerabilities, they enable developers to proactively rectify them before malevolent actors can exploit them. This proactive stance is instrumental in thwarting security breaches and safeguarding sensitive data.
• Bolstering maintainability: Code analysis bestows developers with valuable insights into the maintainability of their code. This, in turn, simplifies the task of understanding, maintaining, and enhancing the codebase over extended periods of time – a particularly vital aspect for long-term projects and open-source software initiatives.
• Streamlining code reviews: Code analysis tools lend an efficient hand in the code review process. They automatically flag potential issues and offer details about them. This not only streamlines the review procedure but also ensures that reviewers concentrate on the crucial facets of the code.
• Enforcing consistency: Code analysis plays a pivotal role in preserving coding standards and fostering consistency throughout the codebase. This consistency is upheld even in expansive and distributed development teams.

In summary, code analysis stands as an indispensable practice for IT projects. It serves as the bedrock for maintaining code quality, reinforcing security, and upholding consistency. This proactive approach empowers developers to unearth and rectify issues early in the development process, thus minimizing the likelihood of defects and vulnerabilities in the final software product.

Current state of openIMIS

The project has undergone significant improvements with a focus on enhancing and maintaining project quality. Initially, unit tests were primarily implemented in backend modules, and while there was an existing CI/CD process, it was distributed across repositories and challenging to maintain. 

Over time, our team, in collaboration with the SwissTPH team, has worked to improve the process, and enhance verification of the application lifecycle. When a pull request is created, tests and static code analysis checks are provided by Sonar for a specific module and its dependencies in scope of the openIMIS infrastructure by the automatically executed GitHub Actions pipeline. Naturally, we’ve raised a number of  tests (various types, such as integration and unit tests) in scope of the modules. Examples of tests have also been included in module templates. 

Subsequently, we also configured SonarCloud code scan in each pull request. This initiative was driven by our desire to ensure we have analysis for security, which is a critically important factor in terms of having big projects working on production environments. 

With these checks in place, we have high confidence that the application is constructed correctly and the tests cover critical business aspects. Code coverage measures the extent to which the code in a software application has been tested by automated tests. It helps identify areas of code that may not be adequately tested, ensuring a more comprehensive assessment of the software’s reliability and functionality. This provides us with a safety net when making any changes, whether they are minor or substantial, to the codebase.

In the context of the frontend, we implemented checks to validate the correct building of modules, ensuring that no errors occur during the application building process. We’ve made some interesting discoveries through these analyses, including a significant number of issues related to how regular expressions are defined. Additionally, we’ve come across problems in the code related to properties (props) within the ‘insuree’ module (which is also categorized as a ‘security’ concern). Further details will be provided in our upcoming blog posts.

However, it’s important to note that there are currently no automatic tests at the frontend part, resulting in a lack of test-related data and reporting. This has led to a code coverage of zero, indicating the need for future work in this area. We plan to implement End to End (E2E) tests for openIMIS frontend modules using Playwright. E2E tests are a type of software testing that assess the functionality of an application from start to finish, simulating real user scenarios to ensure all components work together as intended.

Our plan is to add automatic checks for security standards in the industry. This kind of checks will rely on The Open Worldwide Application Security Project (OWASP) standards, SAST (Static Application Security Testing), DAST (Dynamic Application Security Testing), and SCA (Security Control Assessor) into our project’s security measures. Our initial focus is on OWASP Top Ten security standards, and you can find more details about these upcoming improvements in the ‘Goals and Expectations’ section of this article.

Centralized Sonar Pipeline Setup

In this section, we will provide an explanation of the centralized Sonar pipeline, its configuration, and significance within the domain of a microkernel architecture in openIMIS.

Centralized Sonar Pipeline

By the ‘centralized Sonar pipeline’ we understand an organized method for examining and overseeing code quality, security, and various software metrics. Static code analysis tools play a significant role in this process.

The pipeline serves as a unified template for running tests across our modules. We specify parameters to initiate scans and tests in each module, while the overall context is defined in the assembly module. This approach enables us to easily manage module configurations, streamlining the process of running Sonar scans and tests. It enhances our ability to maintain consistency and efficiency in our testing workflows. Thanks to this pipeline, we can effortlessly incorporate configuration into the new or existing modules. It allows us to execute scans and tests with convenience and efficiency.

Setup of the Centralized Sonar Pipeline

The openIMIS system architecture is designed around a micro-kernel approach, featuring a core module known as the assembly. This approach is implemented on both the backend and frontend sides of the system. Below, you’ll find a brief comparison between the standard method of configuring a project on SonarCloud and the approach adopted in the openIMIS project.

Initial setup steps of project in SonarCloud with GitHub Actions:

• Login to SonarCloud
• Select the Organization
• Create a New Project
• Generate an Authentication Token
• Integrate with GitHub
• Configure Analysis Settings
• Set Up the CI/CD Pipeline Integration
• Run the First Analysis
• Review and Customize Analysis Results
• Schedule and Automate Analysis Runs
• Monitor Code Quality and Security
• Collaborate and Share Reports

Sonar’s Guide to CI based analysis

Setup in micro-kernel architecture based on openIMIS example:

• The solution relies on GitHub workflows, with configuration carried out through YAML format files with usage of third-party component SonarSource.
• In the assembly module, you can find a GitHub workflow file. This file is responsible for setting up test execution for each module. Additionally, it includes jobs that generate reports based on the flake-8 linter, coverage, and Sonar analyses.
• The final step of the Sonar scan configuration.

sonar_scan:
   name: Run Sonar Scan
   runs-on: ubuntu-20.04
   needs: [ci_module_psql_test, flake-8-linter]
   if: always()
   env:
     SONAR_PROJECT_KEY: ${{ inputs.SONAR_PROJECT_KEY }}
     SONAR_ORGANIZATION: ${{ inputs.SONAR_ORGANIZATION }}
     SONAR_PROJECT_NAME: ${{ inputs.SONAR_PROJECT_NAME }}
     SONAR_PROJECT_VERSION: ${{ inputs.SONAR_PROJECT_VERSION }}
     SONAR_PYTHON_VERSION: 3.8
     SONAR_FLAKE8_REPORT_PATHS: ../report/flake8-report.txt
     SONAR_SOURCES: ${{ inputs.SONAR_SOURCES }}
   steps:
     - name: Set environment variables with default values
       run: |
         if [ -z "${{ inputs.SONAR_PROJECT_VERSION }}" ]; then
           echo "SONAR_PROJECT_VERSION=1.0" >> $GITHUB_ENV
         else
           echo "SONAR_PROJECT_VERSION=${{ inputs.SONAR_PROJECT_VERSION }}" >> $GITHUB_ENV
         fi


         if [ -z "${{ inputs.SONAR_EXCLUSIONS }}" ]; then
           echo "SONAR_EXCLUSIONS='**/migrations/**,**/static/**,**/media/**,**/tests/**'" >> $GITHUB_ENV
         else
           echo "SONAR_EXCLUSIONS=${{ inputs.SONAR_EXCLUSIONS }}" >> $GITHUB_ENV
         fi

• We expect those environmental variables that are required by Sonar execution to be set by the module calling the job.
• Below, you’ll find the most critical aspect for comprehending how the setup for the modular approach differs from the standard one. This setup leverages the SonarSource component. With the ‘args’ section, we can readily parameterize the workflow triggers and apply this across all the modules within openIMIS.

- name: SonarCloud Scan
       uses: SonarSource/sonarcloud-github-action@master
       with:
         args: >
           -Dsonar.projectKey=${{ inputs.SONAR_PROJECT_KEY }}
           -Dsonar.organization=${{ inputs.SONAR_ORGANIZATION }}
           -Dsonar.projectName=${{ inputs.SONAR_PROJECT_NAME }}
           -Dsonar.projectVersion=${{ env.SONAR_PROJECT_VERSION }}
           -Dsonar.python.version=3.8
           -Dsonar.sources=current-module
           -Dsonar.python.coverage.reportPaths=report/coverage1.xml,report/coverage2.xml
           -Dsonar.python.flake8.reportPaths=report/flake8-report.txt
           -Dsonar.exclusions=${{ env.SONAR_EXCLUSIONS }}
           -X
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}  # Needed to get PR information, if any
         SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}

• Defined workflow is executed by each module using GitHub Action job call. The ‘uses’ allows executing the workflow defined in the assembly module with the appropriate module setup details provided as arguments for its execution. In this setup, each module executes this workflow when a developer creates a pull request, synchronizes it or reopens it. Additionally, it is worth noting the presence of parameterized properties within the ‘with’ clause, which was used instead of the standard sonar-properties.config file from the repository. Below, we have a code example of how the action call looks like in a module which is a part of the assembly package. 

name: Module CI
on:
 pull_request:
   types: [opened, synchronize, reopened]
 push:
   branches:
     - main
     - 'release/**'
     - develop
     - 'feature/**'
 workflow_dispatch:
   inputs:
     comment:
       description: Just a simple comment to know the purpose of the manual build
       required: false


jobs:
 call:
   name: Default CI Flow
   uses: openimis/openimis-be_py/.github/workflows/ci_module.yml@develop
   secrets:
     SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
   with:
     SONAR_PROJECT_KEY: openimis_openimis-be-invoice_py
     SONAR_ORGANIZATION: openimis-1
     SONAR_PROJECT_NAME: openimis-be-invoice_py
     SONAR_PROJECT_VERSION: 1.0
     SONAR_SOURCES: invoice
     SONAR_EXCLUSIONS: "**/migrations/**,**/static/**,**/media/**,**/tests/**"

• SONAR_EXCLUSIONS allows us to exclude files primarily associated with migration files and static files, which are typically auto-generated or irrelevant from the code analysis perspective. Thanks to this configuration, we have a more accurate coverage report, with no linter for unnecessary items like automatically added migrations.
• Once the configuration is done, we can test this by creating sample PR and check if tests are run with execution steps related to Sonar code scanning with reports.

This GitHub module CI workflow schema reveals a backend module build and a successful Sonar scanning process. However, it indicates issues with running unit tests and encountering flake-related problems which have to be fixed before merging PR. The final step produces a Sonar report containing valuable metrics. Additionally, we also obtain side products from running CI, such as coverage reports and linter files.

Project page in SonarCloud software

This section provides fundamental information about the Sonar project. Here, you can observe the project’s association with a specific openIMIS module. Additionally, you’ll find essential settings and properties related to the programming language used and its connection to the organization, among other details. The settings provided here are crucial for the scanning process to effectively gather the required information about the state of a particular component within the entire application.

On the project’s overview page, you’ll find essential information and statistics related to code quality. This includes metrics such as code coverage (the percentage of code covered by tests), the percentage of reviewed security hotspots, and the percentage of duplicated code. In the screenshot above, we have core metrics that are typically presented after completing the scanning process and reflecting a high level quality overview. 

The first metric is the ‘Security Hotspots Percentage of Review,’ which plays a key role in our upcoming blog series focusing on security management. The next metric pertains to code coverage, as mentioned earlier. The final metric relates to the percentage of code duplication. 

The code coverage metric reveals areas lacking test coverage, while the code duplication percentage highlights places where code improvements are needed for enhanced code quality. Additionally, the quality gate provides us with an overall grade, which can be configured at the Sonar organization level.

The most important type of metrics provided by scanning in Sonar are:

Maintainability:

• This metric focuses on the maintainability of your code. It assesses how easy it is to understand and modify the code over time.
• Metrics related to maintainability often include code complexity, code duplications, and the presence of coding standards. Higher maintainability scores indicate code that is more straightforward to maintain and extend.

Security Review:

• The Security Review section deals with code security. It examines the code for potential security vulnerabilities and threats.
• It looks for issues like SQL injection, cross-site scripting (XSS), and other security vulnerabilities. A high score in this section indicates a codebase that is more secure against common security threats.

Security (Vulnerability):

• This section is closely related to the Security Review but is specifically focused on identifying and addressing security vulnerabilities within the code.
• It provides detailed information about the types of vulnerabilities found in the code, their severity, and recommendations for mitigating these issues.

Reliability:

• The Reliability section assesses the reliability and stability of our code. It identifies potential issues that could lead to system failures or unexpected behavior.
• Metrics in this section often include error-prone code patterns, exception handling, and potential bugs. A high reliability score means that your code is less likely to encounter issues related to software stability and reliability.

Goals and Expectations

By strengthening our security measures, we not only protect the digital product from potential threats but also have a positive impact on its maintainability. Improved security practices result in a stronger and more stable codebase, which reduces the chances of urgent maintenance due to security issues. This allows our team to concentrate on planned and proactive maintenance activities, ultimately enhancing the overall sustainability and long-term value of the digital assets. It’s a significant step to ensure the security of applications by identifying and addressing potential vulnerabilities. Code with security issues should not be merged and existing issues have to be addressed. 

openIMIS project currently conducts periodic security tests to ensure application safety; we want to elevate it by adding automatic security tests into the CI. Our objective is to establish continuous, near real-time monitoring of the application’s security, ensuring that security is an ongoing and integral part of our development process.

Additionally, we are aware that the default SonarCloud scanner sometimes produces false positives when examining files related to tests, particularly when configuring test-related credentials. That is also the reason why we want to extend security checks. Our upcoming security checks will also encompass aspects like SAST, DAST, and SCA. We can identify potential issues that can be addressed in the near future. By implementing these additional checks, we ensure that our projects are heading in the right direction.

Our long-term goal is to inform and foster the culture of the secure and quality driven approach for creating the software solutions. In such projects the extensive and highly sensitive data is processed in production environments. Therefore, security is an absolute necessity, and we’re determined to contribute to the security-first approach. Our aim is not only to make things simpler but also to play a part in ensuring software projects treat security as an important indicator of quality. 

Conclusion

We have the plan to add additional security checks/reports into the CI/CD pipeline. Our goal is to enhance the software’s security, reliability, ease of maintenance, and overall quality to deliver increased value. We’ve begun by implementing essential tools to gather code quality metrics, for example in the openIMIS project.

We’ve highlighted the importance of code analysis, and how it helps us identify and prevent issues, enhance code quality, fortify security, and maintain code in the long term. Our plan includes expanding our checks to comply with industry quality standards in every project we work on.

In the near future, we’ll be sharing blog posts rich in insights on how we manage security and performance settings. This is crucial, especially in large-scale commercial projects where we handle extensive and sensitive data in production environments. Demonstrating our commitment to addressing these concerns positions us as a trusted partner for similar projects in the future.

Join us on this journey, as we prioritize security and performance to deliver high-quality solutions for modern challenges. We’re fostering the culture of security-first high quality solutions for modern day challenges.

Technologies used