Skip to content

phoenixcitoolkitpython logo

pipeline status Latest release coverage report documentation Phoenix2 documentation DOI

Phoenix CI Toolkit Python

PhoenixCIToolkitPython is a gitlab's CI/CD component catalog implementing CI and CD operations shared by most phoenix2 repositories implemented in python, while managing their environments with pixi

The catalog includes CI components to - run tests, generating test and coverage reports - full pipeline component combining all the above, plus components from PhoenixCIToolkit in a standard pipeline for phoenix2 python projects.

The catalog's components can usually be configured to run in specific docker images. The workflow component however imposes the docker images to come from the phoenixcontainer registry

Table of content:

[[TOC]]

Repository content

The components are tested in the CI pipeline of this repository, which is therefore structured as a standard python phoenix project. The project rely on pixi to manage environments and dependencies, and on pixi tasks to standardize usual development operations.

🏛️ Repository structure: 

📁 doc
|─── 📁 doc                                ──► Documentation sources and configuration for mkdocs and mkdocstring
📁 src/phoenixcitoolkitpython              ──► Dummy python package with an entrypoint
📁 tests                                   ──► Dummy test suite of the package
📁 templates
|─── 📄 phoenix-workflow-python.yml        ──► "Meta" component that combines all others in a complete pipeline suited to most phoenix2 projects
|─── 📄 pixi-test-python.yml               ──► Component to run tests, producing test and coverage reports
📄 .coveragerc                             ──► coverage configuration file
📄 .gitlab-ci.yml                          ──► CI/CD testing the components
📄 .notareleaserc.yml                      ──► Semantic release configuration for this repository. It is moved to .releaserc.yml in the CI, so the default configuration from the CI component can be tested
📄 pyproject.toml                          ──► python project configuration file
📄 LICENSE                                 ──► CeCILL-C license
📄 pixi.toml                               ──► Contains the information about pixi's environment, tasks and dependencies
📄 pytest.ini                              ──► Pytest configuration file
📄 ruff.toml                               ──► ruff configuration for code formatting and linting

Components

Component pixi-test-python

The pixi-test-python component calls pixi run --frozen $[[ inputs.pixi-task ]], calling a "test" task defined in the pixi manifest. The task is passed arguments to define the output reports: "coverage_dir", "junit_report", and "cobertura_report".

Example

  - component: $CI_SERVER_FQDN/CTA-LAPP/PHOENIX_LIBS2/citoolkit/phoenixcitoolkitpython/pixi-test-python@4.0.0
    inputs:
      rules:
        - if: $CI_COMMIT_TAG
        - if: $CI_PIPELINE_SOURCE != "merge_request_event"
      needs:
        - job: ReplacePkgVersion
      cobertura-report: cobertura_report.xml
      coverage-dir: htmlcov
      job-name-prefix: myproject
      job-image: "gitlab-registry.in2p3.fr/cta-lapp/phoenix_libs2/citoolkit/phoenixcontainer/pixi:8.0.0@sha256:9d87d297b497b48820a30b34c0aa1388d9314b066c9d0ba2452767f830537046"
      junit-report: test_report.xml
      pixi-task: test
      tags: ["runner_tag"]

Inputs

Name Type Default Value Description
additional-artifacts-paths array [] Paths to the additional artifacts. Use this to add as artifacts such as log files written by test programs to the job artifacts
artifacts-expire-in string "8d" Expiration time for the coverage artifacts
cobertura-report string Filename, relative to coverage-dir, to use to write the coverage report in cobertura (xml) format.
coverage-dir string Directory where the coverage report is located
job-image string Gitlab "image" section of a CI/CD job definition, defining the image the job will run in
job-name-prefix string Job Name prefix, can be used to differentiate several inclusion of this component in a pipeline. Final job name will be job-name-prefix-test
junit-report string Filename to write the test report in junit format, relative to the current directory
needs array [] Gitlab needs array with the jobs that must run before the job (also controls artifacts available in job)
pixi-env string "" Pixi environment to run the test task into. If empty, it is left to pixi to determine (default environment or task default environment)
pixi-task string Name of the pixi task running the coverage. Will be used as pixi run --frozen $[[ inputs.pixi-task ]] args
rules array Gitlab rules to trigger the job
tags array [] Tag list for the job (to control which runner is used)

Component pixi-ruff-lint

The pixi-ruff-lint component calls pixi run --frozen $[[ inputs.pixi-task ]], calling a "lint" task defined in the pixi manifest. The task should call ruff check ; the arguments --output-format=gitlab and output-file=$[[ inputs.code-quality-report ]] will be passed passthrough for ruff check to generate a code quality report available in merge request.

Example

  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/pixi-ruff-lint@4.0.0
    inputs:
      rules:
        - if: $CI_COMMIT_TAG
        - if: $CI_PIPELINE_SOURCE != "merge_request_event"
      needs:
        - job: replace-pkg-version
      code-quality-report: ruff_check.json
      job-image: "gitlab-registry.in2p3.fr/cta-lapp/phoenix_libs2/citoolkit/phoenixcontainer/pixi:8.0.0@sha256:9d87d297b497b48820a30b34c0aa1388d9314b066c9d0ba2452767f830537046"
      job-name-prefix: myproject
      pixi-env: lint
      pixi-task: lint
      tags: ["MUST_big_runner"]

Inputs

Name Type Default Value Description
additional-artifacts-paths array [] Paths to the additional artifacts. Use this to add as artifacts such as log files written by test programs to the job artifacts
allow_failure string true Allow the job to succeed even if ruff check finds issues
artifacts-expire-in string "8d" Expiration time for the coverage artifacts
code-quality-report string Filename to write the code ruff report as a code quality report for gitlab.
job-image string Gitlab "image" section of a CI/CD job definition, defining the image the job will run in
job-name-prefix string Job Name prefix, can be used to differentiate several inclusion of this component in a pipeline. Final job name will be job-name-prefix-lint
needs array [] Gitlab needs array with the jobs that must run before the job (also controls artifacts available in job)
pixi-env string "" Pixi environment to run the test task into. If empty, it is left to pixi to determine (default environment or task default environment)
pixi-task string Name of the pixi task running the coverage. Will be used as pixi run --frozen $[[ inputs.pixi-task ]] args
rules array Gitlab rules to trigger the job
tags array [] Tag list for the job (to control which runner is used)

Component phoenix-workflow-python

The phoenix-workflow-python component implements a complete pipeline for standard phoenix2 python projects, by using the test component defined in this repository and other components from PhoenixCIToolkit. The pipeline will: - determine the version of the package to produce - either the branch's name if on a branch, or the tag value if on a tag - and replace it in the relevant files - build the project package - run the test, generating test and coverage reports - generate the documentation - upload the package to a conda channel - publish the documentation - publish a release if the commit should trigger one

Example

  - component: $CI_SERVER_FQDN/$CI_PROJECT_PATH/phoenix-workflow-python@4.0.0
    inputs:
      pixi-env-documentation: doc
      pixi-env-lint: lint
      pixi-env-test: test
      pixi-env-snippets: snippets
      pixi-task-documentation: doc
      pixi-task-lint: lint
      pixi-task-test: test
      pixi-task-snippets: snippets
      snippets-dir: doc/doc_src/snippets
      package-name: mypackage
      tags: ["runner_tag"]

Inputs

Name Type Default Value Description
additional-test-artifacts-paths array [] Paths to the additional artifacts. Use this to add as artifacts such as log files written by test programs to the job artifacts
eossr-extra-args string "" Extra arguments passed to eossr-upload-repository. Default is "" (no argument passed).
pixi-env-documentation string Pixi environment to run the doc task into
pixi-env-lint string "" Pixi environment to run the lint task into
pixi-env-test string Pixi environment to run the test task into
pixi-env-snippets string "" Pixi environment to run the snippets task into. Can be left empty if there are no snippets
pixi-task-documentation string Pixi task that generates the documentation. Forwarded to the "pixi-run-doc" component
pixi-task-lint string "notatask" Pixi task that runs ruff check. Forwarded to the "pixi-ruff-lint" component. If not specified, the default value "notatask" deactivates the ruff lint job
pixi-task-test string Pixi task that runs the test and generates junit test report. Forwarded to the "pixi-test-cpp" component
pixi-task-snippets string "" Pixi task to execute/render code snippets/notebooks. Default is "notatask" so users don't have to specify a task if snippets-dir is not set and snippets job won't run
package-name string Name of the package as written in the pixi.toml file
snippets-dir string "notasnippetsdir" Optional directory containing code snippets which would be executed/rendered during the snippets task. Default is "notasnippetsdir" in which case the job doesn't run
tags array [] Tag list of the job (to select runner)
upload-channel-dev string phoenix-dev Channel at the Host where the "dev" package will be uploaded, for instance "phoenix-dev". Dev packages are the packages build for commits that do not correspond to a tag or release.
upload-channel-stable string phoenix Channel at the Host where the "stable" package will be uploaded, for instance "phoenix". Stable packages are the packages build for tags or release.
zenodo-project-id string "" Zenodo project id, passed as "-id xxx" to eossr-upload-repository. Default is "", in which case no ID is used