Mar 16, 2023

JupyterLab base project template

JupyterLab has become integral to my daily workflow. Whether it is creating a single processing pipeline or multi-notebook data visualizations.

It was always a pain to set up the repositories, with automatic versioning, and a Continuous Integration (CI) pipeline that pushes results and HTML conversions to a public web server.

I have created a git template repository that does all that automatically: https://gitlab.hrz.tu-chemnitz.de/tud-ifk/jupyter_base_template

It is based on cookiecutter and asks a number of questions, before initializing the repository and CI, using the Gitlab API.

These are the full steps:

  1. The project is cloned into a subdirectory with name project_slug.
  2. README.md is updated with title, referenced (badge-urls, internal links) are updated.
  3. Semantic-release config setup.cfg is configured with the Gitlab Domain
  4. .gitlab-ci.yml is configured with the remote pages namespace (project_slug)
  5. The template notebook in md/notebook.md is updated with project_name and description. Links to badged and repository are updated.
  6. The git submodule py/modules/base is cloned and added to the project
  7. The git remote repository and remote path are set
  8. The Gitlab repository is created via API
  9. All protected variables that exist in the Gitlab template repository are fetched and set in the new Gitlab repository.

Here’s an example for the final layout of HTML conversion.

There’s also a submodule automatically added to projects that includes my base-python tools (downloading unzipping etc.) that I can use in many notebooks, but only want to version in a single central repository.

The CI pipeline is set up using a restricted SSH key (see rrsync) that is limited (server-side) to a specific directory - should it become compromised.

See in action

Here’s a recording of the tool, where I create a sample project from the IOER template.

asciicast

See Also