Create a GPG key for verification on Github:
At step 3 I had to use
gpg --default-key rsa4096 --gen-key.
Create a PyPI account for yourself if you don’t already have one. Sign up page. Get an owner to add you as a maintainer or owner.
setup.pyfiles, usually at the top level of a repository.
CHANGES.mdtext in each repository.
CONTRIBUTING.md. Ensure that all contributors are listed. You can do this using Github, or locally with
git shortlog -sn
git logto ensure that you have the correct version and tag.
cd my/repository/ # This may seem slow as this will build the wheel as well as the source # distributions python3 setup.py bdist_wheel sdist # check that you've successfully created wheel and source files ls dist/ # Tidy up rm -r dist
For any projects which are auto-documented by cylc-doc, currently:
Ensure the docs build against master by manually triggering the test workflow in cylc-doc.
This will catch syntax errors, broken urls etc which need to be fixed prior to releasing the project.
# Examine previous tags using git tag -ln # Create a new tag "tag_name" is usually a version number. git tag -a -s <tag_name> # Push tags: git push --tags upstream <tag_name>
# Build your distribution inside the repository folder this time cd my/repository # Create your build python3 setup.py bdist_wheel sdist # Check that the dist folder contains the right artifacts: ls dist/ # Upload your build to PyPI. n.b. This will not work if your build has the # same version number as one already on PyPI. twine upload dist/*
Check PyPI for your upload.
NOTE: the GitHub and PYPI releases are prerequisites for releasing to Conda Forge!
The Conda Forge release process is mostly automated by bots. These bots are set up during the initial project bootstrap, which has been done for all the projects that are part of Cylc 8.
As there are inter-dependencies amongst the Cylc 8 parts, you should know the dependency tree, and start by the leaf nodes (i.e. a module A may have multiple dependants, but no dependency to other modules).
For example, Cylc UI Server depends on Cylc Flow. So unless you are releasing only Cylc UI Server, you should release Cylc Flow first.
On GitHub, navigate to the project repository on GitHub, e.g.
On each of these repositories, the release process should be pretty
much the same. Your work will be mainly (if not exclusive) on
This is a possible order of action:
sha256value (you can get this value from PYPI, or use some tool like
From past experience, I’ve seen Conda packages available in Conda Forge in under 30 minutes, but have also had to wait hours (<8 hours) for it to show in Conda Forge. For example, if Azure or GitHub infrastructure have issues (as I experienced during the first releases) it may take days. So keep that in mind before announcing the Conda packages.
To test packages locally, first you should make sure that Conda is
configured to avoid automatically upload the package. Open your
and check that you have something similar to:
channels: - defaults - conda-forge ssl_verify: true anaconda_upload: false
Now create a Conda environment for your tests, e.g.:
conda create -n cylc1,
and then activate it
conda activate cylc1. Then to build and install
# Where $CONDA_FORGE_REPOSITORY could be, for example, # cylc-uiserver-feedstock. cd $CONDA_FORGE_REPOSITORY # Your package should not be listed! conda list # This will take some minutes and print useful information. conda build recipe/ # The following command will install the locally created package. Before # installing it will ask you to confirm. Scroll up and search the # package name. The right-side column must show a location like # .../anaconda3/conda-bld/linux-64::cylc-uiserver-0.1-py37_1. # This confirms you are installing the local build. Here $PACKAGE_NAME # could be something like cylc-uiserver. conda install --use-local $PACKAGE_NAME
At this point you should be good to go. Test the package with commands
which $PACKAGE_NAME, or
$PACKAGE_NAME --version, etc.
Of if you are testing the metapackage, try running the complete system
with a workflow and the UI or tui, and check if there is nothing wrong.
conda-buildand `conda install
If you found problems while testing locally, try troubleshooting locally, and either mark the pull request as draft, and close it. Merging the pull request will create a release.
To undo a release, you will have to liaise with the maintainers via Gitter or Github. Or, alternatively, bump up the build number +1, and release it again.
conda-installis picking the wrong version or build of a package
conda install cylc returns the incorrect build number, it could be
an issue that is very difficult to debug (a posteriori). In fact it may not even be a bug:
Question to conda Gitter channel: (@hjoliver Jul 21 17:23):
I have a conda metapackage just published to conda-forge, which (on conda install) does not install the latest build numbers of several of our component packages. The conda docs cite bug fixes as a primary use case for incrementing build numbers, in which case I would have thought the environment solver should never select older builds? If I install the component packages directly, I do get the latest builds, but not via the metapackage. Anyone know what’s going on here?
@hjoliver this article describes how the conda solver works: https://www.anaconda.com/blog/understanding-and-improving-condas-performance. It specifically says that build numbers are only maximized for explicit specs
@jjhelmus: @hjoliver changes to the build number can also occur when a package is built against a different version of a library which is incompatible with earlier version. For example, one build of python 3.7.7 might be compiled against libffi 3.2 while the next build number against libffi 3.3. Depending on what other packages are installed in the environment it may not be possible to install the package with the largest built number. In the example if cffi is only available as a package built against libffi 3.2 then the older build of python 3.7.7 would be required.
Interpretation: build number is treated much like version number, if not pinned down exactly conda can choose an older build if that works best for solving all the dependencies in the system. Conda would presumably(?) choose the newest build if it does not affect dependencies; we may just need to check what happens if dependencies get changed by a new build.
If you are sure that you have your new version or build available from
Anaconda.org, and that
conda install --dry-run $package should
work, then you could have hit a bug in either
conda client or in
Conda Forge repository data.
You can try the following:
Conda documentation has details about their package math specification. Basically, there are three parts, where the last one is optional.
So you will have to specify the exact version, and use a glob expression for the build number. For example, from our alpha2 release:
run: - python - cylc-flow 8.0a2 *_3 - cylc-uiserver 0.2 *_2
In the example above, we are specifying the packages and versions we want.
Plus, we also give it a pattern to the build string, so conda will match
cylc-flow-8.0a2-py37_2, or ``cylc-flow-8.0a2-py38_2
cylc-flow-8.0a2-py37h2387c_2` (all valid build strings).
Bask in the glory of having created a release.