Developing Python Projects with poetry

I have been developing Python modules and applications for quite a while now using pipenv as my tool of choice for managing dependencies. However, most of the time, I find it slow when locking dependencies. pipenv is a nice tool but it is lacking features.

I recently got introduced to poetry from work. I have heard the tool before but I didn't got curious. Maybe because at the time, I find the documentations incomplete or not beginner-friendly. But right now it is a different story. poetry has progressed over the time adding great features and developers really support it and adopted it.

In this article, I will be migrating the Python module amortization from pipenv to poetry.

Cloning Amortization

The first step that we will be taking is to clone the project. You should have git installed in order to do this.

git clone git@github.com:roniemartinez/amortization.git
cd amortization

Installing poetry

Use pip to install poetry:

pip install poetry

Initializing the project for poetry

To initialize our project for poetry, run poetry init. In in example, we will be specifying required metadata for the project and the dependencies, interactively. I also used the version 1.0.1-rc1 in order to publish a test version first. amortization depends on tabulate but also depends on codecov, pytest, pytest-cov and pycodestyle for development. Notice that I did not include twine because poetry already has the ability to publish our package.

% poetry init

This command will guide you through creating your pyproject.toml config.

Package name [amortization]:  
Version [0.1.0]:  1.0.1-rc1
Description []:  Python library for calculating amortizations and generating amortization schedules
Author [Ronie Martinez <ronmarti18@gmail.com>, n to skip]:  
License []:  MIT
Compatible Python versions [^3.7]:  >=3.5

Would you like to define your main dependencies interactively? (yes/no) [yes] 
You can specify a package in the following forms:
  - A single name (requests)
  - A name and a constraint (requests ^2.23.0)
  - A git url (git+https://github.com/python-poetry/poetry.git)
  - A git url with a revision (git+https://github.com/python-poetry/poetry.git#develop)
  - A file path (../my-package/my-package.whl)
  - A directory (../my-package/)
  - An url (https://example.com/packages/my-package-0.1.0.tar.gz)

Search for package to add (or leave blank to continue): tabulate
Found 20 packages matching tabulate

Enter package # to add, or the complete package name if it is not listed: 
 [0] tabulate
 [1] django-tabulate
 [2] csv2tabulate
 [3] tabulatehelper
 [4] rtbhouse-sdk
 [5] amortization
 [6] paypal_reporter
 [7] kudne_list4
 [8] kudneHaru
 [9] kudne_list3
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^0.8.6 for tabulate

Add a package: 

Would you like to define your development dependencies interactively? (yes/no) [yes] 
Search for package to add (or leave blank to continue): codecov
Found 20 packages matching codecov

Enter package # to add, or the complete package name if it is not listed: 
 [0] codecov
 [1] robpol86-codecov
 [2] aiomessenger
 [3] Flask-Annex
 [4] ci
 [5] drfdocs-cadasta
 [6] slack-on
 [7] bottender
 [8] yaenv
 [9] import-ready
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^2.0.16 for codecov

Add a package: pytest
Found 20 packages matching pytest

Enter package # to add, or the complete package name if it is not listed: 
 [0] pytest
 [1] 131228_pytest_1
 [2] pytest123
 [3] pytest-winnotify
 [4] pytest-flakefinder
 [5] pytest-virtualenv
 [6] pytest-pmisc
 [7] pytest-stepwise
 [8] pytest-mypyd
 [9] pytest-couchdbkit
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^5.3.5 for pytest

Add a package: pytest-cov
Found 20 packages matching pytest-cov

Enter package # to add, or the complete package name if it is not listed: 
 [0] pytest-cov
 [1] pytest-cov-exclude
 [2] metrics.pytest-cov
 [3] pytest-func-cov
 [4] cov-core
 [5] doc-cov
 [6] nose-cov
 [7] nose2-cov
 [8] pytest-cover
 [9] resilient
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^2.8.1 for pytest-cov

Add a package: pycodestyle
Found 20 packages matching pycodestyle

Enter package # to add, or the complete package name if it is not listed: 
 [0] pycodestyle
 [1] pytest-pycodestyle
 [2] pycodestyle_magic
 [3] pytest-codestyle
 [4] yala
 [5] gsyslog
 [6] flake8-tabs
 [7] PycodestyleBear
 [8] flake8-checkstyle
 [9] flake8-expandtab
 > 0
Enter the version constraint to require (or leave blank to use the latest version): 
Using version ^2.5.0 for pycodestyle

Add a package: 

Generated file

[tool.poetry]
name = "amortization"
version = "1.0.1-rc1"
description = "Python library for calculating amortizations and generating amortization schedules"
authors = ["Ronie Martinez <ronmarti18@gmail.com>"]
license = "MIT"

[tool.poetry.dependencies]
python = ">=3.5"
tabulate = "^0.8.6"

[tool.poetry.dev-dependencies]
codecov = "^2.0.16"
pytest = "^5.3.5"
pytest-cov = "^2.8.1"
pycodestyle = "^2.5.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"


Do you confirm generation? (yes/no) [yes] 

Install dependencies

To install our project dependencies, run poetry install. In terms of creating .lock files, poetry is faster compared to pipenv.

% poetry install       
Updating dependencies
Resolving dependencies... (20.2s)

Writing lock file


Package operations: 21 installs, 0 updates, 0 removals

  - Installing zipp (1.2.0)
  - Installing importlib-metadata (1.5.0)
  - Installing pyparsing (2.4.6)
  - Installing six (1.14.0)
  - Installing attrs (19.3.0)
  - Installing certifi (2019.11.28)
  - Installing chardet (3.0.4)
  - Installing idna (2.9)
  - Installing more-itertools (8.2.0)
  - Installing packaging (20.3)
  - Installing pluggy (0.13.1)
  - Installing py (1.8.1)
  - Installing urllib3 (1.22)
  - Installing wcwidth (0.1.8)
  - Installing coverage (4.4.2)
  - Installing pytest (5.3.5)
  - Installing requests (2.23.0)
  - Installing codecov (2.0.16)
  - Installing pycodestyle (2.5.0)
  - Installing pytest-cov (2.8.1)
  - Installing tabulate (0.8.6)
  - Installing amortization (1.0.1-rc1)

Run tests

Now, we want to verify that nothing is broken. For this, we just run poetry run pytest:

% poetry run pytest
============================= test session starts =============================
platform darwin -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: <hidden>/amortization
plugins: cov-2.8.1
collected 2 items                                                             

tests/test_amortization.py ..                                           [100%]

============================== 2 passed in 0.02s ==============================

Great! Our tests still passed.

Publishing to TestPyPI

To publish our module, we first need to build it. Since we will be using poetry's system we, will also need to delete our existing setup.py and setup.cfg. They will not be needed in the project anymore.

% poetry build              
Building amortization (1.0.1-rc1)
 - Building sdist
 - Built amortization-1.0.1rc1.tar.gz

 - Building wheel
 - Built amortization-1.0.1rc1-py3-none-any.whl

Next step is to configure the TestPyPI repository and credentials. You can skip configuring credentials but you will be asked later to enter it, instead.

poetry config repositories.testpypi https://test.pypi.org/legacy/
poetry config http-basic.testpypi username password

Now, we can publish it to TestPyPI:

% poetry publish -r testpypi

Publishing amortization (1.0.1-rc1) to testpypi
 - Uploading amortization-1.0.1rc1-py3-none-any.whl 100%
 - Uploading amortization-1.0.1rc1.tar.gz 100%

To verify it, we need to check if our project is now available in TestPyPI by going to the URL https://test.pypi.org/project/amortization/1.0.1rc1/.

Adding metadata and console scripts

As you have noticed, metadata have been missing compared to the original project version 1.0.0.

Before modifying pyproject.toml manually, let us bump the version first:

% poetry version patch
Bumping version from 1.0.1-rc1 to 1.0.1

We can now modify pyproject.toml:

[tool.poetry]
name = "amortization"
version = "1.0.1"
repository = "https://github.com/roniemartinez/amortization"
description = "Python library for calculating amortizations and generating amortization schedules"
authors = ["Ronie Martinez <ronmarti18@gmail.com>"]
license = "MIT"
readme = "README.md"
classifiers = [
    "Development Status :: 5 - Production/Stable",
    "License :: OSI Approved :: MIT License",
    "Topic :: Office/Business :: Financial :: Accounting",
    "Topic :: Scientific/Engineering :: Mathematics",
    "Topic :: Software Development :: Libraries :: Python Modules",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.5",
    "Programming Language :: Python :: 3.6",
    "Programming Language :: Python :: 3.7",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: Implementation :: CPython",
]

[tool.poetry.scripts]
amortize = 'amortization.amortize:main'

[tool.poetry.dependencies]
python = ">=3.5"
tabulate = "^0.8.6"

[tool.poetry.dev-dependencies]
codecov = "^2.0.16"
pytest = "^5.3.5"
pytest-cov = "^2.8.1"
pycodestyle = "^2.5.0"

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

To validate pyproject.toml, run poetry check:

% poetry check
All set!

We can again publish our project:

poetry build
poetry publish -r testpypi

Publishing to PyPI

You may publish your module by running poetry publish but I still prefer to use Travis in order to test my module first. In .travis.yml, add before_deploy and update deploy.

before_deploy:
  - poetry config http-basic.pypi username password
  - poetry build

deploy:
  provider: script
  script: poetry publish
  skip_cleanup: true

Conclusion

Poetry makes it easier for developers to deploy and publish their applications. It is fast at creating .lock files compared to pipenv. Writing setup.py/setup.cfg for the Python modules is also not required anymore.

poetry is a tool that every Python developer need.

Read more articles like this in the future by buying me a coffee!

Buy me a coffee