Upgrading a Database Project to Python 3.12
Upgrading to Python versions can be tricky, particularly for Python 3.12. This article describes an experience for a database-oriented project.
Join the DZone community and get the full member experience.
Join For FreeUpgrading to major versions of Python (e.g., 3.12) can be non-trivial; here's a good article for reference. I recently upgraded API Logic Server, and offer this information in hopes it can make things a bit easier for you.
Aside: API Logic Server is open source. It creates executable API/Admin App projects from a database with 1 command. Customize with rules and Python in your IDE.
There were 2 areas that required attention:
- Packaging: Preparing a project for
pip install
access- This issue was unique to Python 3.12: the old setup procedures have been removed.
- Dependent libraries: This is a consideration for any new release. In general, I found this page helpful.
My project is database-oriented (using SQLAlchemy), so key risk areas usually involve database access. MySQL and Oracle are generally straightforward, but I always need to address Postgres (psycopg) and SQL/Server (pyodbc). These affect requirements.txt
and product packaging.
Let's consider packaging first.
Project Packaging
My project requires packaging for PyPi. This has changed in Python 3.12.
Let's go over some quick background. To make a package available for pip install
, you must upload it to PyPi. Here's an uploaded example. This is 2 step process as follows:
- Build local install files: This gathers your dependent libraries, CLI entry points, and so forth.
- Upload to PyPi: This is unchanged:
python3 -m twine upload --skip-existing dist/*
.
The first step has changed in two ways:
- How you run the setup process
- How you specify your dependent libraries
Run Setup (Dependencies)
This process prepares for python3 -m twine upload...
, by creating local files that identify the libraries you require, CLI entry points, and so forth.
In the past, you ran python3 setup.py sdist bdist_wheel
. That is no longer supported. It has been replaced by:
python3 -m build
pyproject.toml (Not setup.py)
In the past, your setup.py
file identified the libraries you require, CLI entry points, and so forth. setup.py
is no longer supported in Python 3.12. Instead, you must provide a pyproject.toml
file, as described in this guide. The python3 -m build
uses this file.
For me, this set off a mild panic: I was unable to find a setup-to-toml migration utility, except for those looking to replace the entire pip install
workflow.
As it turned out, migrating setup.py
was not so painful by hand; mainly a series of copy/paste procedures as shown below. Here's a working pyproject.toml
shown in the diagram below.
psycopg2-binary: Postgres
This is used by SQLAlchemy for Postgres access. In addition to pyproject.toml
, I had to change requirements.txt
, as shown here. I changed psycopg2-binary==2.9.5
to:
psycopg2-binary==2.9.9
My project is large, so I found it convenient to create a small venv
, and test the install. It took a few tries to straighten out the -binary
bit.
odbc: SQL/Server
Microsoft SQL/Server requires 3 packages (this on a Mac):
unixodbc
Install unixobdbc. You might get:
==> Running `brew cleanup unixodbc`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Removing: /opt/homebrew/Cellar/unixodbc/2.3.11... (48 files, 2.3MB)
Warning: The following dependents of upgraded formulae are outdated but will not
be upgraded because they are not bottled:
msodbcsql18
(venv) val@Vals-MPB-14 Desktop %
It seemed to work.
odbc driver
I required the Microsoft odbc driver.
pyodbc
This is used by SQLAlchemy. In requirements.txt
and pyproject.toml
, I had to change pyodbc==4.0.34 --> pyodbc==5.0.0.
Minor Issues: Escape Characters
As noted in Python docs, mistakes in strings (e.g., \but I forgot the n
) were previously not flagged; now they are.
I mention this because unexpected messages show up when you start your program under the debugger.
Opinions expressed by DZone contributors are their own.
Comments