r/Python • u/Business-Feedback635 • 2d ago
Discussion 684 PyPI packages run code at every Python start, and only 13 are ready for PEP 829
Python runs any line that starts with import in a .pth file in site-packages, every time it starts. You don't have to import the package. This year, the malicious releases of litellm (March and October) and agent-framework (November) all used a .pth file, so installing them was enough to run the credential stealer.
I wanted to see how many ordinary packages use the same feature. I took the latest release of 354,763 PyPI projects, found every .pth file with an import line, downloaded them and read them. The results:
- 684 packages install a
.pthfile that runs on every Python start. - 244 of them run their own code: setuptools, coverage, coloredlogs, pywin32 and others. The other 440 only run old setuptools namespace boilerplate (
-nspkg.pth). - coverage and coloredlogs do nothing unless you set an environment variable. urllib3-future copies its own files over your
urllib3folder at startup. - 52 packages put the file where Python never reads it on Linux or macOS, including files from temporary build folders that were packed by mistake.
- Python 3.15 deprecates these
importlines (PEP 829), and 3.18 will ignore them. Only 13 of the 244 packages already ship the replacement.startfile.
The post includes a command that lists the .pth files with import lines in your own environment, and a table of what the 15 most downloaded ones do: https://py-code.org/blog/python-pth-files-pypi/
90
u/zurtex 2d ago
PEP 829 was only approved earlier this year, Python 3.15 only just came out and all it does is deprecate .pth.
It's common for libraries not to migrate until things are about to break or just after they do break. Welcome to software maintenance.
38
u/michallandry62 2d ago
Worth saying for anyone skimming: .start files don't really fix the supply chain part. A malicious wheel can still ship one and get code running on every interpreter start. What changes is it has to be a named module:function entry point instead of arbitrary code crammed onto an import line, so it's a lot easier to grep for and audit. The real guard is still at install time (pinning with hashes, waiting a few days before taking new releases, actually looking at new deps).
The urllib3-future thing is wild though. A package overwriting another package's files at startup is exactly the kind of behavior I'd never think to check when debugging some weird urllib3 issue.
10
u/Nangangagatngpokpok 1d ago
this is one of those Python behaviors that don't know people exists until it shows up in supply chain attack. Kind crazy that hundreds of packages can run code on startup and only 13 have already moved to the replacement mechanism, esp with PEP 829 giving plenty of warning
7
u/shinitakunai 2d ago edited 2d ago
Wouldn't the same happen with .start file?
7
u/RedEyed__ 2d ago
How about code execution during pip install?
1
u/patryk_builds 22h ago
Good timing for me, i published my first package to pypi today and had no idea a wheel could do this. i checked what's inside mine before uploading code, py.typed, licence, but only because i was worried about shipping the wrong files. does anything in here run at interpreter start wasn't a question i knew to ask.The 52 packages that put the file where python never reads it are the funny part. dead on arrival, and probably nobody noticed, because the feature silently not working looks the same as it working.
Two questions:
- The 440 with only the old namespace boilerplate: are those all abandoned, or do they include packages people still install a lot? if they're still popular, 3.18 breaks namespace imports for them without anyone having done anything wrong recently
- 13 of 244 is about 5%. did you open issues on the big ones, or is the plan to let the deprecation warning in 3.15 do the work?
Going to run the command from your post on my own environment tonight
-10
u/bedel99 2d ago
Wow some one discovered AI