Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scheduled monthly dependency update for January #205

Closed
wants to merge 14 commits into from
Closed

Scheduled monthly dependency update for January #205

wants to merge 14 commits into from

Conversation

pyup-bot
Copy link
Collaborator

@pyup-bot pyup-bot commented Jan 1, 2024

Update toil[cwl] from 5.0.0 to 5.12.0.

Changelog

5.11.0

Breaking Changes
* Imported files will be symlinked by default, unless the user sets `--noLinkImports` or the workflow imports with `symlink=False`. (3949)

WDL
* Toil will now stop if it encounters an error polling a possible import URL for a WDL workflow input file. (4479)
* WDL workflows will be protected against imported files with no basenames. (4477)

Misc
* Toil batch system ID numbers for issued jobs now start at 1. (4482)
* Attempts to import files from URLs when the implementing job store is missing an extra are now better reported. (4479)
* Include tests in the source distribution that gets published to PyPI (4499)

_Bug Fixes_
* Toil should no longer crash when a delete wins a race against a load in `FileJobStore` (4484)
* Prevent local root jobs (such as WDLRootJob) from being run twice. (4482)
* Slurm and other grid batch system jobs will now have more informative names (4472)
* WDL workflows can no longer import `""` as a File. (4477)

Thank you to our contributors: stxue1, DailyDreaming, mr-c, adamnovak

5.10.0

Changelog

Highlighted Features Added
* Add a `--caching` option which explicitly states whether to use caching with a workflow. Uses a default value depending on whether or not we are using the file job store if not specified. (4218)
* New prototype WDL runner `python -m toil.wdl.wdltoil` using MiniWDL (3468)
* MiniWDL-based WDL implementation can now run the vg Giraffe WDL workflow ( 4353)
* Toil now tests against our own tiny set of WDL conformance tests (4351)
* Toil can run the HPRC assembly WDL workflows (4435)
* Toil can now use Mesos roles (4455)

Breaking Changes

* Replace "preemptable" with "preemptible", add example of using --defaultPreemptible flag to Preemptibility documentation (1951)

CWL
* CWL: run all ExpressionTools on the Leader node, instead of submitting separate jobs (4157)

Kubernetes
* Kubernetes batch system: Delete jobs individually when batch delete fails (3403)
* Documentation for running a Toil leader for a Kubernetes workflow outside Kubernetes now covers examples and common problems for running CWL workflows (document toil-cwl-runner + "Running the Leader Outside Kubernetes" 3422)
* Kubernetes batch system: support `--maxCores`, `--maxDisk`, and `--maxMemory` (2864)
* Add tutorial for Kubernetes launch cluster (3743)


Dependencies

* Require htcondor 10 exactly (4315)
* Toil jobs now have a `local` parameter which determines if they should run on the leader. (4388)

Misc

* The offline tests can now be run in parallel (3493)
* Code updated to be more idiomatic for Python3.7 (4295)
* Support for a `--network` for `toil launch-cluster` for Google cloud (4196)
* Support for a `--use_private_ip` for `toil launch-cluster` to dial nodes by private IP instead of public IP (4196)
* GPU scheduling should now be supported on Slurm (4308)
* Toil now supports a `--batchLogsDir` option and `TOIL_BATCH_LOGS_DIR` environment variable, to provide a directory other than the work dir where Toil will instruct HPC batch systems to save their captured job logs.
* `htcondor` batch system should now work again, and will retry connections
*  Updated the --coalesceStatusCalls help documentation to reflect the current state of https://github.com/DataBiosphere/toil/issues/4431 (#4437)
* Toil no longer trusts XDG_RUNTIME_DIR under Slurm (fixes some of the issues behind 4395 when Slurm is configured not to follow the XDG spec) (4435)
* Toil now puts it lock files for Singularity cache directories for WDL in those directories (4435)
* Toil's WDL interpreter can now use local-to-the-leader jobs for evaluating WDL code that doesn't need appreciable resources (4388)
* Toil now tolerates more possible exceptions related to the panasas network file system (4440)
* Type hinting to functions in resource.py (938)
* Added return type to inVirtualEnv() in `__init__.py` (938)
* Added None checks to some function bodies (938)



_Bug Fixes_
* Stop crashing when predefined batch job exit reasons are used and need to go into the message bus log file (4321)
* Added `import subprocess` to restore the behavior of 588. (4429)
* Toil will no longer use the stored message bus path from an old execution of a workflow when deciding where to save the message bus log when restarting a workflow (4438)
* Fix --custom-net mutual exclusivity bug. (4458)


Thank you to our contributors: stxue1 , DailyDreaming , mr-c , adamnovak , jfennick , misterbrandonwalker , w-gao , stephanaime , glennhickey , Hexotical , manabuishii gmloose , boukn , and thiagogenez !

5.9.2

Changelog

_Bug Fixes_
* Change build tag import (4329)

Thank you to our contributors: adamnovak , Hexotical !

5.9.0

Changelog

_Bug Fixes_
* Fix --provisioner and --metrics together (4328)
* Ignore incorrect type hint from boto3, remove json.loads (4330)
* Warn about missing --bypass-file-store with in-place update (4337)
* Replace prepareHTSubmission with prepareSubmission in HTCondor (4319)
* Merge "Google fixes" (4293)
* Support (only) current htcondor (4320)
* Delete k8s jobs individually when batch delete fails (4306)

Misc
* Update aws spot documentation (4310)
* Enable parallel testing (3493)
* Add documentation for running CWL workflows on non-Toil-managed Kubernetes clusters (4332)
* Export all slurm args by default (4237)
* Allow for subclasses of base types in messages (4322)
* Non cache default (4299)

Dependencies
* Bump mypy from 0.982 to 0.991 (4345)
* Bump schema-salad>=8.4.20230128170514,<9 to schema-salad>=8.3.20220913105718,<8.4 (4342) (4341)
* Bump cwltool from 3.1.20221008225030 to 3.1.20221201130942 (4338)
* Bump pyupgrade to 3.7 (4295)

Thank you to our contributors: adamnovak , Hexotical , w-gao, mr-c , gmloose , boukn , and thiagogenez !

5.8.0

Changelog

Highlighted Features Added

* Toil server now exposes workflow tasks via WES (4046).
* Toil server now has a `--wes_dialect agc` option that will hide any tasks that don't have Amazon Batch job IDs, and put the IDs in the task names for those that do (4047).
* Toil jobs now accept an `accelerators` requirement, like `accelerators=1` or `accelerators={'kind': 'gpu', 'brand': 'nvidia', 'count': 2}` (4163)
* Include total requested cores for each job type in `toil stats` (4173)
* Toil jobs now expose `job.accelerators` to workflow 
* Add prefix suffix params to `AbstractFileStore.getLocalTempFile` and `AbstractFileStore.getLocalTempFileName` (4273)
* CWL: `--no-compute-checksum`, `--strict-cpu-limit`, `--disable-validate`, and `--fast-parser` are now available


Breaking Changes
* Toil's built-in autoscaler now guesses that some memory and disk space on nodes will not actually be available for jobs; pass `--assumeZeroOverhead` to revert to the old behavior (2103) 

CWL
* CWL job unit and display names have been changed to make more sense as task names, and management of them has been unified into a `CWLNamedJob`. (4046/4047)
* CWL `CUDARequirement` is parsed by `cwltool` and turned into a requirement for the minimum requested number of nvidia GPU accelerators (3982)
* fix false warning when outputSource contains only one None value (4300)


Kubernetes
* `KubernetesBatchSystem` can add `nvidia.com/gpu` and `amd.com/gpu` resource requests for jobs that request those accelerators (4163)
* `KubernetesBatchSystem` can request GPUs by `model` key, if nodes are labeled appropriately (4163)

Dependencies

Misc
* Toil WES server now accepts requests that leave out workflow_params. (4037)
* The `MessageBus` has been expanded to use `pypubsub`, and now has `MessageInbox` and `MessageOutbox` objects to represent connections to it. (4046/4047)
* `ToilMetrics` now rides on the `MessageBus` rails. (4046/4047)
* Toil workflows now have a `--writeMessages` option, which takes a file to which a line-oriented stream of `MessageBus` messages will be written. Reading this file will allow you to recover the current state of the workflow. (4046/4047)
* Add code for warning check to be used when launching cluster with AWS. (3514)
* Use a CI prebake image for gitlab testing. (4185)
* Toil clusters now have `/var/tmp` as the default temporary directory, since they often make large temporary files (4148)
* Adds basic testing for slurm using a slurm docker cluster by running sample workflows. (3856)
* Add message bus documentation (4239)
* `SingleMachineBatchSystem` can schedule nvidia GPU accelerators, limiting the concurrent jobs to no more than there are accelerators to support, and setting `CUDA_VISIBLE_DEVICES` in the tasks' environments to tell them which nvidia GPU(s) to use. (4163)
* `AWSBatchBatchSystem` can use AWS Batch's GPU resource to provide nvidia GPU accelerators (4163)
* Toil jobs no longer need to re-run after their child/followOn/service jobs in order to delete themselves. (3188)
* Message bus is now thread safe (4276)
* Docker build has been updated with new Aventer Mesos deb URL (fixes 4290)
* `docker` binary in the container has been updated to that included in the Ubuntu repos (fixes 4282)
* Singularity in the appliance has been updated to 3.10 which is >=3.9, for cgroups v2 support.
* Base Ubuntu container image for the appliance has been updated to 22.04, which has a new enough libc for Debian's Singularity 3.10 debs.
* Safer type usage checking for systems without boto3 installed
* Tests are now more runnable post-installation. Temporary paths are not selected based upon the location of the tests themselves. (4287)

_Bug Fixes_
* Only use `/var/run/user` if XDG tells us we have it in our session. Otherwise we will try other places, including `/run/lock/toil`. (4170)
* `toil destroy-cluster`: terminate stopped instances when destroying the cluster (4271)
* fileJobStore: handle arbitrary `os.link` errors to work on some filesystems (2232)

Thank you to our contributors!

5.7.1

Changelog


Highlighted Features Added

AWS Batch Batch System (3956)
AGC Integration (4039) + More AGC integration (4067) + AGC megabranch (4113)
Scale TES to be able to run reasonably-sized workflows on Funnel on Kubernetes with the AWS job store (3927)

CWL

Run CWL conformance tests via WES (4052)
Implement and test CWL loadContents from URLs to fix 4125 (4126)
Add CWL tests under ARM (4038)
Cache results of cwltool version lookup (4141)

Misc

SGE batch system change to support serial jobs. (4022)
Performance testing for Graviton instances (4123)
Stop waiting on hostpath volumes to exist (4146)
Catch and warn about jobs going away too slowly on FileJobStore (4149)
Add documentation for the type-checking hooks (4117)
Pod murder bot (4060)
Contrib hook scripts (4105)
Allow newer google-cloud-storage (4114)
Use environment variable to set parallel partition name (4096)
Register pytest markers (4103)
Mention --export=ALL for SLURM environments (4100) (4102)
Allow persisting workflow state in WES server across container recreation (4082)
Change `toil kill` to use the job store shared file API to find `pig.log` (4075)
Bring back kill loop in the single_machine batch system but with a timeout (4070)
Reorganize Locking (4059)
Add and test preemptability constraints (4044)
Enhanced types (3975)
Use an `init` process that reaps zombies on toil clusters (3974)
Add launch cluster support for ARM (3971)
Feat: square bracket to period separator (4008)
Add AGC health check endpoint (3997)
Tolerate and require typed Werkzeug (4011)
Add more static URLs for Singularity debs (4007)

_Bug Fixes_

Update WES set up docs (4027)
Add real time logs (4031)
Fail fast if Docker builder is missing (4001)
Make Toil version be reported as a string in WES (4013)
Fix assorted typos within assorted comments (4023)
Make file store case insensitive (4153)
Pre-lex commands for qsub (4150)
Update Cactus and exclude broken networkx (4107)
Make `toil kill` work when the leader is on another machine (4084)
Wrong filename in output (4139)
Tolerate a missing VersionID key to fix 4129 (4130)
Only import from typing_extensions on old Python where we install it (4090)
Allow missing username and fix Docker build (4077)
Leave more time for concurrency measurement to fix 4012 (4068)
Stop people asking for ARM Mesos clusters to fix 4057 (4058)

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, Hexotical, gmloose, kannon92, douglowe, gcapes, and pmiddend!

5.6.0

Changelog


Highlighted Features Added
* Integrate ARM Docker builds to make multi-arch images. (3802)
* WES support and server mode. (3779)
* TES batch system prototype. (3821)
* Support for new resource syntax in PBSPro. (3048)
* Toil now looks for lost jobs every minute instead of every hour (3948)

Breaking Changes
* Remove --disableCaching's true/false argument. (3869)

CWL
* CWL helper jobs: better disk & memory requirements. (3834)
* CWL: safer test path generation for post-install testing. (3818)
* More detailed names, which show up in job names sent to BatchSystem schedulers. 3941 (was 3893)
* If you use scatter and collect files,  duplicates are correctly dealt with and renamed. (3968)
* CWL: at the end of a job, ask cwltool to cleanup (3965)

Kubernetes
* Assign Kubernetes jobs an explicit TTL. (3936)
* Wait for node creation. (3934)

Dependencies
* Enable Dependabot updates. (3827)
* Multiple consolidated dependabot updates. (3851)
* Update addict requirement from <2.3,>=2.2.1 to >=2.2.1,<2.5. (3861)
* Bump cwltool to 3.1.20211107152837. (3833 3866 3909)
* Bump cwltest from 2.1.20210626101542 to 2.2.20210901154959. (3848)
* Bump flake8 from 3.8.4 to 4.0.1. (3847)
* Allow more docker-py versions. (3860)
* Remove pyyaml dependency. (3858)

Misc
* Update cleanup script. (3937)
* Remove use of sys.maxsize. (3824)
* Spelling fixes. (3814)
* Add codeql-analysis for Python. (3825)
* Update jobstore function names. (3809)
* Move AMI functions to lib. (3810)
* Add "make pyupgrade" (py36-plus). (3805)
* Type hints.  (3930)
* Coalesce status calls in slurm. (3822)
* Python logging takes format values as *args. (3852)
* Change quick test to a 10 minute timeout. (3843)
* Add make uninstall to makefile. (3883)
* Fix toil kill to find shared pid.log file (with unit test). 3941 (was 3932)
* Update documentation. (3947)
+ Remove remains of Travis. (3976)

_Bug Fixes_
* Stop checkpoints from being reissued multiple times. (3931)
* Don't consult LSF config when explicitly defining memory units. (3820)
* Robustly remove state dirs. (3836)
* Fix exception checking for exit_code. (3830)
* Use exitStatus instead of exitReason for batch exit type comparison. (3839)
* Update cwltest to improve K8 runs. (3935)
* Consolidated CI Fixes. (3887)
* Fix CWL conformance tests. (3891)
* Toil-managed cluster scaling should work again with `--metrics`. (3943)

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, Hexotical, tmooney, nikhil, kannon92, douglowe, mhpopescu, Phhere, and gmloose!

5.5.0

Changelog

CWL
* Add podman support; and other fixes from recent cwltool 3799
* Add streaming feature for cwltoil 3694
* Warn users if a different cwltool version is installed 3686
* Turn on all Kubernetes CWL tests that are expected to work on Singularity 3720
* Fix CWL in toil docs jobstore usage 3728
* DOC: update versions of CWL support
* Allow filestore bypass 3652

Misc
* Numerous Type Hints. 3705 3701 3693 3691 3663 3688 3642 3684 3682 3680 3666 3675
* Single source of truth for job state 3776
* Do not set default for statePollingWait 3774
* Use absolute local paths when exportFile/importFile do not detect a schema 3767
* Multi-zone balancing within regions for AWS autoscaling groups 3746
* Migrate cloud-config to ignition 3488
* 🎡 Wheel Of Issues 🎰 3760
* Add back addBatchSystemFactory function 3754
* Redirect stderr to /dev/null of lsf conf queries 3751
* Set number of cores based on job.cores for OpenMP applications 3739
* Google jobstore batching 3740
* Locations of CLI option docs
* Add AWS provisioner storage system 3727
* Set cls.bucket 3726
* Stream vs dowload jobs 3722
* Update Toil's main python test version to 3.8. 3669
* Move Travis tests to Gitlab 3675

_Bug Fixes_
* Don't leak symlinks 3795
* Prevent exception from being raised when modifying dir permissions for clean up 3778
* Fix scontrol output parsing 3793
* Fix AttributeError 3742
* Workaround for S3 in us-east-1 3710
* Time data format 3708
* Fix leader.py batch system std files prefix glob 3679

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, Hexotical, ionox0, gmloose, juanesarango, mhpopescu, mberacochea, nikhil!

5.4.0

Changelog

CWL
* Fix cwl and wdl dependency bleed and add stand alone tests. 3582
* Use MpiConfig.load() to handle MPI config file 3574
* Add support for MPI with CWL. 3525

Misc
* Numerous Type Hints. 3571 3634 3626 3625 3616 3614 3601 3592 3590 3581
* Configurable Grafana port 3597
* Handle streaming reads from the cache when the data isn't written back yet 3595
* Balance over pools using AWS ASGs 3490
* Change how Kubernetes schedules and scales for hopefully better scale-down behavior 3587
* Add Owner tags to our AWS buckets when testing. 3577
* Allow workdir override 3586
* Additional info for "Permission denied" error 3579
* Consolidate memory functions 3529
* Pen children 3482
* Sniff raid better 3526
* Add a propagation policy to batch delete 3522
* Cleanup script for buckets, sdb domains, instance profiles, and roles. 3373
* Add decorator for flaky tests. 3510

_Bug Fixes_
* Enforce valid uuid from getNodeId() 3611
* Removed extra memory multiplication 3608
* Fix non integers lsf memory requested 3609
* Fixes jobCommand unpack 3610
* Fix CWL test 20 on Kubernetes 3572
* Fix 3579 breaking AWS docs 3596
* Update cactus test to fix broken bucket links. 3594
* Allow cleaning up a job whose overlargeID fell off 3584
* Only deploy a user script if we can deploy user scripts 3518
* Reduce path name lengths. 3438
* Insist on credentials for testing AMI finding 3524

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, thiagogenez, julian-klode, darafferty, nikhil, mhpopescu!

5.3.0

Changelog

CWL
* Run CWL conformance tests on Kubernetes. 3323
* CWL symlinking files into work directory 3445
* set resource reqs for all CWL_INTERNAL_JOBS 3442

WDL
* Add WDL flatten() function + tests. 3485
* Add WDL collect_by_key() function + tests. 3476
* Add WDL keys() function + tests. 3460
* Add WDL as_pairs() function + tests. 3364
* Add as_map() WDL function 3448
* Add basic WDL development support 3434
* Add basic WDL 1.0 support 3421
* Update dictionary structure in AnalyzeWDL. 3416

Misc
* Add extra context for disk usage warning. 3495
* Update developing.rst 3479
* Allow text/encode options for read and write functions in file store 3428
* Type hinting and mypy checking additions 3470  3458 3456
* Quality of life improvements for cactus. 3463
* Add checklists for reviewing and merging PRs 3432
* Ensure that python3 and not python (2) is used 3446
* Support Kubernetes clusters in toil launch-cluster 3357
* Additional check for predecessors. 3417
* Allow setOptions to accept an argparse group. 3426

_Bug Fixes_
* AWS: Handle socket timeout during AWS discovery 3503
* Use back-up flatcar AMI if not found. 3513
* Lsf command parser 3475
* Retry on s3 throttling. 3504
* Stop deleting all ASGs with tags 3474
* Only determine execute permissions on files we copy/move. 3437
* cope with an invalid HTTP_PROXY 3447
* Fix failing Google jobstore test 3420
* Update Google job store 3412

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, Arthur Rand, mberacochea, Jessime, thiagogenez, julian-klode!

5.2.0

Changelog

CWL
* Update to the latest cwltool (3.0.20201121085451 -> 3.0.20201203173111).  3375
* Add better handling for potentially mis-ordered CWL args. 3395
* Confirm CommandInputParameter expression can receive a File object. 3350

WDL
* Make AnalyzeWDL abstract to allow implementation for different WDL versions 3391

Misc
* Clean up memoize.py.  3374
* Refactor bioio library, simplify logging, and small adjustments to utils.  3351
* Use regular division for memory calculation on LSF. 3387
* Remove unused --nodeOptions arg. 3397
* Get boto S3 args for minio from environment variables. (3370)
* Update EC2 lists. 3376
* Add a retry to AWS destroy. 3379
* Regular division for memory calculation in LSF. 3387
* Add option to blank mem allocation for SLURM. 3399
* Preserve file permissions on imported/exported files 3322

_Bug Fixes_
* Preserve file permissions on imported/exported files. 3322
* Make building a flattened list of live jobs non-recursive. 3394
* Move AWS job store uploads/downloads to boto3 (reopened) 3400
* Use a Docker Hub mirror when pulling Docker images. 3411
* Corrected gridengine maxMEM check to use memorystring 3410

Thank you to our contributors: mr-c, adamnovak, w-gao, jonathanxu18, douglowe, stevekm , thiagogenez!

5.1.0

Changelog

WDL
* Add WDL cross function.  3360

_Bug Fixes_
* Better error message for insufficient disk.  3362
* Fix maxMEM comparison in LSF.  3369

Thank you to our contributors: mr-c , adamnovak , w-gao , jonathanxu18 !
Links

Update pytz from 2020.1 to 2023.3.post1.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

Update typing from 3.7.4.3 to 3.10.0.0.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

Update ruamel.yaml from 0.16.5 to 0.18.5.

The bot wasn't able to find a changelog for this release. Got an idea?

Links

Update pip from 20.2.3 to 23.3.2.

Changelog

23.3.2

===================

Bug Fixes
---------

- Fix a bug in extras handling for link requirements (`12372 <https://github.com/pypa/pip/issues/12372>`_)
- Fix mercurial revision "parse error": use ``--rev={ref}`` instead of ``-r={ref}`` (`12373 <https://github.com/pypa/pip/issues/12373>`_)

23.3.1

===================

Bug Fixes
---------

- Handle a timezone indicator of Z when parsing dates in the self check. (`12338 <https://github.com/pypa/pip/issues/12338>`_)
- Fix bug where installing the same package at the same time with multiple pip processes could fail. (`12361 <https://github.com/pypa/pip/issues/12361>`_)

23.3

=================

Process
-------

- Added reference to `vulnerability reporting guidelines <https://www.python.org/dev/security/>`_ to pip's security policy.

Deprecations and Removals
-------------------------

- Drop a fallback to using SecureTransport on macOS. It was useful when pip detected OpenSSL older than 1.0.1, but the current pip does not support any Python version supporting such old OpenSSL versions. (`12175 <https://github.com/pypa/pip/issues/12175>`_)

Features
--------

- Improve extras resolution for multiple constraints on same base package. (`11924 <https://github.com/pypa/pip/issues/11924>`_)
- Improve use of datastructures to make candidate selection 1.6x faster. (`12204 <https://github.com/pypa/pip/issues/12204>`_)
- Allow ``pip install --dry-run`` to use platform and ABI overriding options. (`12215 <https://github.com/pypa/pip/issues/12215>`_)
- Add ``is_yanked`` boolean entry to the installation report (``--report``) to indicate whether the requirement was yanked from the index, but was still selected by pip conform to :pep:`592`. (`12224 <https://github.com/pypa/pip/issues/12224>`_)

Bug Fixes
---------

- Ignore errors in temporary directory cleanup (show a warning instead). (`11394 <https://github.com/pypa/pip/issues/11394>`_)
- Normalize extras according to :pep:`685` from package metadata in the resolver
for comparison. This ensures extras are correctly compared and merged as long
as the package providing the extra(s) is built with values normalized according
to the standard. Note, however, that this *does not* solve cases where the
package itself contains unnormalized extra values in the metadata. (`11649 <https://github.com/pypa/pip/issues/11649>`_)
- Prevent downloading sdists twice when :pep:`658` metadata is present. (`11847 <https://github.com/pypa/pip/issues/11847>`_)
- Include all requested extras in the install report (``--report``). (`11924 <https://github.com/pypa/pip/issues/11924>`_)
- Removed uses of ``datetime.datetime.utcnow`` from non-vendored code. (`12005 <https://github.com/pypa/pip/issues/12005>`_)
- Consistently report whether a dependency comes from an extra. (`12095 <https://github.com/pypa/pip/issues/12095>`_)
- Fix completion script for zsh (`12166 <https://github.com/pypa/pip/issues/12166>`_)
- Fix improper handling of the new onexc argument of ``shutil.rmtree()`` in Python 3.12. (`12187 <https://github.com/pypa/pip/issues/12187>`_)
- Filter out yanked links from the available versions error message: "(from versions: 1.0, 2.0, 3.0)" will not contain yanked versions conform PEP 592. The yanked versions (if any) will be mentioned in a separate error message. (`12225 <https://github.com/pypa/pip/issues/12225>`_)
- Fix crash when the git version number contains something else than digits and dots. (`12280 <https://github.com/pypa/pip/issues/12280>`_)
- Use ``-r=...`` instead of ``-r ...`` to specify references with Mercurial. (`12306 <https://github.com/pypa/pip/issues/12306>`_)
- Redact password from URLs in some additional places. (`12350 <https://github.com/pypa/pip/issues/12350>`_)
- pip uses less memory when caching large packages. As a result, there is a new on-disk cache format stored in a new directory ($PIP_CACHE_DIR/http-v2). (`2984 <https://github.com/pypa/pip/issues/2984>`_)

Vendored Libraries
------------------

- Upgrade certifi to 2023.7.22
- Add truststore 0.8.0
- Upgrade urllib3 to 1.26.17

Improved Documentation
----------------------

- Document that ``pip search`` support has been removed from PyPI (`12059 <https://github.com/pypa/pip/issues/12059>`_)
- Clarify --prefer-binary in CLI and docs (`12122 <https://github.com/pypa/pip/issues/12122>`_)
- Document that using OS-provided Python can cause pip's test suite to report false failures. (`12334 <https://github.com/pypa/pip/issues/12334>`_)

23.2.1

===================

Bug Fixes
---------

- Disable :pep:`658` metadata fetching with the legacy resolver. (`12156 <https://github.com/pypa/pip/issues/12156>`_)

23.2

=================

Process
-------

- Deprecate support for eggs for Python 3.11 or later, when the new ``importlib.metadata`` backend is used to load distribution metadata. This only affects the egg *distribution format* (with the ``.egg`` extension); distributions using the ``.egg-info`` *metadata format* (but are not actually eggs) are not affected. For more information about eggs, see `relevant section in the setuptools documentation <https://setuptools.pypa.io/en/stable/deprecated/python_eggs.html>`__.

Deprecations and Removals
-------------------------

- Deprecate legacy version and version specifiers that don't conform to `PEP 440
<https://peps.python.org/pep-0440/>`_ (`#12063 <https://github.com/pypa/pip/issues/12063>`_)
- ``freeze`` no longer excludes the ``setuptools``, ``distribute``, and ``wheel``
from the output when running on Python 3.12 or later, where they are not
included in a virtual environment by default. Use ``--exclude`` if you wish to
exclude any of these packages. (`4256 <https://github.com/pypa/pip/issues/4256>`_)

Features
--------

- make rejection messages slightly different between 1 and 8, so the user can make the difference. (`12040 <https://github.com/pypa/pip/issues/12040>`_)

Bug Fixes
---------

- Fix ``pip completion --zsh``. (`11417 <https://github.com/pypa/pip/issues/11417>`_)
- Prevent downloading files twice when :pep:`658` metadata is present (`11847 <https://github.com/pypa/pip/issues/11847>`_)
- Add permission check before configuration (`11920 <https://github.com/pypa/pip/issues/11920>`_)
- Fix deprecation warnings in Python 3.12 for usage of shutil.rmtree (`11957 <https://github.com/pypa/pip/issues/11957>`_)
- Ignore invalid or unreadable ``origin.json`` files in the cache of locally built wheels. (`11985 <https://github.com/pypa/pip/issues/11985>`_)
- Fix installation of packages with :pep:`658` metadata using non-canonicalized names (`12038 <https://github.com/pypa/pip/issues/12038>`_)
- Correctly parse ``dist-info-metadata`` values from JSON-format index data. (`12042 <https://github.com/pypa/pip/issues/12042>`_)
- Fail with an error if the ``--python`` option is specified after the subcommand name. (`12067 <https://github.com/pypa/pip/issues/12067>`_)
- Fix slowness when using ``importlib.metadata`` (the default way for pip to read metadata in Python 3.11+) and there is a large overlap between already installed and to-be-installed packages. (`12079 <https://github.com/pypa/pip/issues/12079>`_)
- Pass the ``-r`` flag to mercurial to be explicit that a revision is passed and protect
against ``hg`` options injection as part of VCS URLs. Users that do not have control on
VCS URLs passed to pip are advised to upgrade. (`12119 <https://github.com/pypa/pip/issues/12119>`_)

Vendored Libraries
------------------

- Upgrade certifi to 2023.5.7
- Upgrade platformdirs to 3.8.1
- Upgrade pygments to 2.15.1
- Upgrade pyparsing to 3.1.0
- Upgrade Requests to 2.31.0
- Upgrade rich to 13.4.2
- Upgrade setuptools to 68.0.0
- Updated typing_extensions to 4.6.0
- Upgrade typing_extensions to 4.7.1
- Upgrade urllib3 to 1.26.16

23.1.2

===================

Vendored Libraries
------------------

- Upgrade setuptools to 67.7.2

23.1.1

===================

Bug Fixes
---------

- Revert `11487 <https://github.com/pypa/pip/pull/11487>`_, as it causes issues with virtualenvs created by the Windows Store distribution of Python. (`#11987 <https://github.com/pypa/pip/issues/11987>`_)

Vendored Libraries
------------------

- Revert pkg_resources (via setuptools) back to 65.6.3

Improved Documentation
----------------------

- Update documentation to reflect the new behavior of using the cache of locally
built wheels in hash-checking mode. (`11967 <https://github.com/pypa/pip/issues/11967>`_)

23.1

=================

Deprecations and Removals
-------------------------

- Remove support for the deprecated ``--install-options``. (`11358 <https://github.com/pypa/pip/issues/11358>`_)
- ``--no-binary`` does not imply ``setup.py install`` anymore. Instead a wheel will be
built locally and installed. (`11451 <https://github.com/pypa/pip/issues/11451>`_)
- ``--no-binary`` does not disable the cache of locally built wheels anymore. It only
means "don't download wheels". (`11453 <https://github.com/pypa/pip/issues/11453>`_)
- Deprecate ``--build-option`` and ``--global-option``. Users are invited to switch to
``--config-settings``. (`11859 <https://github.com/pypa/pip/issues/11859>`_)
- Using ``--config-settings`` with projects that don't have a ``pyproject.toml`` now prints
a deprecation warning. In the future the presence of config settings will automatically
enable the default build backend for legacy projects and pass the settings to it. (`11915 <https://github.com/pypa/pip/issues/11915>`_)
- Remove ``setup.py install`` fallback when building a wheel failed for projects without
``pyproject.toml``. (`8368 <https://github.com/pypa/pip/issues/8368>`_)
- When the ``wheel`` package is not installed, pip now uses the default build backend
instead of ``setup.py install`` and ``setup.py develop`` for project without
``pyproject.toml``. (`8559 <https://github.com/pypa/pip/issues/8559>`_)

Features
--------

- Specify egg-link location in assertion message when it does not match installed location to provide better error message for debugging. (`10476 <https://github.com/pypa/pip/issues/10476>`_)
- Present conflict information during installation after each choice that is rejected (pass ``-vv`` to ``pip install`` to show it) (`10937 <https://github.com/pypa/pip/issues/10937>`_)
- Display dependency chain on each Collecting/Processing log line. (`11169 <https://github.com/pypa/pip/issues/11169>`_)
- Support a per-requirement ``--config-settings`` option in requirements files. (`11325 <https://github.com/pypa/pip/issues/11325>`_)
- The ``--config-settings``/``-C`` option now supports using the same key multiple
times. When the same key is specified multiple times, all values are passed to
the build backend as a list, as opposed to the previous behavior, where pip would
only pass the last value if the same key was used multiple times. (`11681 <https://github.com/pypa/pip/issues/11681>`_)
- Add ``-C`` as a short version of the ``--config-settings`` option. (`11786 <https://github.com/pypa/pip/issues/11786>`_)
- Reduce the number of resolver rounds, since backjumping makes the resolver more efficient in finding solutions. This also makes pathological cases fail quicker. (`11908 <https://github.com/pypa/pip/issues/11908>`_)
- Warn if ``--hash`` is used on a line without requirement in a requirements file. (`11935 <https://github.com/pypa/pip/issues/11935>`_)
- Stop propagating CLI ``--config-settings`` to the build dependencies. They already did
not propagate to requirements provided in requirement files. To pass the same config
settings to several requirements, users should provide the requirements as CLI
arguments. (`11941 <https://github.com/pypa/pip/issues/11941>`_)
- Support wheel cache when using ``--require-hashes``. (`5037 <https://github.com/pypa/pip/issues/5037>`_)
- Add ``--keyring-provider`` flag. See the Authentication page in the documentation for more info. (`8719 <https://github.com/pypa/pip/issues/8719>`_)
- In the case of virtual environments, configuration files are now also included from the base installation. (`9752 <https://github.com/pypa/pip/issues/9752>`_)

Bug Fixes
---------

- Fix grammar by changing "A new release of pip available:" to "A new release of pip is available:" in the notice used for indicating that. (`11529 <https://github.com/pypa/pip/issues/11529>`_)
- Normalize paths before checking if installed scripts are on PATH. (`11719 <https://github.com/pypa/pip/issues/11719>`_)
- Correct the way to decide if keyring is available. (`11774 <https://github.com/pypa/pip/issues/11774>`_)
- More consistent resolution backtracking by removing legacy hack related to setuptools resolution (`11837 <https://github.com/pypa/pip/issues/11837>`_)
- Include ``AUTHORS.txt`` in pip's wheels. (`11882 <https://github.com/pypa/pip/issues/11882>`_)
- The ``uninstall`` and ``install --force-reinstall`` commands no longer call
``normalize_path()`` repeatedly on the same paths. Instead, these results are
cached for the duration of an uninstall operation, resulting in improved
performance, particularly on Windows. (`11889 <https://github.com/pypa/pip/issues/11889>`_)
- Fix and improve the parsing of hashes embedded in URL fragments. (`11936 <https://github.com/pypa/pip/issues/11936>`_)
- When package A depends on package B provided as a direct URL dependency including a hash
embedded in the link, the ``--require-hashes`` option did not warn when user supplied hashes
were missing for package B. (`11938 <https://github.com/pypa/pip/issues/11938>`_)
- Correctly report ``requested_extras`` in the installation report when extras are
specified for a local directory installation. (`11946 <https://github.com/pypa/pip/issues/11946>`_)
- When installing an archive from a direct URL or local file, populate
``download_info.info.hashes`` in the installation report, in addition to the legacy
``download_info.info.hash`` key. (`11948 <https://github.com/pypa/pip/issues/11948>`_)

Vendored Libraries
------------------

- Upgrade msgpack to 1.0.5
- Patch pkg_resources to remove dependency on ``jaraco.text``.
- Upgrade platformdirs to 3.2.0
- Upgrade pygments to 2.14.0
- Upgrade resolvelib to 1.0.1
- Upgrade rich to 13.3.3
- Upgrade setuptools to 67.6.1
- Upgrade tenacity to 8.2.2
- Upgrade typing_extensions to 4.5.0
- Upgrade urllib3 to 1.26.15

Improved Documentation
----------------------

- Cross-reference the ``--python`` flag from the ``--prefix`` flag,
and mention limitations of ``--prefix`` regarding script installation. (`11775 <https://github.com/pypa/pip/issues/11775>`_)
- Add SECURITY.md to make the policy offical. (`11809 <https://github.com/pypa/pip/issues/11809>`_)
- Add username to Git over SSH example. (`11838 <https://github.com/pypa/pip/issues/11838>`_)
- Quote extras in the pip install docs to guard shells with default glob
qualifiers, like zsh. (`11842 <https://github.com/pypa/pip/issues/11842>`_)
- Make it clear that requirements/constraints file can be a URL (`11954 <https://github.com/pypa/pip/issues/11954>`_)

23.0.1

===================

Features
--------

- Ignore PIP_REQUIRE_VIRTUALENV for ``pip index`` (`11671 <https://github.com/pypa/pip/issues/11671>`_)
- Implement ``--break-system-packages`` to permit installing packages into
``EXTERNALLY-MANAGED`` Python installations. (`11780 <https://github.com/pypa/pip/issues/11780>`_)

Bug Fixes
---------

- Improve handling of isolated build environments on platforms that
customize the Python's installation schemes, such as Debian and
Homebrew. (`11740 <https://github.com/pypa/pip/issues/11740>`_)
- Do not crash in presence of misformatted hash field in ``direct_url.json``. (`11773 <https://github.com/pypa/pip/issues/11773>`_)

23.0

=================

Features
--------

- Change the hashes in the installation report to be a mapping. Emit the
``archive_info.hashes`` dictionary in ``direct_url.json``. (`11312 <https://github.com/pypa/pip/issues/11312>`_)
- Implement logic to read the ``EXTERNALLY-MANAGED`` file as specified in :pep:`668`.
This allows a downstream Python distributor to prevent users from using pip to
modify the externally managed environment. (`11381 <https://github.com/pypa/pip/issues/11381>`_)
- Enable the use of ``keyring`` found on ``PATH``. This allows ``keyring``
installed using ``pipx`` to be used by ``pip``. (`11589 <https://github.com/pypa/pip/issues/11589>`_)
- The inspect and installation report formats are now declared stable, and their version
has been bumped from ``0`` to ``1``. (`11757 <https://github.com/pypa/pip/issues/11757>`_)

Bug Fixes
---------

- Wheel cache behavior is restored to match previous versions, allowing the
cache to find existing entries. (`11527 <https://github.com/pypa/pip/issues/11527>`_)
- Use the "venv" scheme if available to obtain prefixed lib paths. (`11598 <https://github.com/pypa/pip/issues/11598>`_)
- Deprecated a historical ambiguity in how ``egg`` fragments in URL-style
requirements are formatted and handled. ``egg`` fragments that do not look
like :pep:`508` names now produce a deprecation warning. (`11617 <https://github.com/pypa/pip/issues/11617>`_)
- Fix scripts path in isolated build environment on Debian. (`11623 <https://github.com/pypa/pip/issues/11623>`_)
- Make ``pip show`` show the editable location if package is editable (`11638 <https://github.com/pypa/pip/issues/11638>`_)
- Stop checking that ``wheel`` is present when ``build-system.requires``
is provided without ``build-system.build-backend`` as ``setuptools``
(which we still check for) will inject it anyway. (`11673 <https://github.com/pypa/pip/issues/11673>`_)
- Fix an issue when an already existing in-memory distribution would cause
exceptions in ``pip install`` (`11704 <https://github.com/pypa/pip/issues/11704>`_)

Vendored Libraries
------------------

- Upgrade certifi to 2022.12.7
- Upgrade chardet to 5.1.0
- Upgrade colorama to 0.4.6
- Upgrade distro to 1.8.0
- Remove pep517 from vendored packages
- Upgrade platformdirs to 2.6.2
- Add pyproject-hooks 1.0.0
- Upgrade requests to 2.28.2
- Upgrade rich to 12.6.0
- Upgrade urllib3 to 1.26.14

Improved Documentation
----------------------

- Fixed the description of the option "--install-options" in the documentation (`10265 <https://github.com/pypa/pip/issues/10265>`_)
- Remove mention that editable installs are necessary for pip freeze to report the VCS
URL. (`11675 <https://github.com/pypa/pip/issues/11675>`_)
- Clarify that the egg URL fragment is only necessary for editable VCS installs, and
otherwise not necessary anymore. (`11676 <https://github.com/pypa/pip/issues/11676>`_)

22.3.1

===================

Bug Fixes
---------

- Fix entry point generation of ``pip.X``, ``pipX.Y``, and ``easy_install-X.Y``
to correctly account for multi-digit Python version segments (e.g. the "11"
part of 3.11). (`11547 <https://github.com/pypa/pip/issues/11547>`_)

22.3

=================

Deprecations and Removals
-------------------------

- Deprecate ``--install-options`` which forces pip to use the deprecated ``install``
command of ``setuptools``. (`11358 <https://github.com/pypa/pip/issues/11358>`_)
- Deprecate installation with 'setup.py install' when no-binary is enabled for
source distributions without 'pyproject.toml'. (`11452 <https://github.com/pypa/pip/issues/11452>`_)
- Deprecate --no-binary`` disabling the wheel cache. (`11454 <https://github.com/pypa/pip/issues/11454>`_)
- Remove ``--use-feature=2020-resolver`` opt-in flag. This was supposed to be removed in 21.0, but missed during that release cycle. (`11493 <https://github.com/pypa/pip/issues/11493>`_)
- Deprecate installation with 'setup.py install' when the 'wheel' package is absent for
source distributions without 'pyproject.toml'. (`8559 <https://github.com/pypa/pip/issues/8559>`_)
- Remove the ability to use ``pip list --outdated`` in combination with ``--format=freeze``. (`9789 <https://github.com/pypa/pip/issues/9789>`_)

Features
--------

- Use ``shell=True`` for opening the editor with ``pip config edit``. (`10716 <https://github.com/pypa/pip/issues/10716>`_)
- Use the ``data-dist-info-metadata`` attribute from :pep:`658` to resolve distribution metadata without downloading the dist yet. (`11111 <https://github.com/pypa/pip/issues/11111>`_)
- Add an option to run the test suite with pip built as a zipapp. (`11250 <https://github.com/pypa/pip/issues/11250>`_)
- Add a ``--python`` option to allow pip to manage Python environments other
than the one pip is installed in. (`11320 <https://github.com/pypa/pip/issues/11320>`_)
- Document the new (experimental) zipapp distribution of pip. (`11459 <https://github.com/pypa/pip/issues/11459>`_)
- Use the much faster 'bzr co --lightweight' to obtain a copy of a Bazaar tree. (`5444 <https://github.com/pypa/pip/issues/5444>`_)

Bug Fixes
---------

- Fix ``--no-index`` when ``--index-url`` or ``--extra-index-url`` is specified
inside a requirements file. (`11276 <https://github.com/pypa/pip/issues/11276>`_)
- Ensure that the candidate ``pip`` executable exists, when checking for a new version of pip. (`11309 <https://github.com/pypa/pip/issues/11309>`_)
- Ignore distributions with invalid ``Name`` in metadata instead of crashing, when
using the ``importlib.metadata`` backend. (`11352 <https://github.com/pypa/pip/issues/11352>`_)
- Raise RequirementsFileParseError when parsing malformed requirements options that can't be successfully parsed by shlex. (`11491 <https://github.com/pypa/pip/issues/11491>`_)
- Fix build environment isolation on some system Pythons. (`6264 <https://github.com/pypa/pip/issues/6264>`_)

Vendored Libraries
------------------

- Upgrade certifi to 2022.9.24
- Upgrade distlib to 0.3.6
- Upgrade idna to 3.4
- Upgrade pep517 to 0.13.0
- Upgrade pygments to 2.13.0
- Upgrade tenacity to 8.1.0
- Upgrade typing_extensions to 4.4.0
- Upgrade urllib3 to 1.26.12

Improved Documentation
----------------------

- Mention that --quiet must be used when writing the installation report to stdout. (`11357 <https://github.com/pypa/pip/issues/11357>`_)

22.2.2

===================

Bug Fixes
---------

- Avoid  ``AttributeError`` when removing the setuptools-provided ``_distutils_hack`` and it is missing its implementation. (`11314 <https://github.com/pypa/pip/issues/11314>`_)
- Fix import error when reinstalling pip in user site. (`11319 <https://github.com/pypa/pip/issues/11319>`_)
- Show pip deprecation warnings by default. (`11330 <https://github.com/pypa/pip/issues/11330>`_)

22.2.1

===================

Bug Fixes
---------

- Send the pip upgrade prompt to stderr. (`11282 <https://github.com/pypa/pip/issues/11282>`_)
- Ensure that things work correctly in environments where setuptools-injected
``distutils`` is available by default. This is done by cooperating with
setuptools' injection logic to ensure that pip uses the ``distutils`` from the
Python standard library instead. (`11298 <https://github.com/pypa/pip/issues/11298>`_)
- Clarify that ``pip cache``'s wheels-related output is about locally built wheels only. (`11300 <https://github.com/pypa/pip/issues/11300>`_)

22.2

=================

Deprecations and Removals
-------------------------

- Remove the ``html5lib`` deprecated feature flag. (`10825 <https://github.com/pypa/pip/issues/10825>`_)
- Remove ``--use-deprecated=backtrack-on-build-failures``. (`11241 <https://github.com/pypa/pip/issues/11241>`_)

Features
--------

- Add support to use `truststore <https://pypi.org/project/truststore/>`_ as an
alternative SSL certificate verification backend. The backend can be enabled on Python
3.10 and later by installing ``truststore`` into the environment, and adding the
``--use-feature=truststore`` flag to various pip commands.

``truststore`` differs from the current default verification backend (provided by
``certifi``) in it uses the operating system’s trust store, which can be better
controlled and augmented to better support non-standard certificates. Depending on
feedback, pip may switch to this as the default certificate verification backend in
the future. (`11082 <https://github.com/pypa/pip/issues/11082>`_)
- Add ``--dry-run`` option to ``pip install``, to let it print what it would install but
not actually change anything in the target environment. (`11096 <https://github.com/pypa/pip/issues/11096>`_)
- Record in wheel cache entries the URL of the original artifact that was downloaded
to build the cached wheels. The record is named ``origin.json`` and uses the PEP 610
Direct URL format. (`11137 <https://github.com/pypa/pip/issues/11137>`_)
- Support `PEP 691 <https://peps.python.org/pep-0691/>`_. (`#11158 <https://github.com/pypa/pip/issues/11158>`_)
- pip's deprecation warnings now subclass the built-in ``DeprecationWarning``, and
can be suppressed by running the Python interpreter with
``-W ignore::DeprecationWarning``. (`11225 <https://github.com/pypa/pip/issues/11225>`_)
- Add ``pip inspect`` command to obtain the list of installed distributions and other
information about the Python environment, in JSON format. (`11245 <https://github.com/pypa/pip/issues/11245>`_)
- Significantly speed up isolated environment creation, by using the same
sources for pip instead of creating a standalone installation for each
environment. (`11257 <https://github.com/pypa/pip/issues/11257>`_)
- Add an experimental ``--report`` option to the install command to generate a JSON report
of what was installed. In combination with ``--dry-run`` and ``--ignore-installed`` it
can be used to resolve the requirements. (`53 <https://github.com/pypa/pip/issues/53>`_)

Bug Fixes
---------

- Fix ``pip install --pre`` for packages with pre-release build dependencies defined
both in ``pyproject.toml``'s ``build-system.requires`` and ``setup.py``'s
``setup_requires``. (`10222 <https://github.com/pypa/pip/issues/10222>`_)
- When pip rewrites the shebang line in a script during wheel installation,
update the hash and size in the corresponding ``RECORD`` file entry. (`10744 <https://github.com/pypa/pip/issues/10744>`_)
- Do not consider a ``.dist-info`` directory found inside a wheel-like zip file
as metadata for an installed distribution. A package in a wheel is (by
definition) not installed, and is not guaranteed to work due to how a wheel is
structured. (`11217 <https://github.com/pypa/pip/issues/11217>`_)
- Use ``importlib.resources`` to read the ``vendor.txt`` file in ``pip debug``.
This makes the command safe for use from a zipapp. (`11248 <https://github.com/pypa/pip/issues/11248>`_)
- Make the ``--use-pep517`` option of the ``download`` command apply not just
to the requirements specified on the command line, but to their dependencies,
as well. (`9523 <https://github.com/pypa/pip/issues/9523>`_)

Process
-------

- Remove reliance on the stdlib cgi module, which is deprecated in Python 3.11.

Vendored Libraries
------------------

- Remove html5lib.
- Upgrade certifi to 2022.6.15
- Upgrade chardet to 5.0.0
- Upgrade colorama to 0.4.5
- Upgrade distlib to 0.3.5
- Upgrade msgpack to 1.0.4
- Upgrade pygments to 2.12.0
- Upgrade pyparsing to 3.0.9
- Upgrade requests to 2.28.1
- Upgrade rich to 12.5.1
- Upgrade typing_extensions to 4.3.0
- Upgrade urllib3 to 1.26.10

22.1.2

===================

Bug Fixes
---------

- Revert `10979 <https://github.com/pypa/pip/issues/10979>`_ since it introduced a regression in certain edge cases. (`#10979 <https://github.com/pypa/pip/issues/10979>`_)
- Fix an incorrect assertion in the logging logic, that prevented the upgrade prompt from being presented. (`11136 <https://github.com/pypa/pip/issues/11136>`_)

22.1.1

===================

Bug Fixes
---------

- Properly filter out optional dependencies (i.e. extras) when checking build environment distributions. (`11112 <https://github.com/pypa/pip/issues/11112>`_)
- Change the build environment dependency checking to be opt-in. (`11116 <https://github.com/pypa/pip/issues/11116>`_)
- Allow using a pre-release version to satisfy a build requirement. This helps
manually populated build environments to more accurately detect build-time
requirement conflicts. (`11123 <https://github.com/pypa/pip/issues/11123>`_)

22.1

=================

Process
-------

- Enable the ``importlib.metadata`` metadata implementation by default on
Python 3.11 (or later). The environment variable ``_PIP_USE_IMPORTLIB_METADATA``
can still be used to enable the implementation on 3.10 and earlier, or disable
it on 3.11 (by setting it to ``0`` or ``false``).

Bug Fixes
---------

- Revert `9243 <https://github.com/pypa/pip/issues/9243>`_ since it introduced a regression in certain edge cases. (`#10962 <https://github.com/pypa/pip/issues/10962>`_)
- Fix missing ``REQUESTED`` metadata when using URL constraints. (`11079 <https://github.com/pypa/pip/issues/11079>`_)
- ``pip config`` now normalizes names by converting underscores into dashes. (`9330 <https://github.com/pypa/pip/issues/9330>`_)

22.1b1

===================

Process
-------

- Start migration of distribution metadata implementation from ``pkg_resources``
to ``importlib.metadata``. The new implementation is currently not exposed in
any user-facing way, but included in the code base for easier development.

Deprecations and Removals
-------------------------

- Drop ``--use-deprecated=out-of-tree-build``, according to deprecation message. (`11001 <https://github.com/pypa/pip/issues/11001>`_)

Features
--------

- Add option to install and uninstall commands to opt-out from running-as-root warning. (`10556 <https://github.com/pypa/pip/issues/10556>`_)
- Include Project-URLs in ``pip show`` output. (`10799 <https://github.com/pypa/pip/issues/10799>`_)
- Improve error message when ``pip config edit`` is provided an editor that
doesn't exist. (`10812 <https://github.com/pypa/pip/issues/10812>`_)
- Add a user interface for supplying config settings to build backends. (`11059 <https://github.com/pypa/pip/issues/11059>`_)
- Add support for Powershell autocompletion. (`9024 <https://github.com/pypa/pip/issues/9024>`_)
- Explains why specified version cannot be retrieved when *Requires-Python* is not satisfied. (`9615 <https://github.com/pypa/pip/issues/9615>`_)
- Validate build dependencies when using ``--no-build-isolation``. (`9794 <https://github.com/pypa/pip/issues/9794>`_)

Bug Fixes
---------

- Fix conditional checks to prevent ``pip.exe`` from trying to modify itself, on Windows. (`10560 <https://github.com/pypa/pip/issues/10560>`_)
- Fix uninstall editable from Windows junction link. (`10696 <https://github.com/pypa/pip/issues/10696>`_)
- Fallback to pyproject.toml-based builds if ``setup.py`` is present in a project, but ``setuptools`` cannot be imported. (`10717 <https://github.com/pypa/pip/issues/10717>`_)
- When checking for conflicts in the build environment, correctly skip requirements
containing markers that do not match the current environment. (`10883 <https://github.com/pypa/pip/issues/10883>`_)
- Disable brotli import in vendored urllib3 so brotli could be uninstalled/upgraded by pip. (`10950 <https://github.com/pypa/pip/issues/10950>`_)
- Prioritize URL credentials over netrc. (`10979 <https://github.com/pypa/pip/issues/10979>`_)
- Filter available distributions using hash declarations from constraints files. (`9243 <https://github.com/pypa/pip/issues/9243>`_)
- Fix an error when trying to uninstall packages installed as editable from a network drive. (`9452 <https://github.com/pypa/pip/issues/9452>`_)
- Fix pip install issues using a proxy due to an inconsistency in how Requests is currently handling variable precedence in session. (`9691 <https://github.com/pypa/pip/issues/9691>`_)

Vendored Libraries
------------------

- Upgrade CacheControl to 0.12.11
- Upgrade distro to 1.7.0
- Upgrade platformdirs to 2.5.2
- Remove ``progress`` from vendored dependencies.
- Upgrade ``pyparsing`` to 3.0.8 for startup performance improvements.
- Upgrade rich to 12.2.0
- Upgrade tomli to 2.0.1
- Upgrade typing_extensions to 4.2.0

Improved Documentation
----------------------

- Add more dedicated topic and reference pages to the documentation. (`10899 <https://github.com/pypa/pip/issues/10899>`_)
- Capitalise Y as the default for "Proceed (y/n)?" when uninstalling. (`10936 <https://github.com/pypa/pip/issues/10936>`_)
- Add ``scheme://`` requirement to ``--proxy`` option's description (`10951 <https://github.com/pypa/pip/issues/10951>`_)
- The wheel command now references the build interface section instead of stating the legacy
setuptools behavior as the default. (`10972 <https://github.com/pypa/pip/issues/10972>`_)
- Improved usefulness of ``pip config --help`` output. (`11074 <https://github.com/pypa/pip/issues/11074>`_)

22.0.4

===================

Deprecations and Removals
-------------------------

- Drop the doctype check, that presented a warning for index pages that use non-compliant HTML 5. (`10903 <https://github.com/pypa/pip/issues/10903>`_)

Vendored Libraries
------------------

- Downgrade distlib to 0.3.3.

22.0.3

===================

Features
--------

- Print the exception via ``rich.traceback``, when running with ``--debug``. (`10791 <https://github.com/pypa/pip/issues/10791>`_)

Bug Fixes
---------

- Only calculate topological installation order, for packages that are going to be installed/upgraded.

This fixes an `AssertionError` that occurred when determining installation order, for a very specific combination of upgrading-already-installed-package + change of dependencies + fetching some packages from a package index. This combination was especially common in Read the Docs' builds. (`10851 <https://github.com/pypa/pip/issues/10851>`_)
- Use ``html.parser`` by default, instead of falling back to ``html5lib`` when ``--use-deprecated=html5lib`` is not passed. (`10869 <https://github.com/pypa/pip/issues/10869>`_)

Improved Documentation
----------------------

- Clarify that using per-requirement overrides disables the usage of wheels. (`9674 <https://github.com/pypa/pip/issues/9674>`_)

22.0.2

===================

Deprecations and Removals
-------------------------

- Instead of failing on index pages that use non-compliant HTML 5, print a deprecation warning and fall back to ``html5lib``-based parsing for now. This simplifies the migration for non-compliant index pages, by letting such indexes function with a warning. (`10847 <https://github.com/pypa/pip/issues/10847>`_)

22.0.1

===================

Bug Fixes
---------

- Accept lowercase ``<!doctype html>`` on index pages. (`10844 <https://github.com/pypa/pip/issues/10844>`_)
- Properly handle links parsed by html5lib, when using ``--use-deprecated=html5lib``. (`10846 <https://github.com/pypa/pip/issues/10846>`_)

22.0

=================

Process
-------

- Completely replace :pypi:`tox` in our development workflow, with :pypi:`nox`.

Deprecations and Removals
-------------------------

- Deprecate alternative progress bar styles, leaving only ``on`` and ``off`` as available choices. (`10462 <https://github.com/pypa/pip/issues/10462>`_)
- Drop support for Python 3.6. (`10641 <https://github.com/pypa/pip/issues/10641>`_)
- Disable location mismatch warnings on Python versions prior to 3.10.

These warnings were helping identify potential issues as part of the sysconfig -> distutils transition, and we no longer need to rely on reports from older Python versions for information on the transition. (`10840 <https://github.com/pypa/pip/issues/10840>`_)

Features
--------

- Changed ``PackageFinder`` to parse HTML documents using the stdlib :class:`html.parser.HTMLParser` class instead of the ``html5lib`` package.

For now, the deprecated ``html5lib`` code remains and can be used with the ``--use-deprecated=html5lib`` command line option. However, it will be removed in a future pip release. (`10291 <https://github.com/pypa/pip/issues/10291>`_)
- Utilise ``rich`` for presenting pip's default download progress bar. (`10462 <https://github.com/pypa/pip/issues/10462>`_)
- Present a better error message when an invalid wheel file is encountered, providing more context where the invalid wheel file is. (`10535 <https://github.com/pypa/pip/issues/10535>`_)
- Documents the ``--require-virtualenv`` flag for ``pip install``. (`10588 <https://github.com/pypa/pip/issues/10588>`_)
- ``pip install <tab>`` autocompletes paths. (`10646 <https://github.com/pypa/pip/issues/10646>`_)
- Allow Python distributors to opt-out from or opt-in to the ``sysconfig`` installation scheme backend by setting ``sysconfig._PIP_USE_SYSCONFIG`` to ``True`` or ``False``. (`10647 <https://github.com/pypa/pip/issues/10647>`_)
- Make it possible to deselect tests requiring cryptography package on systems where it cannot be installed. (`10686 <https://github.com/pypa/pip/issues/10686>`_)
- Start using Rich for presenting error messages in a consistent format. (`10703 <https://github.com/pypa/pip/issues/10703>`_)
- Improve presentation of errors from subprocesses. (`10705 <https://github.com/pypa/pip/issues/10705>`_)
- Forward pip's verbosity configuration to VCS tools to control their output accordingly. (`8819 <https://github.com/pypa/pip/issues/8819>`_)

Bug Fixes
---------

- Optimize installation order calculation to improve performance when installing requirements that form a complex dependency graph with a large amount of edges. (`10557 <https://github.com/pypa/pip/issues/10557>`_)
- When a package is requested by the user for upgrade, correctly identify that the extra-ed variant of that same package depended by another user-requested package is requesting the same package, and upgrade it accordingly. (`10613 <https://github.com/pypa/pip/issues/10613>`_)
- Prevent pip from installing yanked releases unless explicitly pinned via the ``==`` or ``===`` operators. (`10617 <https://github.com/pypa/pip/issues/10617>`_)
- Stop backtracking on build failures, by instead surfacing them to the user and aborting immediately. This behaviour provides more immediate feedback when a package cannot be built due to missing build dependencies or platform incompatibility. (`10655 <https://github.com/pypa/pip/issues/10655>`_)
- Silence ``Value for <location> does not match`` warning caused by an erroneous patch in Slackware-distributed Python 3.9. (`10668 <https://github.com/pypa/pip/issues/10668>`_)
- Fix an issue where pip did not consider dependencies with and without extras to be equal (`9644 <https://github.com/pypa/pip/issues/9644>`_)

Vendored Libraries
------------------

- Upgrade CacheControl to 0.12.10
- Upgrade certifi to 2021.10.8
- Upgrade distlib to 0.3.4
- Upgrade idna to 3.3
- Upgrade msgpack to 1.0.3
- Upgrade packaging to 21.3
- Upgrade platformdirs to 2.4.1
- Add pygments 2.11.2 as a vendored dependency.
- Tree-trim unused portions of vendored pygments, to reduce the distribution size.
- Upgrade pyparsing to 3.0.7
- Upgrade Requests to 2.27.1
- Upgrade resolvelib to 0.8.1
- Add rich 11.0.0 as a vendored dependency.
- Tree-trim unused portions of vendored rich, to reduce the distribution size.
- Add typing_extensions 4.0.1 as a vendored dependency.
- Upgrade urllib3 to 1.26.8

21.3.1

===================


Bug Fixes
---------


- Always refuse installing or building projects that have no ``pyproject.toml`` nor
``setup.py``. (`10531 <https://github.com/pypa/pip/issues/10531>`_)
- Tweak running-as-root detection, to check ``os.getuid`` if it exists, on Unix-y and non-Linux/non-MacOS machines. (`10565 <https://github.com/pypa/pip/issues/10565>`_)
- When installing projects with a ``pyproject.toml`` in editable mode, and the build
backend does not support :pep:`660`, prepare metadata using
``prepare_metadata_for_build_wheel`` instead of ``setup.py egg_info``. Also, refuse
installing projects that only have a ``setup.cfg`` and no ``setup.py`` nor

@pyup-bot pyup-bot mentioned this pull request Jan 1, 2024
Closed
@pyup-bot
Copy link
Collaborator Author

pyup-bot commented Feb 1, 2024

Closing this in favor of #206

@pyup-bot pyup-bot closed this Feb 1, 2024
@rhshah rhshah deleted the pyup-scheduled-update-2024-01-01 branch February 1, 2024 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@pyup-bot