REL: pcds-5.8.1 #310

ZLLentz · 2024-01-04T20:53:35Z

Current contents of the security updates file:

aiohttp>=3.8.6
cryptography>=41.0.6
jupyter_server>=2.11.2
paramiko>=3.4.0
pip>=23.3
pyarrow>=14.0.1
pycryptodome>=3.19.1
werkzeug>=3.0.1

PCDS Package Updates

Package	Old	New	Release Notes
atef	1.2.0	1.3.0	https://github.com/pcdshub/atef/releases/tag/v1.3.0
blark	0.7.1	0.8.1	https://github.com/klauer/blark/releases/tag/v0.8.0 https://github.com/klauer/blark/releases/tag/v0.8.1
happi	2.4.0	2.5.0	https://github.com/pcdshub/happi/releases/tag/v2.5.0
lucid	0.10.2	0.10.3	https://github.com/pcdshub/lucid/releases/tag/v0.10.3
pcdsdevices	8.1.0	8.2.0	https://github.com/pcdshub/pcdsdevices/releases/tag/v8.2.0
pcdsutils	0.14.0	0.14.1	https://github.com/pcdshub/pcdsutils/releases/tag/v0.14.1
typhos	3.0.0	3.1.0	https://github.com/pcdshub/typhos/releases/tag/v3.1.0
whatrecord	0.5.0	0.6.0	https://github.com/pcdshub/whatrecord/releases/tag/v0.6.0
tc-release	0.2.4	0.2.5	https://github.com/pcdshub/tc-release/releases/tag/v0.2.5

SLAC Package Updates

Package	Old	New	Release Notes
pydm	1.20.1	1.21.0	https://github.com/slaclab/pydm/releases/tag/v1.21.0

Lab Community Package Updates

Package	Old	New
caproto	1.1.0	1.1.1
hklpy	1.0.3	1.0.4
pyepics	3.5.0	3.5.2
ophyd-async	0.1.0	0.2.0

Python Community Core Package Updates

Package	Old	New
flake8	6.1.0	7.0.0
pre-commit	3.4.0	3.6.0

Other Python Community Major Updates

Package	Old	New
libarrow	13.0.0	14.0.2
libjpeg-turbo	2.1.5.1	3.0.0
libpq	15.4	16.1
pyarrow	13.0.0	14.0.2
pygithub	1.59.1	2.1.1

Added the Following Packages

annotated-types
pydantic-core

Added the Following Dependencies

cirun (required by conda-smithy)
editables (required by hatchling, which is used in ipython, jupyter)
hatch (required by cirun, notebook, which are used in ipython, jupyter)
hatchling (required by cirun, hatch, which are used in ipython, jupyter)
hyperlink (required by hatch, which is used in ipython, jupyter)
jaraco.classes (required by keyring, which is used in ipython, jupyter)
jeepney (required by keyring, secretstorage, which are used in ipython, jupyter)
keyring (required by hatch, which is used in ipython, jupyter)
libarrow-acero (required by libarrow-dataset, libarrow-substrait, pyarrow, which are used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libarrow-dataset (required by libarrow-substrait, pyarrow, which are used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libarrow-flight (required by libarrow-flight-sql, pyarrow, which are used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libarrow-flight-sql (required by pyarrow, which is used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libarrow-gandiva (required by pyarrow, which is used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libarrow-substrait (required by pyarrow, which is used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libparquet (required by libarrow-dataset, pyarrow, which are used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
libre2-11 (required by libarrow, libarrow-gandiva, libgrpc, re2, which are used in bluesky, bluesky-live, databroker, ophyd, ophyd-async, pandas, pcdsdevices, scikit-image, suitcase-tiff, tiled, xarray)
secretstorage (required by keyring, which is used in ipython, jupyter)
spec2nexus (required by hklpy)
tomli-w (required by hatch, which is used in ipython, jupyter)
trove-classifiers (required by hatchling, which is used in ipython, jupyter)
userpath (required by hatch, which is used in ipython, jupyter)

ZLLentz · 2024-01-05T22:22:51Z

another CVE back from December popped up when running the tests locally and the local build was going slow so I'll do another round of ci builds instead
local tests passed

ZLLentz · 2024-01-08T17:19:34Z

After the last update everything is still passing offline and online
I'm going to mark as ready for review, request a review, and also scrutinize the changes myself

ZLLentz · 2024-01-08T17:35:11Z

annotated-types and pydantic-core >=2 snuck in again (gets installed in the conda step, orphaned by downgrading pydantic to <2 in the pip step)

not sure if it's better to just ignore this or maybe manually pin pydantic<2 in the environment spec (disadvantage: may need to manually unpin later, more work for only superficial gain)

tangkong · 2024-01-08T17:56:35Z

annotated-types and pydantic-core >=2 snuck in again (gets installed in the conda step, orphaned by downgrading pydantic to <2 in the pip step)

I was just looking around for this. Is annotated-types a dependency of pydantic? (yes it is)

not sure if it's better to just ignore this or maybe manually pin pydantic<2 in the environment spec (disadvantage: may need to manually unpin later, more work for only superficial gain)

For my reference again, pydantic is installed with bluesky-queueserver then downgraded when tiled is installed in the pip step? My first thought is that if it's not causing any build problems, we can just leave it. We'll probably need it when tiled eventually starts using pydantic 2.0, so it's not a problem if somehow someone finds it and starts using it (a highly unlikely scenario)

tangkong

A pretty straightforward env update I think. No "new packages" so to speak, just new dependencies that are showing up.

A slightly confusing bit in the autogenerated PR text, but it's somewhat inconsequential:

cirun (required by conda-smithy)
...

hatch (required by cirun, notebook, which are used in ipython, jupyter)

The hatch description makes it sound like both cirun and notebook are used in ipython/jupyter, but I think it's actually only notebook? cirun and notebook both use hatch, just separately

tangkong · 2024-01-08T17:39:25Z

envs/pcds/conda-security.txt

@@ -4,6 +4,8 @@
 aiohttp>=3.8.6
 cryptography>=41.0.6
 jupyter_server>=2.11.2
+paramiko>=3.4.0


For my own reference: these were already in our environment, just listed in different places. Good to be a bit clearer about what why the packages exist 👍

tangkong · 2024-01-08T18:02:47Z

envs/pcds/env.yaml

@@ -833,4 +856,4 @@ dependencies:
      - watchgod==0.8.2
      - websockets==11.0.3
      - xraylib==4.1.3
-prefix: /cds/home/z/zlentz/miniconda3/envs/pcds-5.8.0
+prefix: /home/runner/miniconda/envs/pcds-5.8.1


is this path on our filesystems? I know it's not that important but I'm just unfamiliar with it

This path gets generated when you export the env.yaml file, and since this one was exported by the ci job automatically instead of by me manually it shows the github actions filepath. I hand-edited the env name to be "pcds-5.8.1" (it generated as pcds-dev or something like that) but in practice none of this matters as this isn't used when you consume the file to make a new env.

ZLLentz · 2024-01-08T18:56:06Z

The hatch description makes it sound like both cirun and notebook are used in ipython/jupyter, but I think it's actually only notebook? cirun and notebook both use hatch, just separately

Right, the description here is a bit terse and lacking in detail in cases where multiple packages have the new dependency on their dependency list. It also picks up pypi "optional" dependencies and the like (casts the widest possible net) so the output can be confusing. There is room for improvement.

tangkong · 2024-01-08T18:58:26Z

Right, the description here is a bit terse and lacking in detail in cases where multiple packages have the new dependency on their dependency list. It also picks up pypi "optional" dependencies and the like (casts the widest possible net) so the output can be confusing. There is room for improvement.

Extra verbosity might even be a downside, these PR descriptions are long enough as is.

ZLLentz · 2024-01-08T19:12:24Z

@tangkong any more thoughts? Ok to merge and do deploy/post-deploy checks?

tangkong

LGTM

ZLLentz added 4 commits January 4, 2024 12:46

MAINT: use https address for cloning public repos to skip ssh key req

3f58a02

SEC: update for CVE-2023-48795

c457233

BLD: update specs for pcds-5.8.1

781f108

SEC: pycryptodome vuln

4541f2b

BLD: rebuild env with security fix and misc last day of updates

609228d

ZLLentz changed the title ~~WIP: pcds-5.8.1~~ REL: pcds-5.8.1 Jan 8, 2024

ZLLentz marked this pull request as ready for review January 8, 2024 17:19

ZLLentz requested a review from tangkong January 8, 2024 17:19

tangkong reviewed Jan 8, 2024

View reviewed changes

tangkong approved these changes Jan 8, 2024

View reviewed changes

ZLLentz merged commit 619acc6 into pcdshub:master Jan 8, 2024
4 of 9 checks passed

ZLLentz deleted the rel-5.8.1 branch January 8, 2024 19:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REL: pcds-5.8.1 #310

REL: pcds-5.8.1 #310

ZLLentz commented Jan 4, 2024 •

edited

Loading

ZLLentz commented Jan 5, 2024

ZLLentz commented Jan 8, 2024

ZLLentz commented Jan 8, 2024

tangkong commented Jan 8, 2024

tangkong left a comment •

edited

Loading

tangkong Jan 8, 2024

tangkong Jan 8, 2024

ZLLentz Jan 8, 2024 •

edited

Loading

ZLLentz commented Jan 8, 2024

tangkong commented Jan 8, 2024

ZLLentz commented Jan 8, 2024

tangkong left a comment

REL: pcds-5.8.1 #310

REL: pcds-5.8.1 #310

Conversation

ZLLentz commented Jan 4, 2024 • edited Loading

PCDS Package Updates

SLAC Package Updates

Lab Community Package Updates

Python Community Core Package Updates

Other Python Community Major Updates

Added the Following Packages

Added the Following Dependencies

ZLLentz commented Jan 5, 2024

ZLLentz commented Jan 8, 2024

ZLLentz commented Jan 8, 2024

tangkong commented Jan 8, 2024

tangkong left a comment • edited Loading

Choose a reason for hiding this comment

tangkong Jan 8, 2024

Choose a reason for hiding this comment

tangkong Jan 8, 2024

Choose a reason for hiding this comment

ZLLentz Jan 8, 2024 • edited Loading

Choose a reason for hiding this comment

ZLLentz commented Jan 8, 2024

tangkong commented Jan 8, 2024

ZLLentz commented Jan 8, 2024

tangkong left a comment

Choose a reason for hiding this comment

ZLLentz commented Jan 4, 2024 •

edited

Loading

tangkong left a comment •

edited

Loading

ZLLentz Jan 8, 2024 •

edited

Loading