HMS-4932: Add search module streams #897

Andrewgdewar · 2024-11-18T22:27:35Z

Summary

Adds the POST "snapshots/module_streams/search" endpoint.
That takes the following inputs

{
  "uuids": [
    "9d164ac6-b866-4808-af1f-847fbf588658",
    "36e449b9-3d61-43e8-a495-6ac332fdd3a8"
  ],
  "rpm_names": [],
  "search": ""
}

and returns a list of modules with their streams:

{
 [
  {
    "module_name": "mariadb",
    "streams": [
      {
        "name": "mariadb",
        "stream": "10.11",
        "context": "rhel9",
        "arch": "x86_64",
        "cersion": "9040020240126110506",
        "description": "MariaDB is a....",
        "profiles": {
          "client": [
            "mariadb"
          ],
          "galera": [
            "mariadb-server",
            "mariadb-server-galera"
          ],
          "server": [
            "mariadb-server"
          ]
        }
      }
    ]
  },
]

}

Needs to update tangy and tangy reference prior to review

Testing steps

jlsherrill · 2024-11-18T22:30:15Z

https://issues.redhat.com/browse/HMS-4932

jlsherrill · 2024-11-27T15:30:21Z

https://issues.redhat.com/browse/HMS-4932

EDIT: remove duplicated HMS

swadeley · 2024-12-02T01:26:15Z

/retest

jlsherrill · 2024-12-02T01:30:15Z

https://issues.redhat.com/browse/HMS-HMS-4932

pkg/api/rpms.go

pkg/handler/rpms.go

pkg/api/rpms.go

swadeley · 2024-12-03T07:14:56Z

Warning: Unexpected input(s) 'skip-go-installation',

Andrewgdewar · 2024-12-03T16:16:32Z

Warning: Unexpected input(s) 'skip-go-installation',

This requires the tangy PR to be integrated, don't test this yet.

pkg/api/module_streams.go

pkg/handler/module_streams.go

pkg/api/module_streams.go

jlsherrill · 2024-12-12T17:41:10Z

seems to be some linting issue

swadeley · 2024-12-13T02:20:58Z

/retest

swadeley · 2024-12-13T02:56:39Z

Warning: Unexpected input(s) 'skip-go-installation',

This requires the tangy PR to be integrated, don't test this yet.

OK but winter is coming :)

02:31:28 pkg/dao/module_streams.go:65:37: (*config.Tang).RpmRepositoryVersionModuleStreamsList undefined (type tangy.Tangy has no field or method RpmRepositoryVersionModuleStreamsList)

Andrewgdewar · 2024-12-16T17:18:10Z

seems to be some linting issue

tangy..

pkg/handler/module_streams.go

pkg/handler/module_streams_test.go

pkg/api/module_streams.go

jlsherrill · 2024-12-17T18:21:44Z

few small comment, but overall looks good

jlsherrill · 2024-12-19T13:46:40Z

lets merge content-services/tang#14 and tag a new tang version, then pull that update in here?

jlsherrill · 2024-12-20T20:41:30Z

note this require iqe update to merge

swadeley · 2025-01-02T01:51:13Z

@Andrewgdewar please rebase

swadeley · 2025-01-02T01:51:27Z

/retest

swadeley · 2025-01-03T06:26:54Z

Hi

I built the API docs for IQE but its taking forever to get a suitable RHEL repo in ephemeral snapshotted. Will test in stage.

Format for IQE:

In [9]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search='mariadb', uuids=['776a0172-5855-4734-b382-8e4d60a44a84'], rpm_names=[]))

swadeley · 2025-01-03T07:31:31Z

Hi

testing with: https://rverdile.fedorapeople.org/dummy-repos/modules/repo1/

In [9]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search='', uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=['kangaroo']))
2025-01-03 15:29:40.761 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[9]: 
[{'module_name': 'kangaroo',
  'streams': [{'arch': 'noarch',
               'context': 'deadbeef',
               'description': 'A module for the kangaroo 0.2 package',
               'name': 'kangaroo',
               'profiles': {'default': ['kangaroo']},
               'stream': '0',
               'version': '20180704111719'}]}]

but nothing if blank search:

In [12]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search="", uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=[""]))
2025-01-03 15:33:05.554 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[12]: []

Andrewgdewar · 2025-01-06T18:41:58Z

Hi

testing with: https://rverdile.fedorapeople.org/dummy-repos/modules/repo1/

In [9]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search='', uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=['kangaroo']))
2025-01-03 15:29:40.761 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[9]: 
[{'module_name': 'kangaroo',
  'streams': [{'arch': 'noarch',
               'context': 'deadbeef',
               'description': 'A module for the kangaroo 0.2 package',
               'name': 'kangaroo',
               'profiles': {'default': ['kangaroo']},
               'stream': '0',
               'version': '20180704111719'}]}]

but nothing if blank search:

In [12]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search="", uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=[""]))
2025-01-03 15:33:05.554 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[12]: []

This is expected. You need to give the api at least one uuid.

swadeley · 2025-01-07T01:57:57Z

Hi
testing with: https://rverdile.fedorapeople.org/dummy-repos/modules/repo1/

In [9]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search='', uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=['kangaroo']))
2025-01-03 15:29:40.761 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[9]: 
[{'module_name': 'kangaroo',
  'streams': [{'arch': 'noarch',
               'context': 'deadbeef',
               'description': 'A module for the kangaroo 0.2 package',
               'name': 'kangaroo',
               'profiles': {'default': ['kangaroo']},
               'stream': '0',
               'version': '20180704111719'}]}]

but nothing if blank search:

In [12]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search="", uuids=['6d1da6ea-6603-4cb9-bec4-f803dcaee830'], rpm_names=[""]))
2025-01-03 15:33:05.554 [    INFO] [iqe.base.rest_client] REST: POST https://ee-bt0c0eqb.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[12]: []

This is expected. You need to give the api at least one uuid.

OK, thank you

swadeley · 2025-01-16T03:57:59Z

forgot to add this better example output:

In [7]: app.content_sources.rest_client.module_streams_api.search_snapshot_module_streams(dict(search='walrus', uuids=['177e9b6f-0125-4628-9953-40195e368822'], rpm_names=[]))
2025-01-16 11:52:41.209 [    INFO] [iqe.base.rest_client] REST: POST https://ee-fryb0ek4.apps.crc-eph.r9lp.p1.openshiftapps.com/api/content-sources/v1/snapshots/module_streams/search with query params [] and x-rh-insights-request-id=<>
Out[7]: 
[{'module_name': 'walrus',
  'streams': [{'arch': 'x86_64',
               'context': 'c0ffee42',
               'description': 'A module for the walrus 0.71 package',
               'name': 'walrus',
               'profiles': {'default': ['walrus'], 'flipper': ['walrus']},
               'stream': '0.71',
               'version': '20180707144203'},
              {'arch': 'x86_64',
               'context': 'deadbeef',
               'description': 'A module for the walrus 5.21 package',
               'name': 'walrus',
               'profiles': {'default': ['walrus']},
               'stream': '5.21',
               'version': '20180704144203'}]}]

jlsherrill added the qe-testing-needed label Nov 18, 2024

rverdile self-assigned this Nov 20, 2024

jlsherrill changed the title ~~Fixes 4932: Add search module streams~~ Fixes HMS-4932: Add search module streams Nov 27, 2024

Andrewgdewar force-pushed the HMS-4932 branch 3 times, most recently from 76493c7 to 852c0f8 Compare November 29, 2024 00:35

Andrewgdewar marked this pull request as ready for review November 29, 2024 00:36

Andrewgdewar requested a review from rverdile November 29, 2024 00:37

jlsherrill reviewed Dec 2, 2024

View reviewed changes

pkg/api/rpms.go Outdated Show resolved Hide resolved

jlsherrill reviewed Dec 2, 2024

View reviewed changes

pkg/handler/rpms.go Outdated Show resolved Hide resolved

jlsherrill reviewed Dec 2, 2024

View reviewed changes

pkg/api/rpms.go Outdated Show resolved Hide resolved

rverdile mentioned this pull request Dec 2, 2024

Add list module Streams endpoint content-services/tang#14

Merged

Andrewgdewar force-pushed the HMS-4932 branch 2 times, most recently from f94d7c7 to 94715d6 Compare December 3, 2024 21:03

jlsherrill changed the title ~~Fixes HMS-4932: Add search module streams~~ Fixes 4932: Add search module streams Dec 4, 2024

Andrewgdewar force-pushed the HMS-4932 branch from e17a2cb to bdb4598 Compare December 5, 2024 21:01

jlsherrill reviewed Dec 9, 2024

View reviewed changes

pkg/api/module_streams.go Outdated Show resolved Hide resolved

jlsherrill reviewed Dec 9, 2024

View reviewed changes

pkg/handler/module_streams.go Outdated Show resolved Hide resolved

rverdile reviewed Dec 9, 2024

View reviewed changes

pkg/api/module_streams.go Outdated Show resolved Hide resolved

rverdile reviewed Dec 10, 2024

View reviewed changes

pkg/api/module_streams.go Outdated Show resolved Hide resolved

jlsherrill reviewed Dec 16, 2024

View reviewed changes

pkg/handler/module_streams.go Outdated Show resolved Hide resolved

Andrewgdewar force-pushed the HMS-4932 branch from 6acf60b to 8cd9c5c Compare December 16, 2024 19:02

jlsherrill reviewed Dec 16, 2024

View reviewed changes

pkg/handler/module_streams_test.go Show resolved Hide resolved

jlsherrill reviewed Dec 17, 2024

View reviewed changes

pkg/api/module_streams.go Outdated Show resolved Hide resolved

jlsherrill reviewed Dec 17, 2024

View reviewed changes

pkg/api/module_streams.go Outdated Show resolved Hide resolved

jlsherrill changed the title ~~Fixes 4932: Add search module streams~~ HMS-4932: Add search module streams Dec 18, 2024

Andrewgdewar force-pushed the HMS-4932 branch 2 times, most recently from 67e22a1 to fe321db Compare December 20, 2024 19:05

Andrewgdewar requested review from jlsherrill and rverdile December 20, 2024 19:05

jlsherrill approved these changes Dec 20, 2024

View reviewed changes

jlsherrill added the Ready for QE label Dec 20, 2024

Andrewgdewar added 4 commits January 2, 2025 08:15

Fixes 4932: Add search module streams

87aa126

Update tang version

06d4167

Update middleware

499155d

Omit nil check

9d537b7

Andrewgdewar force-pushed the HMS-4932 branch from ef4ac12 to 9d537b7 Compare January 2, 2025 15:15

swadeley merged commit f3dc32a into content-services:main Jan 7, 2025
11 of 12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HMS-4932: Add search module streams #897

HMS-4932: Add search module streams #897

Andrewgdewar commented Nov 18, 2024 •

edited

Loading

jlsherrill commented Nov 18, 2024

jlsherrill commented Nov 27, 2024 •

edited by swadeley

Loading

swadeley commented Dec 2, 2024

jlsherrill commented Dec 2, 2024

swadeley commented Dec 3, 2024

Andrewgdewar commented Dec 3, 2024

jlsherrill commented Dec 12, 2024

swadeley commented Dec 13, 2024

swadeley commented Dec 13, 2024

Andrewgdewar commented Dec 16, 2024

jlsherrill commented Dec 17, 2024

jlsherrill commented Dec 19, 2024

jlsherrill commented Dec 20, 2024

swadeley commented Jan 2, 2025

swadeley commented Jan 2, 2025

swadeley commented Jan 3, 2025

swadeley commented Jan 3, 2025 •

edited

Loading

Andrewgdewar commented Jan 6, 2025

swadeley commented Jan 7, 2025

swadeley commented Jan 16, 2025

HMS-4932: Add search module streams #897

HMS-4932: Add search module streams #897

Conversation

Andrewgdewar commented Nov 18, 2024 • edited Loading

Summary

Testing steps

jlsherrill commented Nov 18, 2024

jlsherrill commented Nov 27, 2024 • edited by swadeley Loading

swadeley commented Dec 2, 2024

jlsherrill commented Dec 2, 2024

swadeley commented Dec 3, 2024

Andrewgdewar commented Dec 3, 2024

jlsherrill commented Dec 12, 2024

swadeley commented Dec 13, 2024

swadeley commented Dec 13, 2024

Andrewgdewar commented Dec 16, 2024

jlsherrill commented Dec 17, 2024

jlsherrill commented Dec 19, 2024

jlsherrill commented Dec 20, 2024

swadeley commented Jan 2, 2025

swadeley commented Jan 2, 2025

swadeley commented Jan 3, 2025

swadeley commented Jan 3, 2025 • edited Loading

Andrewgdewar commented Jan 6, 2025

swadeley commented Jan 7, 2025

swadeley commented Jan 16, 2025

Andrewgdewar commented Nov 18, 2024 •

edited

Loading

jlsherrill commented Nov 27, 2024 •

edited by swadeley

Loading

swadeley commented Jan 3, 2025 •

edited

Loading