Skip to content

Commit

Permalink
feat: update to python 3.11 (#191)
Browse files Browse the repository at this point in the history
* feat: python 3.11 support

This moves pgbelt to python 3.11.

Code updates are from pyupgrade --py311-plus and pre-commit linting

* deps: update a few more dependencies
  • Loading branch information
andrewthetechie authored Mar 30, 2023
1 parent 5b0ad00 commit 3c44681
Show file tree
Hide file tree
Showing 31 changed files with 827 additions and 802 deletions.
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ updates:
- package-ecosystem: pip
directory: "/"
schedule:
interval: daily
interval: daily
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/[email protected]
with:
python-version: "3.9"
python-version: "3.11"

- name: Upgrade pip
run: |
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ repos:
entry: pyupgrade
language: system
types: [python]
args: [--py37-plus, --keep-runtime-typing]
args: [--py311-plus, --keep-runtime-typing]
- id: reorder-python-imports
name: Reorder python imports
entry: reorder-python-imports
Expand Down
2 changes: 1 addition & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
.github/*
.github/*
32 changes: 14 additions & 18 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,36 +2,32 @@

## [0.1.2](https://github.com/Autodesk/pgbelt/compare/v0.1.1...v0.1.2) (2022-10-03)


### Bug Fixes

* the analyze function should be run as root of a database for safest and least restriction [#87](https://github.com/Autodesk/pgbelt/issues/87) ([#90](https://github.com/Autodesk/pgbelt/issues/90)) ([81c5fde](https://github.com/Autodesk/pgbelt/commit/81c5fde8d1db4be797389a0ff361c00ee3329ab2))
* validator supposed to check 100 rows, not 1 ([#93](https://github.com/Autodesk/pgbelt/issues/93)) ([48122dc](https://github.com/Autodesk/pgbelt/commit/48122dca57b51b3f8114380840e510d70c75eaae))
- the analyze function should be run as root of a database for safest and least restriction [#87](https://github.com/Autodesk/pgbelt/issues/87) ([#90](https://github.com/Autodesk/pgbelt/issues/90)) ([81c5fde](https://github.com/Autodesk/pgbelt/commit/81c5fde8d1db4be797389a0ff361c00ee3329ab2))
- validator supposed to check 100 rows, not 1 ([#93](https://github.com/Autodesk/pgbelt/issues/93)) ([48122dc](https://github.com/Autodesk/pgbelt/commit/48122dca57b51b3f8114380840e510d70c75eaae))

## [0.1.1](https://github.com/Autodesk/pgbelt/compare/v0.1.0...v0.1.1) (2022-09-30)


### Documentation

* update README and quickstart now that pgbelt is on pypi ([#79](https://github.com/Autodesk/pgbelt/issues/79)) ([2a2d4c9](https://github.com/Autodesk/pgbelt/commit/2a2d4c91aaa8e961c2958df2f6150982b8b66c77))
- update README and quickstart now that pgbelt is on pypi ([#79](https://github.com/Autodesk/pgbelt/issues/79)) ([2a2d4c9](https://github.com/Autodesk/pgbelt/commit/2a2d4c91aaa8e961c2958df2f6150982b8b66c77))

## 0.1.0 (2022-09-02)


### Features

* convert to poetry ([07f9a10](https://github.com/Autodesk/pgbelt/commit/07f9a102aed7392e91dc7bffddb41e2bdf69eb52))
* fresh upload of pgbelt with GHA CI ([4df9896](https://github.com/Autodesk/pgbelt/commit/4df98960d662a14c99bb2c60ec80ba3c0317b22b))

- convert to poetry ([07f9a10](https://github.com/Autodesk/pgbelt/commit/07f9a102aed7392e91dc7bffddb41e2bdf69eb52))
- fresh upload of pgbelt with GHA CI ([4df9896](https://github.com/Autodesk/pgbelt/commit/4df98960d662a14c99bb2c60ec80ba3c0317b22b))

### Bug Fixes

* ci move to poetry too ([979c988](https://github.com/Autodesk/pgbelt/commit/979c988666dea0705cecc19fdd7cf941f17f2d6b))
* dev setup in Makefile also installs precommit ([c27cb09](https://github.com/Autodesk/pgbelt/commit/c27cb09615b7a448f31ca07fb6cdab6fe8a10f0f))
* flake8 baited, placing ignore, it broke pydantic ([#17](https://github.com/Autodesk/pgbelt/issues/17)) ([560207d](https://github.com/Autodesk/pgbelt/commit/560207df79276ceece0d4df00221e748f2718648))
* need code of conduct for open-sourcing ([#58](https://github.com/Autodesk/pgbelt/issues/58)) ([4aede91](https://github.com/Autodesk/pgbelt/commit/4aede91a9c4344d3fcaa1972bc6b00cdf70f5ce6))
* pre-commit fixed and run ([d3e188b](https://github.com/Autodesk/pgbelt/commit/d3e188bc2b9641614e8e63a5011adecf4db511e2))
* pre-commit fixes and flake8 config ([#15](https://github.com/Autodesk/pgbelt/issues/15)) ([958afd0](https://github.com/Autodesk/pgbelt/commit/958afd04b5f542704b15809274d039d0accc3cbe))
* remove rando .git file ([#5](https://github.com/Autodesk/pgbelt/issues/5)) ([16d38fd](https://github.com/Autodesk/pgbelt/commit/16d38fd2a7c607552191444d69d0f02212aecfc9))
* remove random .git files ([35213a5](https://github.com/Autodesk/pgbelt/commit/35213a518ce4f3450f462aceb6d139b6379f4cc1))
* update black and fix ci poetry ([5a7a88b](https://github.com/Autodesk/pgbelt/commit/5a7a88beb4cac34a7c1e65f686f8cd0fd1f6ac50))
- ci move to poetry too ([979c988](https://github.com/Autodesk/pgbelt/commit/979c988666dea0705cecc19fdd7cf941f17f2d6b))
- dev setup in Makefile also installs precommit ([c27cb09](https://github.com/Autodesk/pgbelt/commit/c27cb09615b7a448f31ca07fb6cdab6fe8a10f0f))
- flake8 baited, placing ignore, it broke pydantic ([#17](https://github.com/Autodesk/pgbelt/issues/17)) ([560207d](https://github.com/Autodesk/pgbelt/commit/560207df79276ceece0d4df00221e748f2718648))
- need code of conduct for open-sourcing ([#58](https://github.com/Autodesk/pgbelt/issues/58)) ([4aede91](https://github.com/Autodesk/pgbelt/commit/4aede91a9c4344d3fcaa1972bc6b00cdf70f5ce6))
- pre-commit fixed and run ([d3e188b](https://github.com/Autodesk/pgbelt/commit/d3e188bc2b9641614e8e63a5011adecf4db511e2))
- pre-commit fixes and flake8 config ([#15](https://github.com/Autodesk/pgbelt/issues/15)) ([958afd0](https://github.com/Autodesk/pgbelt/commit/958afd04b5f542704b15809274d039d0accc3cbe))
- remove rando .git file ([#5](https://github.com/Autodesk/pgbelt/issues/5)) ([16d38fd](https://github.com/Autodesk/pgbelt/commit/16d38fd2a7c607552191444d69d0f02212aecfc9))
- remove random .git files ([35213a5](https://github.com/Autodesk/pgbelt/commit/35213a518ce4f3450f462aceb6d139b6379f4cc1))
- update black and fix ci poetry ([5a7a88b](https://github.com/Autodesk/pgbelt/commit/5a7a88beb4cac34a7c1e65f686f8cd0fd1f6ac50))
2 changes: 2 additions & 0 deletions DISCLAIMER.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# Disclaimers

## Disclaimer of Warranty

This Work is provided "as is". Any express or implied warranties, including but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall Autodesk be liable for any direct, indirect, incidental, special, exemplary or consequential damages (including, but not limited to, procurement of substitute goods or services, loss of use, data or profits, or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this Work, even if advised of the possibility of such damage.

The User of this Work agrees to hold harmless and indemnify Autodesk and its employees from every claim or liability (whether in tort or in contract), including attorneys' fees, court costs, and expenses, arising in direct consequence of Recipient's use of the item, including but not limited to, claims or liabilities made for injury to or death of personnel of User or third parties, damage to or destruction of property of User or third parties, infringement or other violations of intellectual property or technical data rights.

Nothing in this Work is intended to constitute an endorsement, explicit or implied, by Autodesk of any particular manufacturer's product or service.

## Disclaimer of Endorsement

Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, in this Work does not constitute an endorsement, recommendation, or favoring by Autodesk and shall not be used for advertising or product endorsement purposes.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.9.13-slim
FROM python:3.11-slim
COPY ./ /opt/pgbelt
WORKDIR /opt/pgbelt

Expand Down
80 changes: 40 additions & 40 deletions configs/datacenter-name/database-name/config.json
Original file line number Diff line number Diff line change
@@ -1,44 +1,44 @@
{
"db": "<fill-matching-folder-name>",
"dc": "<fill-matching-folder-name>",
"src": {
"host": "<fill>",
"ip": "<fill>",
"db": "<fill>",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "<fill>"
},
"owner_user": {
"name": "owner",
"pw": "<fill>"
},
"pglogical_user": {
"name": "pglogical",
"pw": "<fill-a-random-string>"
},
"other_users": null
"db": "<fill-matching-folder-name>",
"dc": "<fill-matching-folder-name>",
"src": {
"host": "<fill>",
"ip": "<fill>",
"db": "<fill>",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "<fill>"
},
"dst": {
"host": "<fill>",
"ip": "<fill>",
"db": "<fill>",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "<fill>"
},
"owner_user": {
"name": "owner",
"pw": "<fill>"
},
"pglogical_user": {
"name": "pglogical",
"pw": "<fill-a-random-string>"
},
"other_users": null
"owner_user": {
"name": "owner",
"pw": "<fill>"
},
"tables": null,
"sequences": null
"pglogical_user": {
"name": "pglogical",
"pw": "<fill-a-random-string>"
},
"other_users": null
},
"dst": {
"host": "<fill>",
"ip": "<fill>",
"db": "<fill>",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "<fill>"
},
"owner_user": {
"name": "owner",
"pw": "<fill>"
},
"pglogical_user": {
"name": "pglogical",
"pw": "<fill-a-random-string>"
},
"other_users": null
},
"tables": null,
"sequences": null
}
128 changes: 66 additions & 62 deletions docs/config.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Configuring Pgbelt

## Local Configuration Files

You will need to provide pgbelt with credentials it can use to log in to your databases.
Configuration lives in json files in directories relative to the working directory when you
run pgbelt commands: `./configs/$DATACENTER/$DATABASE/config.json`
Expand All @@ -12,78 +14,80 @@ used to refer to specific config files or groups of config files when executing
They will contain database login credentials!**

### Example

Here's an example config file for a database in `dev` named `mydatabase`.
This file is located at `./configs/dev/mydatabase/config.json`

```json
{
"db": "mydatabase",
"dc": "dev",
"src": {
"host": "mydatabase.example.com",
"ip": "10.235.87.141",
"db": "mydatabasename",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "[REDACTED]"
},
"owner_user": {
"name": "owner",
"pw": "[REDACTED]"
},
"pglogical_user": {
"name": "pglogical",
"pw": "[REDACTED]"
},
"other_users": [
{
"name": "someotheruser",
"pw": "[REDACTED]"
},
{
"name": "anotheruser",
"pw": null
}
]
"db": "mydatabase",
"dc": "dev",
"src": {
"host": "mydatabase.example.com",
"ip": "10.235.87.141",
"db": "mydatabasename",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "[REDACTED]"
},
"owner_user": {
"name": "owner",
"pw": "[REDACTED]"
},
"pglogical_user": {
"name": "pglogical",
"pw": "[REDACTED]"
},
"dst": {
"host": "mynewdatabase.example.com",
"ip": "10.235.110.129",
"db": "mydatabasename",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "[REDACTED]"
},
"owner_user": {
"name": "owner",
"pw": "[REDACTED]"
},
"pglogical_user": {
"name": "pglogical",
"pw": "[REDACTED]"
},
"other_users": null
}
"other_users": [
{
"name": "someotheruser",
"pw": "[REDACTED]"
},
{
"name": "anotheruser",
"pw": null
}
]
},
"dst": {
"host": "mynewdatabase.example.com",
"ip": "10.235.110.129",
"db": "mydatabasename",
"port": "5432",
"root_user": {
"name": "postgres",
"pw": "[REDACTED]"
},
"owner_user": {
"name": "owner",
"pw": "[REDACTED]"
},
"pglogical_user": {
"name": "pglogical",
"pw": "[REDACTED]"
},
"other_users": null
}
}
```

### Fields
* `"db"` the name of the directory containing the config file.
* `"dc"` the name of the directory one level higher.
* `"src"` an object describing the source database instance. May be set null
* `"host"` a hostname pointing to the source database instance.
* `"ip"` the vpc internal ip of the source database instance
* `"db"` the name of the database within the instance you want to migrate.
* `"port"` the port used to connect to the source database instance.
* `"root_user"` an object describing a superuser in the source db. This user must be able to create roles and extensions.
* `"name"` this user's username
* `"pw"` this user's password
* `"owner_user"` a user in the source db that owns all the tables and sequences to be replicated.
* `"pglogical_user"` a user that pgbelt will create in the source db to be used with pglogical.
* `"other_users"` a list of other users in the source db. Users in this list may have null passwords. May be set null.
* `"dst"` same as src but describes the destination database instance. May be set null

- `"db"` the name of the directory containing the config file.
- `"dc"` the name of the directory one level higher.
- `"src"` an object describing the source database instance. May be set null
- `"host"` a hostname pointing to the source database instance.
- `"ip"` the vpc internal ip of the source database instance
- `"db"` the name of the database within the instance you want to migrate.
- `"port"` the port used to connect to the source database instance.
- `"root_user"` an object describing a superuser in the source db. This user must be able to create roles and extensions.
- `"name"` this user's username
- `"pw"` this user's password
- `"owner_user"` a user in the source db that owns all the tables and sequences to be replicated.
- `"pglogical_user"` a user that pgbelt will create in the source db to be used with pglogical.
- `"other_users"` a list of other users in the source db. Users in this list may have null passwords. May be set null.
- `"dst"` same as src but describes the destination database instance. May be set null

Some commands require only the src or dst field to be filled in. Check usage.md or
the help for each command to see if this is the case.
Loading

0 comments on commit 3c44681

Please sign in to comment.