Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Isolation level support #481

Open
tuxzz opened this issue Jul 5, 2020 · 25 comments · May be fixed by #3614
Open

Isolation level support #481

tuxzz opened this issue Jul 5, 2020 · 25 comments · May be fixed by #3614

Comments

@tuxzz
Copy link

tuxzz commented Jul 5, 2020

There are four transaction isolation levels in SQL language and most of database support them. But I can't find how to set isolation level for transactions in sqlx.
Is it a missing feature?

@mehcode
Copy link
Member

mehcode commented Jul 12, 2020

It is currently missing, yea. I'm open to API ideas.

// would rename existing ::transaction to ::with_transaction
let mut tx = conn.transaction() // -> TransactionOptions
    .read_only()
    .isolation_level(IsolationLevel::RepeatableRead)
    .begin()
    .await?;
let mut tx = TransactionOptions::new(&mut conn)
    .isolation_level(IsolationLevel::Snapshot)
    .begin()
    .await?;

@pythoneer
Copy link

pythoneer commented Sep 8, 2020

What are the pros and cons of the different approaches? Just from the first look at it i have no strong feelings one way or the other. The first one looks just fine and is maybe more discoverable?

@LucasPickering
Copy link

Is there any workaround for this currently?

@abonander
Copy link
Collaborator

I was wondering if we could start with something like .begin_with("BEGIN READ ONLY, READ COMMITTED").await (on the Connection trait and Pool) which returns a Transaction since that would cover most use cases without needing to figure out database-specifics.

For sanity, we can assert that the connection was put into a transaction if it wasn't already. Both Postgres and MySQL tell us after executing a command whether the connection is in a transaction, and SQLite has get_autocommit().

@ivan
Copy link
Contributor

ivan commented Nov 13, 2020

@LucasPickering if you're using PostgreSQL, I believe you can do

use sqlx::postgres::PgPoolOptions;

let pg_pool = PgPoolOptions::new()
    .after_connect(|conn| Box::pin(async move {
        conn.execute("SET default_transaction_isolation TO 'repeatable read'").await?;
        Ok(())
    }))
    .connect(&uri).await?;

@pythoneer
Copy link

pythoneer commented Nov 13, 2020

@ivan And for different isolation levels create different pools and use the pool that is "holding" the right isolation level? Or is this global?

@Diggsey
Copy link
Contributor

Diggsey commented Mar 14, 2021

I like the begin_with approach.

@LukeMathWalker
Copy link
Contributor

I'd be happy to pick this up if we landed on an API design - is there a consensus among sqlx's maintainers at this point? @abonander

@pythoneer
Copy link

By choosing the "first" approach we would change the current one from

conn.transaction(|conn|Box::pin(async move {
    query("select * from ..").fetch_all(conn).await
})).await

into

conn.with_transaction(|conn|Box::pin(async move {
    query("select * from ..").fetch_all(conn).await
}), IsolationLevel::RepeatableRead).await

right?

@patrick-gu
Copy link

You can set the isolation level by executing SET TRANSACTION after creating the transaction:

let mut tx = conn.begin().await?;
tx.execute("SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;")
    .await?;

Documentation for SET TRANSACTION:

SQLite does not have SET TRANSACTION, and uses SERIALIZABLE isolation except with PRAGMA read_uncommitted turned on, as documented.

It may still be necessary to add a custom BEGIN as with SQLite's BEGIN IMMEDIATE as mentioned in #1182.

Hope this helps!

@abonander
Copy link
Collaborator

@LukeMathWalker I think my proposal for .begin_with("<BEGIN statement with flags>") would be a good start: #481 (comment)

If nothing else it's something we can build on if it's not satisfactory.

@LukeMathWalker
Copy link
Contributor

I've started to work on this 👍🏻

@LukeMathWalker
Copy link
Contributor

Question: what should happen when the user tries to invoke begin_with within the context of an existing transaction?
Right now sqlx determines, based on the current depth, if it should issue a BEGIN or a SAVEPOINT statement.
Do we want to keep the same behaviour? Or do we want to error, given that this is probably not what the user expects? @abonander

@saward
Copy link

saward commented Jul 1, 2022

Question: what should happen when the user tries to invoke begin_with within the context of an existing transaction? Right now sqlx determines, based on the current depth, if it should issue a BEGIN or a SAVEPOINT statement. Do we want to keep the same behaviour? Or do we want to error, given that this is probably not what the user expects? @abonander

Maybe it's too tricky given the way sqlx is written, but would this be a good place for a typestate pattern (http://cliffle.com/blog/rust-typestate/) so that calling begin_with after a transaction has begun is a compile time error?

@davidbnk
Copy link

davidbnk commented Mar 9, 2023

Using this in MySQL:

let mut tx = conn.begin().await?;
tx.execute("SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;")
    .await?;

Gives:

1568 (25001): Transaction characteristics can't be changed while a transaction is in progress

Is there another way?

@jclulow
Copy link

jclulow commented Apr 16, 2023

Is there any plan to finish even a limited SQLite-specific version of this? It is, as far as I can tell, not possible to do safe read-modify-write in SQLite without the ability to use BEGIN IMMEDIATE.

@jvliwanag
Copy link

Using this in MySQL:

let mut tx = conn.begin().await?;
tx.execute("SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;")
    .await?;

Gives:

1568 (25001): Transaction characteristics can't be changed while a transaction is in progress

Is there another way?

For MySQL, the SET TRANSACTION LEVEL should be done prior to starting a transaction. The workaround I have for now is:

use sqlx::Executor;

let mut con = self.pool.acquire().await?;

con.execute("SET TRANSACTION ISOLATION LEVEL READ_COMMITED");
let mut tx =
    sqlx_core::transaction::Transaction::begin(MaybePoolConnection::PoolConnection(con))
        .await?;

Note that for this to work, you'd have to add sqlx-core to your cargo dependencies.

@Reknij
Copy link

Reknij commented Jan 29, 2024

Is there any plan to finish even a limited SQLite-specific version of this? It is, as far as I can tell, not possible to do safe read-modify-write in SQLite without the ability to use BEGIN IMMEDIATE.

Hi, have you found a solution to use BEGIN IMMEDIATE?

@jclulow
Copy link

jclulow commented Jan 29, 2024

Is there any plan to finish even a limited SQLite-specific version of this? It is, as far as I can tell, not possible to do safe read-modify-write in SQLite without the ability to use BEGIN IMMEDIATE.

Hi, have you found a solution to use BEGIN IMMEDIATE?

Yes! I now use rusqlite and sea-query, where I have proper control over transactions.

@glebpom
Copy link

glebpom commented Jul 3, 2024

I have created the PR, which supports transactions with custom SQL. PR link. This generally work for my usecase (MariaDB), but probably requires more testing and it's unclear how to handle this for sqlite. Please comment on the PR if something specific should be addressed.

@lucasyvas
Copy link

lucasyvas commented Jul 3, 2024

API wise, the feature is more useful if it's generalized to not just be about isolation level. For example, I also set session variables per transaction and the workflow for this is cumbersome.

Unrelated, but PoolConnection can be owned, yet Connection is a mutable borrow. So it's challenging to actually pass that tx around besides by explicit parameter to a function call. I found a way around all this but it involves accessing the underlying connection and basically not using the native transaction capability in sqlx at all. If creating a transaction could have it take ownership of the underlying executable connection that would be amazing for my use case (If I understand the API properly). Anyway...

tldr; more flexibility in creating the transaction, whatever it may involve, is appreciated. Isolation level is a great start because it is pretty common.

@Crazytieguy
Copy link

Just wanted to post my workaround for sqlite in case it's helpful. Basically I have a dummy table called "acquire_write_lock" with one row that I populate on pool creation and I write to it before making my reads.

let mut transaction = pool.begin().await?;
sqlx::query!("UPDATE acquire_write_lock SET lock = TRUE WHERE id = 1")
    .execute(transaction.as_mut())
    .await?;

@Reknij
Copy link

Reknij commented Jul 12, 2024

Just wanted to post my workaround for sqlite in case it's helpful. Basically I have a dummy table called "acquire_write_lock" with one row that I populate on pool creation and I write to it before making my reads.

let mut transaction = pool.begin().await?;
sqlx::query!("UPDATE acquire_write_lock SET lock = TRUE WHERE id = 1")
    .execute(transaction.as_mut())
    .await?;

Haha, I also do like this.

@ckampfe
Copy link
Contributor

ckampfe commented Jul 19, 2024

For anyone else finding this issue because you're looking for SQLite's BEGIN_IMMEDIATE transaction support in sqlx, I made this little hack that so far seems to be working. I'm sure it's limited (and possibly broken) in plenty of ways but feel free to take it if you find it useful: https://gist.github.com/ckampfe/cfa85d409874e487c5117b6f242d9a07

@steveklabnik
Copy link

I need SQLite's BEGIN EXCLUSIVE for a project I'm working on. It's only in one place, and so I've ended up also using rusqlite, just for that one query. It ends up needing to use unsafe to build the connection, which is unfortunate, but it does work at least.

clintjedwards added a commit to clintjedwards/gofer that referenced this issue Oct 1, 2024
After ignoring the database settings for a while the it came back to
bite us when implementing the new extension paradigm. Sqlite started
emitting database busy errors when a long running transaction was
interrupted by some other write.

This is because when you start a transaction in sqlite by default
it occurs as DEFFERED. This means that sqlite does not try to lock th
database until it comes across a write call. This means that some other
thread could possibly come in and make a write call before the current
transaction is finished. When this happens sqlite will see that the underlying
data has changed and return an error for the open transaction.

This is obviously bad because losing transactions is not a current situation
we recover gracefully from. Instead it would be better if sqlite simply
followed suite with other databases and just immediatley held a lock
when opening a transaction. Which is actually able to be enabled by
simply using the `BEGIN IMMEDIATE` statement when starting a transaction.

But nothing is ever that simple. The library that we're using sqlx
[does not yet support](launchbadge/sqlx#481)
the ability to call `BEGIN IMMEDIATE` on transactions. Thus we had to
come up with a hack.

The hack being that when opening a transaction we immediately call
a write to a dummy table as our first call. This should immediatley
cause sqlite to hold a lock and more or less mimic the behavior of
`BEGIN IMMEDIATE`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet