Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logfire Only Logging FastAPI Requests, Not SQL Queries with SQLModel/SQLAlchemy #786

Open
alon710 opened this issue Jan 8, 2025 · 5 comments
Labels
Question Further information is requested

Comments

@alon710
Copy link

alon710 commented Jan 8, 2025

Question

Hello Logfire Community,

I'm integrating Logfire with a FastAPI backend that uses SQLModel (built on SQLAlchemy) for database interactions. I aim to have Logfire display detailed logs for both incoming HTTP requests and the SQL queries executed by SQLModel. However, currently, only the FastAPI request data is being logged, and the SQL queries are not appearing in Logfire.

Here's an overview of my setup:

Main Application (main.py):

Configured Logfire with service name, environment, and token.
Instrumented Pydantic and FastAPI:

logfire.instrument_pydantic()
logfire.instrument_fastapi(app)

Included routers and set up the FastAPI app lifecycle.
Database Configuration (database.py):

Created the SQLAlchemy engine:

engine = create_engine(
    url=DATABASE_URL,
    echo=False if IS_CLOUD else True,
    connect_args={} if IS_CLOUD else {"check_same_thread": False},
)

Instrumented SQLAlchemy with Logfire:

logfire.instrument_sqlalchemy(engine=engine)

Provided a session dependency for FastAPI endpoints.
Endpoint Example:

Defined a FastAPI endpoint that uses a service and data access layer to execute database queries.
Issue: Despite the above setup, Logfire only captures and displays logs related to FastAPI requests. The SQL queries executed via SQLModel/SQLAlchemy are not being logged or visible in the Logfire dashboard.

What I've Tried:

Ensured that logfire.instrument_sqlalchemy(engine=engine) is called after creating the engine.
Verified that SQLAlchemy's echo parameter is set appropriately for logging.
Checked Logfire configurations and tokens for correctness.

Questions:

Am I missing any additional configuration steps to enable SQL query logging with SQLModel/SQLAlchemy in Logfire?
Are there any compatibility considerations between Logfire and SQLModel that I should be aware of?
Could there be any issues with the order of instrumentation calls that might prevent SQL logs from being captured?
Additional Information:

Logfire Version: logfire 3.0.0
FastAPI Version: 0.111.1
SQLModel Version: 0.0.19
SQLAlchemy Version: 2.0.36
Environment: Local

 # main.py
import logfire
from fastapi import FastAPI

logfire.configure(
    service_name="Recommendations",
    environment="production",
    token="YOUR_LOGFIRE_TOKEN",
)
logfire.instrument_pydantic()
logfire.instrument_fastapi(app)

# database.py
import logfire
from sqlmodel import create_engine

engine = create_engine(DATABASE_URL)
logfire.instrument_sqlalchemy(engine=engine)
Thank you for your assistance!

image

@alon710 alon710 added the Question Further information is requested label Jan 8, 2025
@alexmojaki
Copy link
Contributor

I see that some debug spans are hidden. If you show them:

Screenshot 2025-01-08 at 17 12 03

then you might see a connect span from sqlalchemy.

With this code:

from fastapi import FastAPI
from sqlmodel import Field, SQLModel, create_engine

import logfire

logfire.configure()
app = FastAPI()
logfire.instrument_fastapi(app)


@app.get('/heroes/{hero_id}')
def read_hero(hero_id: int):
    engine = create_engine('sqlite:///:memory:')
    logfire.instrument_sqlalchemy(engine=engine)

    class Hero(SQLModel, table=True):
        id: int = Field(primary_key=True)

    SQLModel.metadata.create_all(engine)


if __name__ == '__main__':
    import uvicorn

    uvicorn.run(app)

I see logs from both fastapi and sqlalchemy. Can you provide a complete reproduction?

@alon710
Copy link
Author

alon710 commented Jan 8, 2025

I see that some debug spans are hidden. If you show them:

Screenshot 2025-01-08 at 17 12 03

then you might see a connect span from sqlalchemy.

With this code:

from fastapi import FastAPI
from sqlmodel import Field, SQLModel, create_engine

import logfire

logfire.configure()
app = FastAPI()
logfire.instrument_fastapi(app)


@app.get('/heroes/{hero_id}')
def read_hero(hero_id: int):
    engine = create_engine('sqlite:///:memory:')
    logfire.instrument_sqlalchemy(engine=engine)

    class Hero(SQLModel, table=True):
        id: int = Field(primary_key=True)

    SQLModel.metadata.create_all(engine)


if __name__ == '__main__':
    import uvicorn

    uvicorn.run(app)

I see logs from both fastapi and sqlalchemy. Can you provide a complete reproduction?

Thanks! I now see connect, but is there a way to also see the queries? I want to be able to find bottlenecks within the HTTP request business logic

@alexmojaki
Copy link
Contributor

You should be seeing queries. The fact that you see connect suggests that the SQLAlchemy integration is at least partly working, but if queries are actually happening then that's a bug. But we'll need a complete example to reproduce.

@alon710
Copy link
Author

alon710 commented Jan 8, 2025

What exactly do you need? ill be happy to provide more data

image

@alexmojaki
Copy link
Contributor

I've provided complete code combining FastAPI and SQLModel that seems to work. Does it work for you? Can you provide similar code that doesn't work for you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants