-
Notifications
You must be signed in to change notification settings - Fork 600
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HTTPClient Integration with Faraday v1.5.0 causing SystemStackError (stack level too deep) #714
Comments
Hello @ocha However, I am a bit confused by a couple of things, so hopefully you can provide some clarity there. I'm a bit confused about the provided logs, since I'm not seeing a stack level too deep error in there. It looks like it's multiple warnings for I'm also surprised that the issue doesn't occur in production, only in dev environments. I would expect that the warnings in the logs for that error would only happen when there is no license key (so the agent is unable to connect), but I'm not sure how it is related to the stack level too deep errors. If you could provide the stacktrace for the stack level too deep error itself that you're seeing, that should help clarify what's going on as well. |
Hi @tannalynn . I provided log from newrelic agent, since I thought it would have more useful info. Below is the log from rails sever:
Meanwhile newrelic.log prints a lot of repeating messages likes this:
|
Here is how rails server log looks when there is no issue:
|
Seeing those new relic logs where the agent is unable to create a trace context payload is expected if there is no license key, which would prevent the agent from connecting to the server. Those are something I am able to replicate, however I don't see a stack level too deep error at the same time, so I believe those are unrelated. I see in the rails output, it looks like its giving part of the stack trace for the stack level too deep error
Is it possible to get the full stack trace for this error? The full stacktrace should show what the prepend instrumentation is conflicting with. In your original comment, you mentioned you are able to solve the issue by using the chain instrumentation for HTTPClient, and that this only happens on Faraday 1.5.0. I'm able to add a known prepend conflict for that faraday version so the agent will choose chain when that faraday version is detected, but I would still be curious to see the full stack trace so I can see specifically what in faraday is causing the conflict. |
One thing I wanted to note about those agent errors is that I only get one or two of them when everything is normal (no stack level too deep error). But when the error happens, I get a lot of them in newrelic log.
I am running rails in dev mode with |
@ocha That's very interesting. I'm not sure why that would be the case, perhaps it's something related to the stack level too deep error prevents the agent from connecting to the new relic servers, so the trace context payloads are never able to be created? It's hard to say without the full stack trace though, it's unfortunate you're not able to grab it. |
I'll be closing this issue, please see #731. That issue covers all stack level too deep errors related to Module#prepend conflicts. I've added a comment for reporting this conflict for us to add in our autodetection of prepend conflicts in future versions. Thank you for for letting us know about this conflict! |
Description
When using newrelic_rpm gem with Faraday v1.5.0, I keep getting SystemStackError (stack level too deep). If I disable HTTPClient Integration or change it from
prepend
tochain
then error goes away. Also, if I downgrade to Faraday v1.4.3, error goes away. Looking at faraday changes, I can see that they moved to external dependency for HTTPClient adapter. I am guessing that the order of requiring newrelic gem and faraday's external httpclient adapter gem causes this issue.Expected Behavior
I expect newrelic_rpm instrumentation for faraday v1.5.0 to work and not cause SystemStackError (stack level too deep).
Troubleshooting or NR Diag results
Steps to Reproduce
I dont have a stand alone demo and I cannot share our application code. Faraday gem is used by chewy and appengine dependencies. I think if you have rails 6 (latest), newrelic_rpm (latest), appengine (latest), chewy (latest) gems, and make a chew search (causes http request to elastic search server), it will result in error. Also, I think you need to have distributed tracing enabled in your newrelic config.
Your Environment
I am running local rails dev env, and I dont have newrelic license set up and I have distributed tracing enabled in newrelic yml config. When running in production with license key, I dont get the error.
The text was updated successfully, but these errors were encountered: