reporting spans to Jaeger backend with http transport will send an endless loop of reported spans.

[limfriend]

When we configure jager's http reporting, such as "-DJAEGER_ENDPOINT = http: //myjaeger.com/api/traces",
it will send an endless loop problem. This is because jaeger's http reporting uses the okhttp component.
When the span is reported via okhttp, it will be captured by the okhttp plugin to generate span data,
and when this span data is reported to the jaeger collector, new span data will be generated.

@safris could you help to merge the pull and upgrade okhttp plugin to java-specialaget? opentracing-contrib/java-okhttp#36

[safris]

Hi @limfriend, the SpecialAgent should not be instrumenting the tracers. There is a specific mechanism that attempts to prevent such instrumentation from happening. Issue #381 was opened regarding a similar situation with HttpURLConnection, and it was resolved. However, this issue, and issue #531 seem to suggest that there is still "self tracing" happening.

The right way to fix this is at a low level, which would apply to all rules, instead of just OkHttp.

In order to understand what is happening, would you be able to provide a log file with -Dsa.log.level=FINER?

What I'd be looking for are entries like this:

-------> Intercept [OkHttpAgentRule@main]: public java.util.List okhttp3.OkHttpClient.interceptors()

Here, OkHttpAgentRule is the rule class, main is the thread's name, and the rest is the method signature being intercepted.

If you can capture all of these entries, we'll try to understand which are coming from Jaeger, and will figure out why it's not being prevented.

[safris]

Hi @limfriend, I wanted to check in with you about this issue? Do you have a log file you can share?

[RashmiRam]

@safris I am also facing the same issue.

This is my configuration from running a dropwizard app
java -javaagent:/Users/user/Github/tracing/opentracing-specialagent-1.7.2.jar -Dsa.log.level=FINER -Dsa.exporter=jaeger -DJAEGER_PROPAGATION=jaeger -DJAEGER_REPORTER_LOG_SPANS=true -DJAEGER_SAMPLER_TYPE=const -DJAEGER_SAMPLER_PARAM=1 -DJAEGER_SERVICE_NAME=myService -DJAEGER_ENDPOINT=http://localhost:14268/api/traces -jar myservice.jar server config.yml

The following logs
specialagent_debug.log
just repeats for every request to http://localhost:14268/api/traces

[safris]

Hi @RashmiRam, thank you for this log file. It seems that there is indeed a bug with the "self tracing prevention" mechanism. Would you be able to provide a sample application that demonstrates this behavior? The code that is involved with this "self tracing prevention" mechanism is complex, and I would need to debug it directly in order to figure out why the jaeger.RemoteReporter-QueueProcessor thread is not being marked as "non instrumentable".

[safris]

@limfriend and @RashmiRam, this issue has been resolved and released with SpecialAgent v1.7.3.