Telemetry collection

Telemetry data (logs, metrics, and traces) are collected for GlobalDcc.Api.Server, GlobalDcc.Backend.Server, and GlobalDcc.RoadNetManager.Server. For each of these projects their Startup classes calls static methods to take care of setting this up.

The OTEL... environment variables set in \dev\docker-compose\docker-compose.yml and \dev\docker-compose\.env ensures that logs, metrics, and traces are send to AMCS's Grafana Playground environment when Global DCC is started from docker-compose (i.e. when you as a developer start Global DCC locally). To view logs, metrics, and traces in Grafana Playground:

Go to Explore in the Grafana dashboard.
In the search box type "play".
From the search box's drop down select:
- "Mimir Playground" to query metrics.
- "Loki Playground" to query logs.
- "Tempo Playground" to query traces.
To only see data from Global DCC api/backend/road net manager set the service name gdcc-api/gdcc-backend/gdcc-roadnetmanager in the query.

For Dev, Stg and Prd the OTEL endpoints are configured in globaldcc-values.yaml under the respective environment and regional sub folders of GlobalDccDeploymentK8

Explore in the Grafana dashboard and select in the dropdown what you want to query for.

Metrics: AMCS Mimir { DEV / STG / PRD }
Logs: AMCS Loki { DEV / STG / PRD }
Traces: AMCS Tempo { DEV / STG / PRD }

Custom metrics

Custom metrics are easy to implement. By adding OpenTelemetry --> WithMetrics, we automatically enable the IMeterFactory in Asp.Net core. That means that we can use the IMeterFactory via Depency Injection to create/get a Meter which can create/get instruments that we can use throughout the code.

You can get the IMeterFactory by Dependency Injecting it through e.g. the constructor, or getting it from the HttpContext.

The following code line will give you the IMeterFactory in your ApiController: IMeterFactory meterFactory = ControllerContext.HttpContext.RequestServices.GetRequiredService<IMeterFactory>();

Then all you have to do is to create/get your Meter like so: Meter meter = meterFactory.Create("GlobalDcc");, create/get whatever instrument you like, and make that instrument do what you want it to do.

The Meter will have to be added by doing .AddMeter("MyMeterName") within the option of .WithMetrics() in .AddOpenTelemtry() but if you just use the meter name "GlobalDcc" that has already been done for you, and unless you have a good reason for doing so, you shouldn't add any additional meters.

Make sure you give the instruments on the meter a unique describing name, so it doesn't end up with an instrument name that has already been used.

Custom metrics​

Custom metrics