you run LLM features in production and want their traces, latency, errors and cost beside the rest of your telemetry.
you want to know what your developers' AI tools were asked and what they changed in your repositories.
If your organisation runs Datadog, its LLM observability is the path of least resistance for AI in production: traces from your services, alongside the APM, logs and infrastructure data you already correlate against, with the alerting and retention machinery your team knows. For monitoring an AI feature you ship, arguing against it would be silly.
"We already have observability for AI" is true and incomplete, and the incompleteness is invisible on the dashboard. Observability platforms see what your systems do. A developer running Cursor is not one of your systems: there is no agent on that path, no service emitting spans, and no instrumentation point to add. The traffic is between a vendor's desktop client and a vendor's API.
So the coverage question is not about quality of tooling. It is about whether the thing you want observed emits telemetry into your pipeline at all — and for AI coding tools, it does not. What they emit is files on disk: JSONL transcripts, SQLite databases, per-workspace session files.
Your observability stack covers the AI inside your systems. The AI writing those systems leaves its evidence on a laptop, in a format nothing in your pipeline is watching.
It is a fair question, and for some teams it is the right answer — the files are structured, and shipping them as logs is not hard. Two things usually stop it. Cost: AI transcripts are large and log ingest is priced accordingly, so a year of retention on a hundred developers gets expensive fast for data that is read a handful of times. And shape: the useful artefact is not a log line, it is prompt, response and the resulting file diff joined together and browsable as a timeline. Log search will find the string; it will not show you what changed.
If you do decide to build it in-house on top of what you already run, the honest advice is to start by reading the on-disk formats rather than by asking developers to change how they work. That is the part we spent the most time on, and it is where a home-grown version usually stalls: eight tools, five formats, and each one changes between releases.
There is no overlap to rationalise. Datadog owns runtime — is the feature up, is it slow, what does it cost. We own provenance — what was asked, what came back, what changed. If a single budget line covers both, that is an accounting artefact rather than a product decision.