Here is a question most teams can't answer with confidence. If someone asked you to prove that no patient names, phone numbers or PAN numbers are sitting in your tracing backend, could you?
Traces collect a lot. Developers add request bodies as span attributes to debug a tricky bug. An SDK records full URLs with query strings. Someone logs a user ID that turns out to be an email address. None of it is malicious, and all of it ends up stored, often for weeks, in a system that security reviews rarely look at.
Grafana Tempo 3.1, which InfoQ covered this week, adds a tool for cleaning that up.
Redaction by query
Tempo already had a tempo-cli redact command, but you had to give it specific trace IDs. In 3.1 you can give it a TraceQL query instead, so you can remove every trace where, say, a span attribute matches a field you should never have captured.
The query language is limited for now: one spanset filter, equality checks on resource.* and span.* attributes, joined with && and ||. You can narrow the job with start and end times. Compaction for that tenant pauses while it runs.
Two warnings from the release notes deserve to be repeated. Run it with --dry-run first to see how many traces match, because the deletion can't be undone. And only use the time window flags once every scheduler and worker is on 3.1. Older workers ignore the window and delete every match in each block, which is a lot more than you asked for.
Other useful changes
If you sample traces, metrics built from them have always undercounted. An experimental hint, with(extrapolate=true), now scales results using the sampling rate that OpenTelemetry records, so at 50% sampling each stored span counts as two. It works for rates, counts, sums, averages and percentiles, but not min or max.
Kafka support got proper security. Tempo can now talk to Kafka over TLS with SCRAM, OAUTHBEARER or AWS MSK IAM, where before it only did plain authentication over an unencrypted connection. If you run Tempo with Kafka in a regulated environment, that alone is a reason to upgrade.
New blocks now use the vParquet5 format by default, and simple metrics queries are reported to run about twice as fast.
A cleanup is a symptom
A redaction tool is good to have, but needing it means the data got in. For teams in healthtech, fintech or anything covered by India's DPDP Act or HIPAA, the more lasting fix is upstream. Agree which attributes are allowed in spans, strip or hash sensitive fields in the OpenTelemetry Collector before they leave your network, and add a check in code review for new span attributes.
Then run one dry-run redaction query against last month's traces for the field you are most worried about. If the count comes back as zero, good. If it doesn't, you have just found an incident before an auditor did.
