Skip to main content
Sampling lets you collect a representative subset of traces instead of every single span. This reduces the volume of data sent to your observability platform — cutting storage costs while keeping enough data to find problems. Go to Sampling in the sidebar.

​
Sampling modes

​
Probabilistic sampling

A fixed percentage of all incoming traces is kept. Simple and low-overhead.
  • Global rate — percentage of all traces to keep (e.g. 10 = keep 10%)
  • All decisions are made at ingestion time, before the full trace is assembled
Best for: High-volume services where you want a consistent cost reduction across the board.

​
Tail-based sampling

Sampling decisions are made after the full trace is assembled, so you can make smarter decisions based on outcome.
  • Global rate — percentage of normal (error-free, latency-normal) traces to keep
  • Error traces — always kept (100%)
  • Slow traces (above latency threshold) — always kept (100%)
  • Service overrides — set a different rate for specific services
Best for: Mixed workloads where you need 100% of error and slow traces but can drop healthy fast traces.

​
Enabling sampling

  1. Go to Sampling.
  2. Toggle Enable sampling on.
  3. Choose Probabilistic or Tail-based.
  4. Set the global rate.
  5. Click Save. Obsy syncs the config to your OTel collector within seconds via a Helm upgrade.

​
Filter rules

In addition to sampling, you can completely drop certain traffic from being collected at all. These rules run before sampling.

​
Drop by endpoint

Drop all spans for specific HTTP paths. Useful for noisy health check endpoints. 
/health
/healthz
/readyz
/livez
/ping
/metrics
Obsy always drops /health, /healthz, /readyz, /livez, /ping, and /metrics by default regardless of your custom drop rules. This reduces noise from k8s liveness probes.
To add custom endpoints to drop:
  1. Go to Sampling → Filter rules.
  2. Toggle Custom drop rules on.
  3. Enter endpoint paths (one per line, supports prefix matching).

​
Drop by HTTP method

Drop all spans for specific HTTP methods. Example: drop OPTIONS (CORS preflight) and HEAD requests.

​
Drop by user agent

Drop spans from specific user agents. Useful for excluding internal monitoring bots, load balancer health checks, or synthetic monitors you don’t want appearing in traces.

​
Service-level overrides (tail-based only)

When using tail-based sampling, you can set a different sampling rate for individual services:
  1. Go to Sampling → Service overrides.
  2. Click Add override.
  3. Select the service from your catalog and enter a rate (0–100%).
  4. Click Save.
Example: set payment-service to 100% (keep all) while everything else runs at 10%.

​
Cost impact

The Sampling page shows an estimated cost impact card based on your current ingestion volume and the configured rate. This is an estimate — actual savings depend on trace size distribution.

​
How changes are applied

Sampling config changes trigger an automatic helm upgrade on all clusters. The upgrade is non-blocking (run with --no-wait) so changes appear in the UI immediately. New sampling settings take effect when the collector pods finish their rolling restart, typically within 30–60 seconds.