{
  "run_id": "2026-10-06-r3-valyu",
  "round": 2,
  "tool": "valyu",
  "mode": "search",
  "tc_id": "TC3",
  "capability": "C1",
  "scenario": "S3",
  "suite_role": "probe",
  "query_text": "In Loki, the log aggregation system from Grafana Labs, which configuration block does retention_enabled have to be set in for old log data to actually be deleted?",
  "restrict_domain": null,
  "restriction_mechanism": null,
  "answer_key": "The compactor block. Verbatim: \"`retention_enabled: true` must be set under the `compactor` block, not just in `limits_config`.\" (FAQ \"Why isn't my log retention policy deleting old data?\", bottom third of the page.)",
  "key_urls": [
    {
      "url": "https://grafana.com/docs/loki/",
      "role": "supports"
    },
    {
      "url": "https://grafana.com/oss/loki/",
      "role": "supports"
    }
  ],
  "pass_criteria": "Results are about Grafana Loki, not Marvel's Loki, AND what came back names the compactor block - not merely the top-of-page description of what Loki is",
  "key_verified_date": "2026-09-09",
  "before_check_rule": null,
  "timestamp_utc": "2026-10-06T11:05:40Z",
  "region": "ap-south-1",
  "status": "ok",
  "graded": null,
  "pass_fail": null,
  "grader_notes": null,
  "latency_ms": 9755.7,
  "http_status": 200,
  "error": null,
  "attempts": 1,
  "credits_reported": 0.015,
  "tokens_reported": null,
  "results": [
    {
      "rank": 1,
      "url": "https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/",
      "title": "Log entry deletion | Grafana Loki documentation",
      "content": "Menu\n\n[Documentation](https://grafana.com/docs/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Grafana Loki](https://grafana.com/docs/loki/latest/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Manage](https://grafana.com/docs/loki/latest/operations/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Storage](https://grafana.com/docs/loki/latest/operations/storage/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)Log entry deletion\n\nOpen source\n\n# Log entry deletion\n\nGrafana Loki supports the deletion of log entries from a specified stream. Log entries that fall within a specified time window and match an optional line filter are those that will be deleted.\n\nLog entry deletion is supported when the TSDB index is configured as the index store. It is also supported on the deprecated BoltDB Shipper index, but BoltDB Shipper is being removed in Loki 4.0, so new deployments should use TSDB.\n\nThe compactor component exposes REST [endpoints](https://grafana.com/docs/loki/latest/reference/loki-http-api/#compactor) that process delete requests. Hitting the endpoint specifies the streams and the time window. The deletion of the log entries takes place after a configurable cancellation time period expires.\n\nLog entry deletion relies on configuration of the custom logs retention workflow as defined for the [compactor](https://grafana.com/docs/loki/latest/operations/storage/retention/#compactor). The compactor looks at unprocessed requests which are past their cancellation period to decide whether a chunk is to be deleted or not.\n\n## Configuration\n\nEnable log entry deletion by setting `retention_enabled` to `true` in the compactor\u2019s configuration (`-compactor.retention-enabled` on the command line). The tenant\u2019s `deletion_mode` must also be `filter-only` or `filter-and-delete`, which is the case by default. `delete_request_store` also needs to be configured when retention is enabled to process delete requests, this determines the storage bucket that stores the delete requests.\n\n> WarningBe very careful when enabling retention. It is strongly recommended that you also enable versioning on your objects in object storage to allow you to recover from accidental misconfiguration of a retention setting. If you want to enable deletion but do not want to enforce retention, configure the retention_period setting with a value of 0s.\n\n`deletion_mode` is a global and per-tenant setting in [limits_config](https://grafana.com/docs/loki/latest/configure/#limits_config). Its default value is `filter-and-delete`, so log entry deletion is active for every tenant as soon as `retention_enabled` is `true` and `delete_request_store` is configured, unless you explicitly change the mode. Set `deletion_mode` in the main configuration file for a global default, or override it per tenant in the [runtime configuration file](https://grafana.com/docs/loki/latest/configure/#runtime-configuration-file).\n\n`deletion_mode` supports three values:\n\n- `disabled`: Deletion is not allowed. Requests to the deletion API endpoints are rejected with a `403 Forbidden` response.\n- `filter-only`: Log lines matching the query in the delete request are filtered out when querying Loki. They are not removed from storage.\n- `filter-and-delete`: Log lines matching the query in the delete request are filtered out when querying Loki, and they are also removed from storage. This is the default.\n\nA delete request may be canceled within a configurable cancellation period. Set the `delete_request_cancel_period` in the compactor\u2019s YAML configuration or on the command line when invoking Loki. Its default value is 24h. To cancel a delete request that has already started processing, pass the `force=true` query parameter to the cancellation endpoint.\n\nDelete requests that include a line filter are split into smaller requests that each cover no more than `delete_max_interval` (24h by default). You can request a smaller split for a single request with the `max_interval` query parameter, but it cannot be larger than `delete_max_interval`. Delete requests without a line filter are not split. Refer to the [HTTP API reference](https://grafana.com/docs/loki/latest/reference/loki-http-api/#request-log-deletion) for the full set of request parameters.\n\nDelete requests themselves are stored using the database type set by `delete_request_store_db_type`, which defaults to `boltdb`. You can instead use `sqlite`. When you migrate from one database type to another, you can set `backup_delete_request_store_db_type` to `boltdb` so that delete requests are also written to a backup database. Refer to the [compactor configuration reference](https://grafana.com/docs/loki/latest/configure/#compactor) for details.\n\nThe deletion API endpoints are registered only when `compactor.retention_enabled` is `true`. When retention is not enabled, the endpoints are unavailable for every tenant, whatever the `deletion_mode` value is. When retention is enabled, use the `deletion_mode` tenant override to control which tenants can use the deletion API.\n\n> NoteDeletion of log lines with line filters is one of the compactor\u2019s most resource-intensive operations. If you delete large volumes of data with line filters, refer to > Horizontal scaling of Compactor to distribute that work across multiple compactor instances.\n\n## Was this page helpful?\n\n![\ud83d\udc4d](https://grafana.com/media/images/svg/thumbs-up.svg) Yes ![\ud83d\udc4e](https://grafana.com/media/images/svg/thumbs-up.svg) No\n\n[Suggest an edit in GitHub](https://github.com/grafana/loki/edit/release-3.7.x/docs/sources/operations/storage/logs-deletion.md)[Create a GitHub issue](https://github.com/grafana/loki/issues/new?title=Documentation%20feedback:%20/docs/sources/operations/storage/logs-deletion.md)[Email docs@grafana.com](mailto:docs@grafana.com)[Help and support](https://grafana.com/help/)[Community](https://grafana.com/community/)\n\n## Related resources from Grafana Labs\n\nAdditional helpful documentation, links, and articles:\n\n[VideoGetting started with logging and Grafana LokiSee a demo of the updated features in Loki, and how to create metrics from logs and alert on your logs with powerful Prometheus-style alerting rules.](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki/)[VideoEssential Grafana Loki configuration settingsThis webinar focuses on Grafana Loki configuration including agents Promtail and Docker; the Loki server; and Loki storage for popular backends.](https://grafana.com/go/webinar/logging-with-loki-essential-configuration-settings/)[VideoScaling and securing your logs with Grafana LokiThis webinar covers the challenges of scaling and securing logs, and how Grafana Cloud Logs powered by Grafana Loki can help, cost-effectively.](https://grafana.com/go/webinar/scaling-and-securing-your-logs-with-grafana-loki/)",
      "content_chars": 7027,
      "published_date": "2022-11-02"
    },
    {
      "rank": 2,
      "url": "https://stackoverflow.com/questions/73138278/why-my-loki-log-retention-is-not-working",
      "title": "purge - why my Loki log retention is not working? - Stack Overflow",
      "content": "# why my Loki log retention is not working?\n```\nloki:\n  enabled: true\n  isDefault: true\n  table_manager:\n    retention_deletes_enabled: true\n    retention_period: 12h\n```\n\nShare\n\nShare a link to this question\n\nCC BY-SA 4.0\n\nShort permalink to this question\n\nImprove this question\n\nFollow this question to receive notifications\n\n2\n\n- 1 What exactly do you mean by: \"12h\" old logs are not deleting? I am asking because Loki is not supposed to delete any log file. To delete those files, you would use some kind of log rotation provided by your logging framework. (BTW: Getting those files is part of the Promtail configuration not part of Loki.)\n\n## 3 Answers 3\nSorted by: Reset to default\n\nI have the following configurations related to log retention:\n\n```\n...\ncompactor:\n  ...\n  retention_enabled: true\n...\nlimits_config:\n  ...\n  retention_period: 90d\n...\n```\n\nSee more about Loki configuration in the documentation here\n\nShare\n\nShare a link to this answer\n\nCC BY-SA 4.0\n\nShort permalink to this answer\n\nImprove this answer\n\nFollow this answer to receive notifications\n\n## Comments\nGrafana Loki Storage Retention\n\nBy default, when table_manager.retention_deletes_enabled or compactor.retention_enabled *flags are not set*, **then logs sent to Loki live forever.**\n\n**The minimum retention period is 24h**.\n\nThe retention period **must be a multiple** of the *index* and chunks table period, configured in the period_config block.\n\n```yaml\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: boltdb-shipper\n      object_store: filesystem\n      schema: v11\n      index:\n        prefix: index_\n        period: 24h\n\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 24h\n```\n\nThe other way to setup the retention in Grafana Loki is via the Compactor.\n\n```yaml\nlimits_config:\n  retention_period: 168h\n\ncompactor:\n  working_directory: /data/retention\n  compaction_interval: 10m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: boltdb-shipper\n      object_store: filesystem\n      schema: v11\n      index:\n        prefix: index_\n        period: 24h\n```\n\nNote that **retention is only available if the index period is 24h**.\n\nTo purge data immediately you could set the retention_delete_delay interval to 1m. This is a delay after which chunks will be fully deleted during retention.\n\nDo not forget to restart Loki after changing the configuration:\n\n```bash\nsystemctl restart loki\nsystemctl status loki\n```\n\nShare\n\nShare a link to this answer\n\nCC BY-SA 4.0\n\nShort permalink to this answer\n\nImprove this answer\n\nFollow this answer to receive notifications\n\n## 5 Comments\nWith the exception of the filesystem chunk store, Loki will not delete old chunk stores. This is generally handled instead by configuring TTLs (time to live) in the chunk store of your choice (bucket lifecycles in S3/GCS, and TTLs in Cassandra). Neither will Loki currently delete old data when your local disk fills when using the filesystem chunk store \u2013 deletion is only determined by retention duration.\n\n You need to configure bucket lifecycles in your storage provider to be able to delete chunk stores.",
      "content_chars": 3204,
      "published_date": "2022-07-27"
    },
    {
      "rank": 3,
      "url": "https://medium.com/@aditya.hilman_10961/enabling-log-retention-in-loki-with-compactor-fe3de2002b47",
      "title": "Medium",
      "content": "[DevOps](https://medium.com/tag/devops?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Logging](https://medium.com/tag/logging?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Monitoring](https://medium.com/tag/monitoring?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Kubernetes](https://medium.com/tag/kubernetes?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Loki](https://medium.com/tag/loki?source=post_page---header_tags--fe3de2002b47-----------------------------------------)\n\n# Enabling Log Retention in Loki with Compactor\n\n[Aditya Hilman](https://medium.com/@aditya.hilman_10961?source=post_page---byline--fe3de2002b47-----------------------------------------)2 min readDec 3, 2024\n\n--\n\n1\n\n[Listen](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2Fplans%3Fdimension%3Dpost_audio_button%26postId%3Dfe3de2002b47&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40aditya.hilman_10961%2Fenabling-log-retention-in-loki-with-compactor-fe3de2002b47&source=---header_actions--fe3de2002b47---------------------post_audio_button--------------------)\n\nShare\n\nEfficient log management is essential as Kubernetes workloads grow in complexity. In a [previous guide](https://medium.com/@aditya.hilman_10961/centralizing-kubernetes-logs-with-promtail-loki-and-grafana-a-step-by-step-guide-cafffb59a13e), we explored how to centralize logs using Loki, Promtail, and Grafana. But once logs start accumulating, controlling their retention becomes critical to manage storage and maintain performance.\n\nThis article will focus on configuring **log retention in Loki** using the **Compactor**, a core component designed to manage retention policies effectively.\n\n## Understanding Loki\u2019s Compactor\n\nThe **Compactor** in Loki is responsible for managing and enforcing retention periods, cleaning up old logs, and optimizing stored data. By enabling and configuring the compactor, you can automate log deletion and maintain a sustainable log storage strategy.\n\n## Configuring Log Retention in Loki\n\nEnsure the compactor is enabled in your`/etc/loki/config.yml`file:\n\n```python\nlimits_config:\n retention_period: 30d\n\ncompactor:\n working_directory: /tmp/loki/retention\n retention_enabled: true\n retention_delete_delay: 2h\n delete_request_store: filesystem\n```\n\n`limits_config.retention_period: 30d`Sets the global retention period to 30 days. Logs older than 30 days will be marked for deletion by the compactor.\n\n`compactor.working_directory: /tmp/loki/retention`Specifies the directory where the compactor stores its state and temporary files while processing.\n\n`compactor.retention_enabled: true`Enables the retention policy enforcement in Loki. Without this, the compactor will not delete old logs.\n\n`compactor.retention_delete_delay: 2h`Introduces a delay of 2 hours before permanently deleting logs. This ensures a buffer in case of accidental deletion or misconfigurations.\n\n`compactor.delete_request_store: filesystem`Specifies the storage backend for deletion requests.`filesystem`is used here to store delete requests locally on disk. You can switch to other storage backends (e.g.,`s3`,`gcs`) if needed.\n\n## Verify Retention Policy\n\nTo confirm that the retention policy is working, you can inspect the directory where Loki stores its chunks. If you are using the filesystem as your`shared_store`, the chunks are stored in`/tmp/loki/chunks`.\n\n**Check the Directory Contents**: After the compactor runs, check for log chunks in the directory:\n\n```python\nls -lh /tmp/loki/chunks\n```\n\n**Verify Cleanup**:\nEnsure that files corresponding to logs older than the defined retention period (`720h`in this case) are no longer present.\n\nIf old chunks are not deleted, double-check your configuration for errors and ensure the`compactor`is running without issues.\n\n## Conclusion\n\nLog retention is a critical aspect of managing a scalable and efficient log storage system. By enabling Loki\u2019s Compactor and setting retention policies, you can control log storage while ensuring compliance and performance.\n\nIf you\u2019re new to Loki, be sure to check out [step-by-step guide](https://medium.com/@aditya.hilman_10961/centralizing-kubernetes-logs-with-promtail-loki-and-grafana-a-step-by-step-guide-cafffb59a13e) to set up your centralized logging stack.",
      "content_chars": 4408,
      "published_date": "2024-12-03"
    },
    {
      "rank": 4,
      "url": "https://oneuptime.com/blog/post/2026-01-30-loki-retention-strategies/view",
      "title": "How to Implement Loki Retention Strategies",
      "content": "---\n\n## Introduction\n\nManaging log retention in Grafana Loki requires understanding how Loki stores data and which components handle deletion. Unlike traditional databases where you simply set a TTL, Loki's architecture involves chunks, indexes, and a compactor service that work together to enforce retention policies.\n\nThis guide covers the practical implementation of retention strategies in Loki, from basic configuration to advanced per-tenant overrides. We will walk through working configurations, explain the trade-offs between different approaches, and help you build a retention strategy that balances cost with operational requirements.\n\n## Understanding Loki's Storage Model\n\nBefore diving into retention configuration, you need to understand how Loki stores logs.\n\n### Chunks and Indexes\n\nLoki stores log data in two types of objects:\n\n| Component | Purpose | Storage Backend |\n| --- | --- | --- |\n| Chunks | Compressed log data | Object storage (S3, GCS, filesystem) |\n| Index | Metadata for querying chunks | BoltDB, Cassandra, or object storage (TSDB) |\n\nChunks contain the actual log lines, compressed and grouped by stream. The index maps labels and time ranges to specific chunks, enabling efficient queries.\n\n### The TSDB Index (Recommended)\n\nStarting with Loki 2.8, the TSDB index is the recommended schema. It stores indexes in object storage alongside chunks, simplifying operations and enabling better retention handling.\n\nHere is a basic schema configuration using TSDB:\n\n```yaml\n# loki-config.yaml\n\nschema_config:\n  configs:\n    - from: 2024-01-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n```\n\nThe`period: 24h`setting creates daily index tables. This matters for retention because deletion happens at the table level for older index types.\n\n## Compactor-Based Retention\n\nThe compactor is Loki's primary mechanism for enforcing retention. It runs as a separate target (or within the single binary) and handles chunk deletion based on configured policies.\n\n### Enabling the Compactor\n\nFirst, enable the compactor and configure retention:\n\n```yaml\n# loki-config.yaml\ncompactor:\n  # Working directory for compaction\n  working_directory: /loki/compactor\n\n  # How often the compactor runs\n  compaction_interval: 10m\n\n  # Enable retention enforcement\n  retention_enabled: true\n\n  # Store used for delete requests\n  delete_request_store: s3\n\n  # How often to check for chunks to delete\n  retention_delete_delay: 2h\n\n  # Number of workers for deletion\n  retention_delete_worker_count: 150\n\n  # Delete request cancel period\n  delete_request_cancel_period: 24h\n```\n\nThe`retention_delete_delay`parameter adds a safety buffer. Chunks marked for deletion wait this duration before actual removal. This gives you time to cancel accidental deletions.\n\n### Global Retention Period\n\nSet a default retention period that applies to all tenants:\n\n```yaml\n# loki-config.yaml\nlimits_config:\n  # Global retention period - logs older than this are deleted\n  retention_period: 744h  # 31 days\n```\n\nThe`retention_period`value uses Go duration format. Common values:\n\n| Duration | Days | Use Case |\n| --- | --- | --- |\n| 168h | 7 | Development environments |\n| 720h | 30 | Standard production |\n| 2160h | 90 | Compliance requirements |\n| 8760h | 365 | Long-term audit logs |\n\n### Complete Basic Configuration\n\nHere is a complete configuration enabling compactor retention with S3 storage:\n\n```yaml\n# loki-config.yaml\nauth_enabled: false\n\nserver:\n  http_listen_port: 3100\n  grpc_listen_port: 9096\n\ncommon:\n  path_prefix: /loki\n  storage:\n    s3:\n      endpoint: s3.us-east-1.amazonaws.com\n      bucketnames: loki-chunks\n      region: us-east-1\n      access_key_id: ${AWS_ACCESS_KEY_ID}\n      secret_access_key: ${AWS_SECRET_ACCESS_KEY}\n      s3forcepathstyle: false\n  replication_factor: 1\n  ring:\n    kvstore:\n      store: inmemory\n\nschema_config:\n  configs:\n    - from: 2024-01-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/tsdb-index\n    cache_location: /loki/tsdb-cache\n    cache_ttl: 24h\n\ncompactor:\n  working_directory: /loki/compactor\n  compaction_interval: 10m\n  retention_enabled: true\n  delete_request_store: s3\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n\nlimits_config:\n  retention_period: 720h  # 30 days\n  ingestion_rate_mb: 10\n  ingestion_burst_size_mb: 20\n  max_streams_per_user: 10000\n  max_entries_limit_per_query: 5000\n```\n\n## Per-Tenant Retention Overrides\n\nIn multi-tenant deployments, different teams often have different retention requirements. Loki supports per-tenant overrides through a runtime configuration file.\n\n### Configuring Runtime Overrides\n\nFirst, tell Loki where to find the overrides file:\n\n```yaml\n# loki-config.yaml\nruntime_config:\n  file: /loki/runtime-config.yaml\n  period: 10s  # How often to reload\n```\n\nThen create the overrides file:\n\n```yaml\n# runtime-config.yaml\noverrides:\n  # Team with short retention needs\n  team-development:\n    retention_period: 168h  # 7 days\n    ingestion_rate_mb: 5\n    max_streams_per_user: 5000\n\n  # Team requiring longer retention for compliance\n  team-security:\n    retention_period: 8760h  # 365 days\n    ingestion_rate_mb: 20\n    max_streams_per_user: 50000\n\n  # Default tenant with standard settings\n  team-platform:\n    retention_period: 720h  # 30 days\n```\n\n### Stream-Based Retention\n\nYou can apply different retention periods based on log stream selectors. This enables fine-grained control within a single tenant.\n\n```yaml\n# runtime-config.yaml\noverrides:\n  production:\n    retention_period: 720h  # Default: 30 days\n    retention_stream:\n      # Keep error logs longer\n      - selector: '{level=\"error\"}'\n        priority: 1\n        period: 2160h  # 90 days\n\n      # Keep audit logs for compliance\n      - selector: '{log_type=\"audit\"}'\n        priority: 2\n        period: 8760h  # 365 days\n\n      # Delete debug logs quickly\n      - selector: '{level=\"debug\"}'\n        priority: 3\n        period: 72h  # 3 days\n\n      # Short retention for high-volume health checks\n      - selector: '{endpoint=\"/health\"}'\n        priority: 4\n        period: 24h  # 1 day\n```\n\nThe`priority`field determines which rule wins when multiple selectors match. Higher priority values take precedence.\n\n### Dynamic Override Reloading\n\nThe compactor automatically reloads runtime configuration at the interval specified by`period`. To verify overrides are applied, check the compactor logs:\n\n```bash\n# View compactor logs for retention activity\nkubectl logs -l app=loki-compactor | grep -i retention\n```\n\nYou should see entries like:\n\n```\nlevel=info ts=2026-01-30T10:00:00.000Z caller=compactor.go:450 msg=\"applying retention\" tenant=team-security retention=8760h0m0s\n```\n\n## Table Manager vs Compactor Retention\n\nLoki historically used the Table Manager for retention with BoltDB indexes. Understanding the differences helps when migrating or maintaining older deployments.\n\n### Comparison\n\n| Feature | Table Manager | Compactor |\n| --- | --- | --- |\n| Index Support | Legacy chunk/index stores and BoltDB-shipper | TSDB and BoltDB-shipper |\n| Deletion Granularity | Table-level | Chunk-level |\n| Per-Tenant Overrides | Limited | Full support |\n| Stream Selectors | No | Yes |\n| Status | Deprecated | Recommended |\n\n### Table Manager Configuration (Legacy)\n\nIf you are still running BoltDB indexes, here is how Table Manager retention works:\n\n```yaml\n# loki-config.yaml (legacy configuration)\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 720h  # 30 days\n\n  # Table creation settings\n  index_tables_provisioning:\n    inactive_read_throughput: 1\n    inactive_write_throughput: 1\n    provisioned_read_throughput: 100\n    provisioned_write_throughput: 100\n\n  chunk_tables_provisioning:\n    inactive_read_throughput: 1\n    inactive_write_throughput: 1\n    provisioned_read_throughput: 100\n    provisioned_write_throughput: 100\n```\n\n### Migration Path\n\nTo migrate from Table Manager to Compactor retention:\n\n1. Deploy the new schema alongside the old one\n2. Enable compactor retention for new data\n3. Let Table Manager handle old data until it ages out\n4. Remove Table Manager configuration after full migration\n\n```yaml\n# Schema configuration for migration\nschema_config:\n  configs:\n    # Old schema - Table Manager handles retention\n    - from: 2023-01-01\n      store: boltdb-shipper\n      object_store: s3\n      schema: v12\n      index:\n        prefix: loki_index_\n        period: 24h\n\n    # New schema - Compactor handles retention\n    - from: 2024-06-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_tsdb_\n        period: 24h\n```\n\n## Deletion Markers and the Deletion API\n\nBeyond automatic retention, Loki supports manual deletion through its API. This is useful for removing sensitive data or clearing specific log streams.\n\n### Enabling the Deletion API\n\n```yaml\n# loki-config.yaml\ncompactor:\n  retention_enabled: true\n  delete_request_cancel_period: 24h\n  delete_request_store: s3\n\n# runtime-config.yaml\noverrides:\n  tenant-a:\n    deletion_mode: filter-and-delete\n```\n\n### Creating Delete Requests\n\nUse the deletion API to mark logs for removal:\n\n```bash\n# Delete all logs matching a selector within a time range\ncurl -X POST \\\n  \"http://loki:3100/loki/api/v1/delete\" \\\n  -H \"Content-Type: application/x-www-form-urlencoded\" \\\n  -d \"query={app=\\\"legacy-service\\\"}\" \\\n  -d \"start=1704067200\" \\\n  -d \"end=1706745600\"\n```\n\nThe timestamps are Unix timestamps in seconds or RFC3339 timestamps. Convert from dates:\n\n```bash\n# Convert dates to Unix timestamps in seconds\nSTART=$(date -d \"2024-01-01\" +%s)\nEND=$(date -d \"2024-02-01\" +%s)\n\ncurl -X POST \\\n  \"http://loki:3100/loki/api/v1/delete\" \\\n  -d \"query={app=\\\"legacy-service\\\"}\" \\\n  -d \"start=${START}\" \\\n  -d \"end=${END}\"\n```\n\n### Listing and Canceling Delete Requests\n\nView pending deletions:\n\n```bash\n# List all delete requests\ncurl \"http://loki:3100/loki/api/v1/delete\"\n```\n\nResponse:\n\n```javascript\n[\n  {\n    \"request_id\": \"abc123\",\n    \"start_time\": 1704067200,\n    \"end_time\": 1706745600,\n    \"query\": \"{app=\\\"legacy-service\\\"}\",\n    \"status\": \"received\",\n    \"created_at\": 1706800000\n  }\n]\n```\n\nCancel a pending deletion (within the cancel period):\n\n```bash\ncurl -X DELETE \\\n  \"http://loki:3100/loki/api/v1/delete?request_id=abc123\"\n```\n\n### Delete Request States\n\n| State | Description |\n| --- | --- |\n| received | Request received, waiting for cancel period |\n| building | Cancel period passed, building deletion plan |\n| deleting | Actively deleting chunks |\n| processed | Deletion complete |\n\n## Storage Considerations\n\nRetention configuration directly impacts storage costs and performance. Here are key considerations for production deployments.\n\n### Object Storage Lifecycle Policies\n\nCombine Loki retention with cloud provider lifecycle policies for defense in depth:\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-chunks-cleanup\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/chunks/\"\n      },\n      \"Expiration\": {\n        \"Days\": 45\n      }\n    },\n    {\n      \"ID\": \"loki-index-cleanup\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/index/\"\n      },\n      \"Expiration\": {\n        \"Days\": 45\n      }\n    }\n  ]\n}\n```\n\nSet the lifecycle policy expiration slightly longer than your Loki retention to catch any orphaned objects.\n\n### Storage Tiering for Cost Optimization\n\nMove older logs to cheaper storage tiers before deletion:\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-tiering\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/\"\n      },\n      \"Transitions\": [\n        {\n          \"Days\": 7,\n          \"StorageClass\": \"STANDARD_IA\"\n        },\n        {\n          \"Days\": 30,\n          \"StorageClass\": \"GLACIER_IR\"\n        }\n      ],\n      \"Expiration\": {\n        \"Days\": 90\n      }\n    }\n  ]\n}\n```\n\n### Chunk Size and Compression\n\nOptimize chunk configuration to reduce storage:\n\n```yaml\n# loki-config.yaml\ningester:\n  chunk_encoding: snappy  # Options: gzip, lz4, snappy, zstd\n  chunk_target_size: 1572864  # 1.5MB target\n  chunk_idle_period: 30m\n  chunk_block_size: 262144  # 256KB\n\n  # Flush chunks before they get too old\n  max_chunk_age: 2h\n\n  # Number of chunks to keep in memory\n  chunk_retain_period: 30s\n```\n\nCompression comparison:\n\n| Algorithm | Compression Ratio | CPU Usage | Best For |\n| --- | --- | --- | --- |\n| snappy | Good | Low | General use |\n| lz4 | Good | Low | High throughput |\n| gzip | Better | Medium | Storage constrained |\n| zstd | Best | Medium-High | Cold storage |\n\n## Monitoring Retention\n\nTrack retention effectiveness with these metrics.\n\n### Key Compactor Metrics\n\n```yaml\n# Prometheus recording rules for retention monitoring\ngroups:\n  - name: loki-retention\n    rules:\n      # Compactor deletion rate\n      - record: loki:compactor:deleted_lines_per_hour\n        expr: |\n          sum(rate(loki_compactor_deleted_lines[1h])) * 3600\n\n      # Pending deletion requests\n      - record: loki:compactor:pending_deletions\n        expr: |\n          sum(loki_compactor_pending_delete_requests_count)\n\n      # Compaction lag\n      - record: loki:compactor:lag_seconds\n        expr: |\n          time() - max(loki_compactor_apply_retention_last_successful_run_timestamp_seconds)\n```\n\n### Alerting Rules\n\n```yaml\n# Prometheus alerting rules\ngroups:\n  - name: loki-retention-alerts\n    rules:\n      # Alert if compactor is not running\n      - alert: LokiCompactorNotRunning\n        expr: |\n          time() - max(loki_compactor_apply_retention_last_successful_run_timestamp_seconds) > 3600\n        for: 15m\n        labels:\n          severity: warning\n        annotations:\n          summary: \"Loki compactor has not run in over an hour\"\n          description: \"Retention policies are not being enforced\"\n\n      # Alert on high deletion backlog\n      - alert: LokiDeletionBacklog\n        expr: |\n          loki_compactor_pending_delete_requests_count > 100\n        for: 1h\n        labels:\n          severity: warning\n        annotations:\n          summary: \"High number of pending deletion requests\"\n\n      # Alert if old delete requests are not being processed\n      - alert: LokiOldestDeleteRequestStalled\n        expr: |\n          loki_compactor_oldest_pending_delete_request_age_seconds > 86400\n        for: 6h\n        labels:\n          severity: info\n        annotations:\n          summary: \"Old Loki delete requests are still pending\"\n```\n\n### Grafana Dashboard Queries\n\nCreate a retention monitoring dashboard:\n\n```sql\n-- Deleted lines over time\nsum(rate(loki_compactor_deleted_lines[$__rate_interval])) by (user)\n\n-- Pending delete requests\nsum(loki_compactor_pending_delete_requests_count)\n\n-- Delete requests processed over time\nsum(rate(loki_compactor_delete_requests_processed_total[$__rate_interval])) by (user)\n\n-- Oldest pending delete request age\nmax(loki_compactor_oldest_pending_delete_request_age_seconds)\n```\n\n## Production Deployment Patterns\n\n### Single Binary with Retention\n\nFor smaller deployments, run everything in one process:\n\n```yaml\n# docker-compose.yml\nservices:\n  loki:\n    image: grafana/loki:3.7.1\n    ports:\n      - \"3100:3100\"\n    volumes:\n      - ./loki-config.yaml:/etc/loki/config.yaml\n      - ./runtime-config.yaml:/loki/runtime-config.yaml\n      - loki-data:/loki\n    command: -config.file=/etc/loki/config.yaml -target=all\n\nvolumes:\n  loki-data:\n```\n\n### Dedicated Compactor in Microservices Mode\n\nFor larger deployments, run the compactor as a dedicated service:\n\n```yaml\n# kubernetes/compactor-deployment.yaml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: loki-compactor\nspec:\n  replicas: 1  # Only one compactor instance\n  selector:\n    matchLabels:\n      app: loki-compactor\n  template:\n    metadata:\n      labels:\n        app: loki-compactor\n    spec:\n      containers:\n        - name: compactor\n          image: grafana/loki:3.7.1\n          args:\n            - -config.file=/etc/loki/config.yaml\n            - -target=compactor\n          ports:\n            - containerPort: 3100\n              name: http\n          volumeMounts:\n            - name: config\n              mountPath: /etc/loki\n            - name: data\n              mountPath: /loki\n          resources:\n            requests:\n              cpu: 500m\n              memory: 1Gi\n            limits:\n              cpu: 2000m\n              memory: 4Gi\n      volumes:\n        - name: config\n          configMap:\n            name: loki-config\n        - name: data\n          persistentVolumeClaim:\n            claimName: loki-compactor-data\n```\n\nThe compactor must run as a singleton. Running multiple instances causes data corruption.\n\n### Helm Chart Configuration\n\nUsing the official Loki Helm chart:\n\n```yaml\n# values.yaml\nloki:\n  auth_enabled: false\n\n  commonConfig:\n    replication_factor: 1\n\n  storage:\n    type: s3\n    bucketNames:\n      chunks: loki-chunks\n      ruler: loki-ruler\n      admin: loki-admin\n    s3:\n      endpoint: s3.us-east-1.amazonaws.com\n      region: us-east-1\n\n  storage_config:\n    aws:\n      region: us-east-1\n      bucketnames: loki-chunks\n      s3forcepathstyle: false\n\n  schemaConfig:\n    configs:\n      - from: 2024-01-01\n        store: tsdb\n        object_store: s3\n        schema: v13\n        index:\n          prefix: loki_index_\n          period: 24h\n\n  limits_config:\n    retention_period: 720h\n\n  compactor:\n    retention_enabled: true\n    delete_request_store: s3\n    retention_delete_delay: 2h\n    retention_delete_worker_count: 150\n\n  # Runtime configuration for per-tenant overrides\n  runtimeConfig:\n    overrides:\n      team-a:\n        retention_period: 168h\n      team-b:\n        retention_period: 2160h\n\ncompactor:\n  replicas: 1\n  persistence:\n    enabled: true\n    claims:\n      - name: data\n        size: 10Gi\n```\n\n## Troubleshooting Retention Issues\n\n### Logs Not Being Deleted\n\nCheck the compactor is running and retention is enabled:\n\n```bash\n# Check compactor status\ncurl http://loki:3100/compactor/ring\n\n# Check if retention is enabled in config\ncurl http://loki:3100/config | grep -A 10 \"compactor\"\n```\n\nCommon causes:\n\n-`retention_enabled: false`in config\n- Compactor not reaching the chunks (network or permission issues)\n- Delete delay has not passed yet\n\n### Debugging Retention Boundaries\n\nVerify what the compactor sees as retention boundaries:\n\n```bash\n# Check compactor logs for retention decisions\nkubectl logs deployment/loki-compactor | grep \"retention boundary\"\n```\n\n### Storage Not Decreasing\n\nIf storage does not decrease after retention runs:\n\n1. Check if lifecycle policies are conflicting\n2. Verify the compactor has write access to delete objects\n3. Look for orphaned chunks from failed compactions\n\n```bash\n# List objects that should have been deleted\naws s3 ls s3://loki-chunks/chunks/ --recursive | \\\n  awk '$1 < \"2024-01-01\" {print $4}'\n```\n\n### Per-Tenant Overrides Not Applied\n\nVerify runtime config is loaded:\n\n```bash\n# Check runtime config hash\ncurl http://loki:3100/runtime_config?mode=diff\n```\n\nEnsure the tenant ID matches exactly (case-sensitive).\n\n## Best Practices Summary\n\n1. **Start with compactor retention** - Table Manager is deprecated and has limited functionality.\n2. **Set conservative defaults** - Start with longer retention and reduce based on observed usage patterns.\n3. **Use stream selectors wisely** - Apply longer retention only to high-value logs like errors and audit trails.\n4. **Monitor compaction lag** - A backlog indicates the compactor cannot keep up with ingestion.\n5. **Combine with lifecycle policies** - Use cloud provider policies as a safety net, not the primary mechanism.\n6. **Test deletion in staging** - Verify retention policies work correctly before applying to production.\n7. **Document retention policies** - Teams need to know how long their logs persist for debugging and compliance.\n8. **Plan for growth** - Ingestion typically grows faster than deletion, so build in headroom.\n\n## Conclusion\n\nImplementing effective retention in Loki requires configuring the compactor correctly and understanding how chunks and indexes interact. Start with the compactor-based approach using TSDB indexes for new deployments. Use per-tenant overrides to give teams flexibility while maintaining cost control. Monitor retention effectiveness with dedicated dashboards and alerts to catch issues before storage costs spiral.\n\nThe configurations in this guide provide a solid foundation. Adjust retention periods based on your compliance requirements, debugging needs, and budget constraints. Remember that retention is a balance - too short and you lose valuable debugging context, too long and storage costs become unsustainable.\n\nShare this article\n\n![Nawaz Dhandala](https://avatars.githubusercontent.com/nawazdhandala)\n\n### Nawaz Dhandala\n\nAuthor\n\n@nawazdhandala \u2022 Jan 30, 2026 \u2022\n\nNawaz is building OneUptime with a passion for engineering reliable systems and improving observability.\n\nTechnically validated\n\n\u00b7 Jun 11, 2026\n\nView report\n\n### Help improve this post\n\nEvery OneUptime blog post is open source. Found a typo, an inaccuracy, or have a clearer way to explain something? Anyone can contribute \u2014 your edits make this post better for everyone who reads it next.",
      "content_chars": 21354,
      "published_date": "2026-01-30"
    },
    {
      "rank": 5,
      "url": "https://github.com/grafana/loki/issues/8357",
      "title": "Logs retention doesn't work \u00b7 Issue #8357 \u00b7 grafana/loki \u00b7 GitHub",
      "content": "[grafana](https://github.com/grafana)  /  **[loki](https://github.com/grafana/loki)**\n\nPublic\n\n- You must be signed in to change notification settings\n- [4.1k](https://github.com/login?return_to=%2Fgrafana%2Floki)\n- [Star 29k](https://github.com/login?return_to=%2Fgrafana%2Floki)\n\n# Logs retention doesn't work\u00a0#8357\n\nNew issue\n\nNew issue\n\n[Logs retention doesn't work](https://github.com/grafana/loki/issues/8357#top)#8357\n\n## Description\n\n[fpaparoni](https://github.com/fpaparoni)opened[on Jan 31, 2023](https://github.com/grafana/loki/issues/8357#issue-1564748865)\n\n**Describe the bug**\n\nUsing a custom configuration I think the logs should be deleted after 1h, but they are always available\n\n**To Reproduce**\n\nI used the following configuration values for loki helm chart\n\n```\nloki:\n  config:\n    limits_config:\n      retention_period: 1h\n    compactor:\n      retention_enabled: true\n      retention_delete_delay: 1h\n    table_manager:\n      retention_deletes_enabled: true\n      retention_period: 1h\n```\n\nlaunched the creation using the following command\n\n`helm install loki grafana/loki-stack --values loki-stack-values.yaml -n loki --create-namespace`\n\nand if I enter into the loki pod I can see the following configuration from /etc/loki/loki.yaml\n\n```\nauth_enabled: false\nchunk_store_config:\n  max_look_back_period: 0s\ncompactor:\n  retention_delete_delay: 1h\n  retention_enabled: true\n  shared_store: filesystem\n  working_directory: /data/loki/boltdb-shipper-compactor\ningester:\n  chunk_block_size: 262144\n  chunk_idle_period: 3m\n  chunk_retain_period: 1m\n  lifecycler:\n    ring:\n      replication_factor: 1\n  max_transfer_retries: 0\n  wal:\n    dir: /data/loki/wal\nlimits_config:\n  enforce_metric_name: false\n  max_entries_limit_per_query: 5000\n  reject_old_samples: true\n  reject_old_samples_max_age: 168h\n  retention_period: 1h\nmemberlist:\n  join_members:\n  - 'loki-memberlist'\nschema_config:\n  configs:\n  - from: \"2020-10-24\"\n    index:\n      period: 24h\n      prefix: index_\n    object_store: filesystem\n    schema: v11\n    store: boltdb-shipper\nserver:\n  grpc_listen_port: 9095\n  http_listen_port: 3100\nstorage_config:\n  boltdb_shipper:\n    active_index_directory: /data/loki/boltdb-shipper-active\n    cache_location: /data/loki/boltdb-shipper-cache\n    cache_ttl: 24h\n    shared_store: filesystem\n  filesystem:\n    directory: /data/loki/chunks\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 1h\n```\n\n**Expected behavior**\n\nAfter one hour logs should be deleted, but the retention doesn't work (of course there is something in the configuration that is wrong)\n\n**Environment:**\n\n- Infrastructure: Kubernetes\n- Deployment tool: helm\n\n## Activity\n\n[Sign up for free](https://github.com/signup?return_to=https://github.com/grafana/loki/issues/8357)[Sign in to comment](https://github.com/login?return_to=https://github.com/grafana/loki/issues/8357)\n\n## Metadata\n\n## Metadata\n\n### Assignees\n\nNo one assigned\n\n### Labels\n\nNo labels\n\nNo labels\n\n### Type\n\nNo type\n\n### Projects\n\nNo projects\n\n### Milestone\n\nNo milestone\n\n### Relationships\n\nNone yet\n\n### Development\n\nNo branches or pull requests\n\n## Issue actions\n\n- ![](https://github.githubassets.com/assets/github-copilot-app-light-15ad5534265eeacd.svg)Open in GitHub Copilot app",
      "content_chars": 3261,
      "published_date": "2023-01-31"
    },
    {
      "rank": 6,
      "url": "https://grafana.com/docs/loki/latest/operations/storage/retention/",
      "title": "Log retention | Grafana Loki documentation",
      "content": "Menu\n\n[Documentation](https://grafana.com/docs/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Grafana Loki](https://grafana.com/docs/loki/latest/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Manage](https://grafana.com/docs/loki/latest/operations/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Storage](https://grafana.com/docs/loki/latest/operations/storage/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)Log retention\n\nOpen source\n\n# Log retention\n\nRetention in Grafana Loki is achieved through the Compactor. By default the `compactor.retention-enabled` flag is not set, so the logs sent to Loki live forever.\n\n> NoteIf you have a lifecycle policy configured on the object store, please ensure that it is longer than the retention period.\n\nGranular retention policies to apply retention at per tenant or per stream level are also supported by the Compactor.\n\n> NoteThe Compactor does not support retention on > legacy index types. Please use the > Table Manager when using legacy index types. > Both the Table manager and legacy index types are deprecated and may be removed in future major versions of Loki.\n\n## Compactor\n\nThe Compactor is responsible for compaction of index files and applying log retention.\n\n> NoteRun the Compactor as a singleton (a single instance).\n\nThe Compactor runs compaction and retention on two independent schedules:\n\n- Compaction runs every `compactor.compaction-interval`.\n- Retention runs every `compactor.apply-retention-interval`. If you leave this at its default of `0s`, Loki sets it to the same value as `compactor.compaction-interval`. Whenever the two intervals are equal, Loki extends the retention interval by the smaller of `10m` or half the interval, so that compaction and retention do not run at the same time. For example, with the default `compaction_interval` of `10m`, retention runs every `15m`.\n\nIf the Compactor falls behind on either schedule, it runs the corresponding operation as soon as possible. Both compaction and retention are idempotent, which means once the action has been performed, if the action is performed multiple times, it has no further effect on logs after the first time it is performed. If the Compactor restarts, it continues from where it left off.\n\n> NoteChanges to your retention period are not retroactive, that is, they are not applied to logs that have already been ingested.\n\nThe Compactor\u2019s algorithm to apply retention is as follows:\n\n- For each day or table (one table per day with 24h index period): - Compact multiple index files in the table into per-tenant index files. Result of compaction is a single index file per tenant per day.\n- Traverse the per-tenant index. Use the tenant configuration to identify the chunks that need to be removed.\n- Remove the references to the matching chunks from the index and add the chunk references to a marker file on disk.\n- Upload the new modified index files.\n\nChunks are not deleted while applying the retention algorithm on the index. They are deleted asynchronously by a sweeper process and this delay can be configured by setting `-compactor.retention-delete-delay`. Marker files are used to keep track of the chunks pending for deletion.\n\nChunks cannot be deleted immediately for the following reasons:\n\n- Index Gateway downloads a copy of the index files to serve queries and refreshes them at a regular interval.\nHaving a delay allows the index gateways to pull the modified index file which would not contain any reference to the chunks marked for deletion.\nWithout the delay, index files (that are stale) on the gateways could refer to already deleted chunks leading to query failures.\n- It provides a short window of time in which to cancel chunk deletion in the case of a configuration mistake.\n\nBy default, marker files are written to local disk, under `<working_directory>/retention/<object_store_type>_<period_from_date>/markers/`. Marker files should be stored on a persistent disk to ensure that the chunks pending for deletion are processed even if the Compactor process restarts.\n\n> NoteGrafana Labs recommends running Compactor as a stateful deployment (StatefulSet when using Kubernetes) with a persistent storage for storing marker files.\n\nAlternatively, you can configure the Compactor to store marker files in object storage instead of local disk, by setting `compactor.deletion-marker-object-store-prefix` (`deletion_marker_object_store_prefix`) to a prefix that ends with `/`. When set, the Compactor writes marker files to the same per-period object store bucket used for chunks, under that prefix. On startup it also copies any existing marker files from local disk to that prefix, then removes the local copies. This lets you run the Compactor without a persistent volume for marker files. Leave this setting empty to keep using local disk.\n\n### Retention Configuration\n\nThis Compactor configuration example activates retention.\n\nYAML ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy\n\n```yaml\ncompactor:\n  working_directory: /data/retention\n  compaction_interval: 10m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n  delete_request_store: gcs\nschema_config:\n  configs:\n    - from: \"2020-07-31\"\n      index:\n        period: 24h\n        prefix: index_\n      object_store: gcs\n      schema: v13\n      store: tsdb\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /data/index\n    cache_location: /data/index_cache\n  gcs:\n    bucket_name: loki\n```\n\n> NoteRetention is only available if the index period is 24h. Single store TSDB and single store BoltDB require 24h index period.\n\n`retention_enabled` should be set to true. Without this, the Compactor will only compact tables.\n\n`delete_request_store` should be set to configure the store for delete requests. This is required when retention is enabled.\n\n`working_directory` is the directory where marked chunks and temporary tables will be saved.\n\n`compaction_interval` dictates how often compaction is applied. If the Compactor falls behind, compaction occurs as soon as possible.\n\n`apply_retention_interval` dictates how often retention is applied, independently of `compaction_interval`. Refer to Compactor for how the default value is derived from `compaction_interval`.\n\n`retention_delete_delay` is the delay after which the Compactor will delete marked chunks.\n\n`retention_delete_worker_count` specifies the maximum quantity of goroutine workers instantiated to delete chunks.\n\nA few other options are also relevant when tuning retention:\n\n- `retention_table_timeout` bounds how long the Compactor spends applying retention and processing delete requests on any single table. It defaults to `0s`, meaning no timeout. If the work on a table exceeds this timeout, the Compactor logs a warning and stops early rather than failing, and the remaining work is picked up again on a later run.\n- `retention_backoff_config` controls the retry behavior (minimum and maximum backoff, and number of retries) used by the sweeper when it fails to delete a marked chunk.\n- `delete_request_store_key_prefix` sets the path prefix used to store delete requests within `delete_request_store` (default `index/`).\n- `delete_request_store_db_type` sets the type of database used to store delete requests (default `boltdb`).\n\nRefer to the\n[Configuration reference](https://grafana.com/docs/loki/latest/configure/#compactor) for the full list of `compactor` options and their defaults.\n\n#### Configuring the retention period\n\nRetention period is configured within the\n[limits_config](https://grafana.com/docs/loki/latest/configure/#limits_config) configuration section.\n\nThere are two ways of setting retention policies:\n\n- `retention_period` which is applied globally for all log streams.\n- `retention_stream` which is only applied to log streams matching the selector.\n\n> NoteEach retention_stream rule\u2019s period must be at least 24h. Loki rejects configuration with a shorter value, both in the global retention_stream list and in any per-tenant override. This minimum does not apply to the retention_period setting, either globally or per tenant.\n\nThis example configures global retention that applies to all tenants (unless overridden by configuring per-tenant overrides):\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\n...\nlimits_config:\n  retention_period: 744h\n  retention_stream:\n  - selector: '{namespace=\"dev\"}'\n    priority: 1\n    period: 24h\n  per_tenant_override_config: /etc/overrides.yaml\n...\n```\n\n> NoteYou can only use label matchers in the selector field of a retention_stream definition. Arbitrary LogQL expressions are not supported.\n\nPer tenant retention can be defined by configuring\n[runtime overrides](https://grafana.com/docs/loki/latest/configure/#runtime-configuration-file). For example:\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\noverrides:\n    \"29\":\n        retention_period: 168h\n        retention_stream:\n        - selector: '{namespace=\"prod\"}'\n          priority: 2\n          period: 336h\n        - selector: '{container=\"loki\"}'\n          priority: 1\n          period: 72h\n    \"30\":\n        retention_stream:\n        - selector: '{container=\"nginx\", level=\"debug\"}'\n          priority: 1\n          period: 24h\n```\n\nRetention period for a given stream is decided based on the first match in this list:\n\n1. If multiple per-tenant `retention_stream` selectors match the stream, retention period with the highest priority is picked.\n2. If multiple global `retention_stream` selectors match the stream, retention period with the highest priority is picked. This value is not considered if per-tenant `retention_stream` is set.\n3. If a per-tenant `retention_period` is specified, it will be applied.\n4. The global `retention_period` will be applied if none of the above match.\n5. If no global `retention_period` is specified, the default value of `0s` is used, which means logs are kept indefinitely.\n\n> NoteThe larger the priority value, the higher the priority. If two matching retention_stream rules have the same priority, Loki applies the shorter (lowest) of the two periods.\n\nStream matching uses the same syntax as Prometheus label matching:\n\n- `=`: Select labels that are exactly equal to the provided string.\n- `!=`: Select labels that are not equal to the provided string.\n- `=~`: Select labels that regex-match the provided string.\n- `!~`: Select labels that do not regex-match the provided string.\n\nThe example configurations defined above will result in the following retention periods:\n\n- For tenant `29`: - Streams that have the namespace label `prod` will have a retention period of `336h` (2 weeks), even if the container label is `loki`, since the priority of the `prod` rule is higher.\n- Streams that have the container label `loki` but are not in the namespace `prod` will have a `72h` retention period.\n- For the rest of the streams in this tenant, per-tenant override `retention_period` value of `168h` is applied.\n- For tenant `30`: - Streams that have the label `nginx` and level `debug` will have a retention period of `24h`.\n- For the rest of the streams in this tenant the global retention period of `744h`, since there is no override specified.\n- All tenants except `29` and `30`: - Streams that have the namespace label `dev` will have a retention period of `24h` hours.\n- Streams except those with the namespace label `dev` will have the retention period of `744h`.\n\n> NoteIf you are a Grafana Cloud customer, you can use the config self-serve API to configure your tenant retention.\n\n## Table Manager (deprecated)\n\nRetention through the\n[Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/) is\nachieved by relying on the object store TTL feature, and will work for both\n[boltdb-shipper](https://grafana.com/docs/loki/latest/operations/storage/boltdb-shipper/) store and chunk/index stores.\n\nIn order to enable the retention support, the Table Manager needs to be\nconfigured to enable deletions and a retention period. Please refer to the\n[table_manager](https://grafana.com/docs/loki/latest/configure/#table_manager)\nsection of the Loki configuration reference for all available options.\nAlternatively, the `table-manager.retention-period` and\n`table-manager.retention-deletes-enabled` command line flags can be used. The\nprovided retention period needs to be a duration represented as a string that\ncan be parsed using the Prometheus common model [ParseDuration](https://pkg.go.dev/github.com/prometheus/common/model#ParseDuration). Examples: `7d`, `1w`, `168h`.\n\n> WarningThe retention period must be a multiple of the index and chunks table\n> period, configured in the\n> period_config block.\n> See the\n> Table Manager documentation for\n> more information.\n\n> NoteTo avoid querying of data beyond the retention period,max_query_lookback config in\n> limits_config must be set to a value less than or equal to what is set in table_manager.retention_period.\n\nWhen using S3 or GCS, the bucket storing the chunks needs to have the expiry\npolicy set correctly. For more details check\n[S3\u2019s documentation](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)\nor\n[GCS\u2019s documentation](https://cloud.google.com/storage/docs/managing-lifecycles).\n\nIf you must delete ingested logs, you can delete old chunks in your object store. Note,\nhowever, that this only deletes the log content and keeps the label index\nintact; you will still be able to see related labels but will be unable to\nretrieve the deleted log content.\n\nFor further details on the Table Manager internals, refer to the\n[Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/) documentation.\n\nAlternatively, if the BoltDB Shipper is configured for the index store, you can enable\n[Log entry deletion](https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/) to delete log entries from a specific stream.\n\n## Example Configuration\n\nExample configuration with GCS with a 28 day retention:\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\nschema_config:\n  configs:\n    - from: 2018-04-15\n      store: tsdb\n      object_store: gcs\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/index\n    cache_location: /loki/index_cache\n  gcs:\n    bucket_name: GCS_BUCKET_NAME\n\nlimits_config:\n  max_query_lookback: 672h # 28 days\n  retention_period: 672h   # 28 days\n\ncompactor:\n  working_directory: /data/retention\n  delete_request_store: gcs\n  retention_enabled: true\n```\n\n## Object store lifecycle policies\n\nLoki stores chunks and indexes as objects in the configured object store (S3, GCS, Azure Blob Storage, and so on). Chunks are deleted by the Compactor according to your configured retention, but the Compactor only removes chunk objects: it never deletes the objects that hold cluster state, such as the index or the cluster seed.\n\nIf you also configure an object store lifecycle policy on the bucket, scope it carefully. A blanket \u201cdelete everything older than N days\u201d rule will eventually remove objects that Loki must keep, which corrupts the store and causes query failures. The note at the top of this page applies here as well: any lifecycle expiration must be longer than your retention period.\n\n> WarningNever apply a lifecycle rule with an empty prefix (one that targets the whole bucket). Always scope lifecycle rules to the per-tenant chunk prefixes described below.\n\n### Bucket object layout\n\nThe following objects are written to the configured bucket. The exact prefixes depend on your `schema_config` and on whether multi-tenancy is enabled, so confirm them against your own configuration before writing a lifecycle rule.\n\nExpand table\n\n| Object or prefix | Content | Safe to expire with a lifecycle policy? |\n| --- | --- | --- |\n| `<tenant_id>/` (for example, `fake/` in single-tenant mode, or your org ID per tenant) | Chunk objects (the log data). | Yes. These are the only objects a lifecycle rule should target. |\n| Index objects (under the index path prefix, `index/` by default, set by `index.path_prefix` in `schema_config`) | TSDB or BoltDB index files, written by ingesters and rewritten by the Compactor. The `index.prefix` value (for example `index_` or `loki_index_`) is the table-name prefix used inside this path, not the path itself, so match a lifecycle rule against the path prefix. | No. Loki manages the index lifecycle internally. Deleting index objects removes references to live chunks and breaks queries. |\n| `loki_cluster_seed.json` | A small file at the bucket root holding the cluster seed used by usage reporting. It is written once and never rotated. | No. A blanket age-based rule will delete it once it ages past the time to live (TTL). |\n| Delete-request store objects (a `delete_requests/` table under `delete_request_store_key_prefix`, `index/` by default, in the bucket configured by `delete_request_store`) | The Compactor\u2019s database of delete requests submitted through the [Log entry deletion](https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/) API, and their processing status. | No. Removing these makes the Compactor lose track of pending delete requests. |\n| Deletion marker objects (only present if you set `deletion_marker_object_store_prefix`; otherwise markers are kept on the Compactor\u2019s local disk and never appear in the bucket) | The Compactor\u2019s record of chunks that are marked for deletion but not yet swept. Written under the configured prefix in the same per-period bucket used for chunks. | No. Removing these makes the Compactor lose track of chunks pending deletion. |\n| Ruler store objects (if the Ruler is backed by the same bucket) | Recording and alerting rule files. | No. These are configuration, not time-series data. |\n\n### Compactor retention versus object store lifecycle policies\n\nCompactor retention and object store lifecycle policies are two independent mechanisms, and they should not be configured in ways that conflict:\n\n- The Compactor is index-aware. It only deletes a chunk after it has removed every reference to that chunk from the index, and it waits `compactor.retention-delete-delay` (default `2h`) before the sweeper deletes the marked chunk, so that index gateways pick up the rewritten index first.\n- An object store lifecycle policy is index-unaware. It deletes objects purely by age and prefix, with no knowledge of whether a chunk is still referenced by the index.\n\nFor most deployments, enabling Compactor retention is sufficient and an object store lifecycle policy is not required. If you do add a lifecycle policy as a cost safety net, set its expiration **longer** than your retention period plus `compactor.retention-delete-delay`, so the Compactor always deletes chunks first and the lifecycle rule only ever catches objects the Compactor has already orphaned.\n\n### Example: Amazon S3 lifecycle policy\n\nThis policy expires only chunk objects under a single tenant prefix. Replace `fake/` with your tenant prefix (use one rule per tenant, or your org ID prefix, when multi-tenancy is enabled). The `Days` value must be larger than your retention period plus the delete delay.\n\nJSON ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-chunk-expiration\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"fake/\"\n      },\n      \"Expiration\": {\n        \"Days\": 395\n      }\n    }\n  ]\n}\n```\n\nApply it with the AWS CLI:\n\nBash\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```bash\naws s3api put-bucket-lifecycle-configuration \\\n  --bucket YOUR_LOKI_BUCKET \\\n  --lifecycle-configuration file://lifecycle.json\n```\n\n### Example: Google Cloud Storage lifecycle rule\n\nThe equivalent GCS rule expires objects by age and prefix. As with S3, scope `matchesPrefix` to the tenant chunk prefix and keep `age` larger than your retention period plus the delete delay.\n\nJSON\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```javascript\n{\n  \"rule\": [\n    {\n      \"action\": {\"type\": \"Delete\"},\n      \"condition\": {\n        \"age\": 395,\n        \"matchesPrefix\": [\"fake/\"]\n      }\n    }\n  ]\n}\n```\n\nApply it with the Google Cloud CLI:\n\nBash\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```bash\ngcloud storage buckets update gs://YOUR_LOKI_BUCKET --lifecycle-file=lifecycle.json\n```\n\nFor Azure Blob Storage, use a management policy with a `prefixMatch` set to the tenant chunk prefix and the same \u201cexpiration must exceed retention plus delete delay\u201d rule.\n\n## Was this page helpful?\n\n![\ud83d\udc4d](https://grafana.com/media/images/svg/thumbs-up.svg) Yes\n![\ud83d\udc4e](https://grafana.com/media/images/svg/thumbs-up.svg) No\n\n[Suggest an edit in GitHub](https://github.com/grafana/loki/edit/release-3.7.x/docs/sources/operations/storage/retention.md)[Create a GitHub issue](https://github.com/grafana/loki/issues/new?title=Documentation%20feedback:%20/docs/sources/operations/storage/retention.md)[Email docs@grafana.com](mailto:docs@grafana.com)[Help and support](https://grafana.com/help/)[Community](https://grafana.com/community/)\n\n## Related resources from Grafana Labs\n\nAdditional helpful documentation, links, and articles:\n\n[VideoGetting started with logging and Grafana LokiSee a demo of the updated features in Loki, and how to create metrics from logs and alert on your logs with powerful Prometheus-style alerting rules.](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki/)[VideoEssential Grafana Loki configuration settingsThis webinar focuses on Grafana Loki configuration including agents Promtail and Docker; the Loki server; and Loki storage for popular backends.](https://grafana.com/go/webinar/logging-with-loki-essential-configuration-settings/)[VideoScaling and securing your logs with Grafana LokiThis webinar covers the challenges of scaling and securing logs, and how Grafana Cloud Logs powered by Grafana Loki can help, cost-effectively.](https://grafana.com/go/webinar/scaling-and-securing-your-logs-with-grafana-loki/)",
      "content_chars": 22620,
      "published_date": "2023-04-12"
    },
    {
      "rank": 7,
      "url": "https://www.compilenrun.com/docs/observability/loki/advanced-features/log-retention/",
      "title": "Log Retention | Compile N Run",
      "content": "On this page\n\n# Log Retention\n\n## Introduction\u200b\n\nLog retention is a critical aspect of any logging system, including Grafana Loki. It refers to how long your logs are stored before being automatically deleted or archived. Understanding and configuring log retention properly helps you:\n\n- Control storage costs\n- Maintain system performance\n- Comply with data retention regulations\n- Ensure you have access to logs when you need them\n\nIn this guide, we'll explore how log retention works in Grafana Loki, how to configure retention policies, and best practices for managing your log data effectively.\n\n## Understanding Log Retention in Loki\u200b\n\nLoki's approach to log retention is centered around the concept of \"chunks\" - compressed batches of log data that are stored and managed together. Log retention in Loki involves deciding when these chunks should be deleted or moved to different storage tiers.\n\n### Why Log Retention Matters\u200b\n\nWithout proper retention policies:\n\n1. **Storage costs can skyrocket**: Logs accumulate quickly in production environments\n2. **Query performance degrades**: More data means slower queries\n3. **Compliance risks increase**: Many industries have specific requirements for how long logs must be kept\n\n## Basic Retention Configuration\u200b\n\nLoki supports several methods for configuring retention:\n\n1. **Time-based retention**: Delete logs older than a specified period\n2. **Size-based retention**: Limit the total storage used for logs\n3. **Stream-based retention**: Set different retention policies for different log streams\n\nLet's look at how to configure each approach.\n\n### Time-Based Retention\u200b\n\nTime-based retention is the most common approach. Here's how to configure it in your Loki configuration file:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # 31 days\n```\n\nThis configuration tells Loki to keep logs for 31 days (744 hours) before deleting them.\n\nYou can also configure retention periods for specific tenants:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # Default retention period\n\n  per_tenant_override_config: /etc/loki/retention-overrides.yaml\n```\n\nWith a`retention-overrides.yaml`file that looks like:\n\n```yaml\noverrides:\n\n  \"tenant1\":\n\n    retention_period: 1440h # 60 days\n\n  \"tenant2\":\n\n    retention_period: 168h  # 7 days\n```\n\n### Size-Based Retention\u200b\n\nTo limit storage based on size:\n\n```yaml\ncompactor:\n\n  retention_enabled: true\n\n\n\nlimits_config:\n\n  retention_period: 0   # 0 means no time-based deletion\n\n  retention_stream_size_mb: 100  # Max size per stream\n\n  retention_total_size_mb: 10000  # Max total size\n```\n\nIn this example, logs will be deleted when:\n\n- A single stream exceeds 100MB\n- Total log storage exceeds 10GB\n\n### Stream-Based Retention with Labels\u200b\n\nYou can set different retention periods for different log streams using Loki's stream selector syntax:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # Default retention period\n\n  retention_stream_selectors:\n\n    - '{app=\"critical-service\"}': 8760h  # 1 year\n\n    - '{env=\"development\"}': 72h        # 3 days\n\n    - '{namespace=\"monitoring\"}': 2160h  # 90 days\n```\n\n## Implementing Retention in a Microservices Architecture\u200b\n\nLet's look at a practical example of configuring retention in a microservices environment:\n\nIn this architecture, we want to configure different retention policies:\n\n1. Frontend logs (high volume, less critical): 7 days\n2. Backend logs (medium volume, important): 30 days\n3. Database logs (low volume, critical): 1 year\n\nHere's the configuration:\n\n```yaml\nlimits_config:\n\n  retention_period: 168h # Default: 7 days\n\n  retention_stream_selectors:\n\n    - '{component=\"backend\"}': 720h     # 30 days\n\n    - '{component=\"database\"}': 8760h   # 365 days\n```\n\n## Best Practices for Log Retention\u200b\n\n### 1. Align with Business Requirements\u200b\n\nConfigure retention based on:\n\n- Compliance requirements (GDPR, HIPAA, SOX, etc.)\n- Incident investigation needs\n- Audit requirements\n- Debugging needs\n\n### 2. Consider a Tiered Storage Approach\u200b\n\nLoki supports moving older logs to cheaper storage:\n\n```yaml\nschema_config:\n\n  configs:\n\n    - from: 2022-01-01\n\n      store: boltdb-shipper\n\n      object_store: filesystem\n\n      schema: v11\n\n      index:\n\n        prefix: index_\n\n        period: 24h\n\nstorage_config:\n\n  boltdb_shipper:\n\n    active_index_directory: /loki/index\n\n    cache_location: /loki/cache\n\n    cache_ttl: 24h\n\n    shared_store: s3\n\n\n\n  filesystem:\n\n    directory: /loki/chunks\n\n\n\n  aws:\n\n    s3: s3://bucket/prefix\n\n    region: us-west-2\n```\n\nThis configuration stores recent logs in the filesystem for fast access and moves older logs to S3 for cost-effective long-term storage.\n\n### 3. Optimize for Query Patterns\u200b\n\nIf you frequently query only recent logs, set a shorter retention period for fast queries and keep longer retention for specific critical components.\n\n### 4. Monitor Storage Usage\u200b\n\nSet up alerts for storage usage to avoid unexpected costs:\n\n```yaml\n# In your Prometheus alerting rules\n\n- alert: LokiHighStorageUsage\n\n  expr: sum(loki_ingester_memory_chunks) > 1000000\n\n  for: 10m\n\n  labels:\n\n    severity: warning\n\n  annotations:\n\n    summary: \"High number of chunks in memory\"\n\n    description: \"Loki has a high number of chunks in memory, which might indicate retention policies need adjustment.\"\n```\n\n## Deletion vs. Compaction\u200b\n\nLoki supports two main approaches to managing old data:\n\n1. **Deletion**: Completely removes old logs\n2. **Compaction**: Combines and compresses old logs to reduce storage needs\n\nTo enable compaction:\n\n```yaml\ncompactor:\n\n  working_directory: /loki/compactor\n\n  shared_store: s3\n\n  compaction_interval: 24h\n```\n\n## Implementing a Log Lifecycle Policy\u200b\n\nA complete log management strategy often involves multiple stages:\n\n1. **Hot storage** (0-7 days): Kept in memory/local storage for fast queries\n2. **Warm storage** (8-30 days): Moved to object storage but still indexed\n3. **Cold storage** (31+ days): Archived with limited indexing\n4. **Deletion**: Removed after retention period\n\nHere's how to implement this in Loki:\n\n```yaml\nschema_config:\n\n  configs:\n\n    - from: 2022-01-01\n\n      store: boltdb-shipper\n\n      object_store: filesystem\n\n      schema: v11\n\n      index:\n\n        prefix: index_\n\n        period: 24h\n\nstorage_config:\n\n  boltdb_shipper:\n\n    active_index_directory: /loki/index/hot\n\n    cache_location: /loki/cache\n\n    cache_ttl: 24h\n\n    shared_store: s3\n\n\n\n  aws:\n\n    s3: s3://loki-bucket\n\n    s3forcepathstyle: true\n\n\n\nstorage_schema:\n\n  - regex: \".*\"\n\n    prefix: loki/\n\n    object_store: s3_hot\n\n    schema: v11\n\n    index:\n\n      enabled: true\n\n      store: boltdb-shipper\n\n      prefix: loki/index/\n\n      period: 24h\n\n  - regex: \".*\"\n\n    prefix: loki_cold/\n\n    object_store: s3_cold\n\n    schema: v11\n\n    index:\n\n      enabled: false\n\nruler:\n\n  storage:\n\n    type: local\n\n    local:\n\n      directory: /loki/rules\n\n  rule_path: /loki/rules-temp\n\n  alertmanager_url: http://alertmanager:9093\n\n  ring:\n\n    kvstore:\n\n      store: inmemory\n\n  enable_api: true\n```\n\n## Common Challenges and Solutions\u200b\n\n### Challenge 1: Balancing Performance and Retention\u200b\n\n**Solution**: Use Loki's stream labels to retain important logs longer while keeping retention shorter for high-volume, less critical logs.\n\n### Challenge 2: Compliance with Varying Requirements\u200b\n\n**Solution**: Configure tenant-specific retention policies that align with each department's compliance needs.\n\n### Challenge 3: Cost Management\u200b\n\n**Solution**: Implement a tiered storage approach with automated movement from expensive to cheaper storage.\n\n## Example: Log Retention in Production\u200b\n\nLet's examine a complete production example with a combination of strategies:\n\n```yaml\nlimits_config:\n\n  # Global defaults\n\n  retention_period: 168h # 7 days default\n\n  # Per-stream retention based on importance\n\n  retention_stream_selectors:\n\n    - '{app=\"payment-service\"}': 2160h    # 90 days for payment logs\n\n    - '{app=\"auth-service\"}': 720h        # 30 days for auth logs\n\n    - '{level=\"error\"}': 720h             # 30 days for all errors\n\n    - '{env=\"development\"}': 72h          # 3 days for dev environment\n\ncompactor:\n\n  working_directory: /loki/compactor\n\n  shared_store: s3\n\n  retention_enabled: true\n\n  retention_delete_delay: 2h\n\n  compaction_interval: 10m\n\n\n\nstorage_config:\n\n  aws:\n\n    s3: s3://logs-bucket\n\n    region: us-west-2\n\n    bucketnames: logs-bucket\n\n    access_key_id: ${AWS_ACCESS_KEY_ID}\n\n    secret_access_key: ${AWS_SECRET_ACCESS_KEY}\n\n    s3forcepathstyle: true\n\n    insecure: false\n```\n\n## Summary\u200b\n\nLog retention in Grafana Loki is a powerful feature that helps you balance storage costs, performance, and compliance requirements. Key takeaways include:\n\n- Configure retention based on time, size, or stream characteristics\n- Consider a tiered storage approach for cost optimization\n- Set different retention periods for different log types based on importance\n- Monitor storage usage to avoid unexpected costs\n- Implement compaction to optimize storage while retaining valuable logs\n\nBy implementing a thoughtful log retention strategy, you can ensure your Loki deployment remains performant, cost-effective, and compliant with your organization's requirements.\n\n## Additional Resources\u200b\n\n- [Official Loki Documentation on Retention](https://grafana.com/docs/loki/latest/operations/storage/retention/)\n- [Storage Best Practices](https://grafana.com/docs/loki/latest/operations/storage/)\n- [Compactor Documentation](https://grafana.com/docs/loki/latest/operations/compactor/)\n\n## Exercises\u200b\n\n1. Configure a basic time-based retention policy for Loki that keeps logs for 14 days.\n2. Create a multi-tier retention policy that keeps error logs for 60 days and all other logs for 7 days.\n3. Set up a monitoring dashboard in Grafana to track Loki storage usage over time.\n4. Design a retention policy that complies with GDPR requirements for a web application.\n\n\ud83d\udca1 Found a typo or mistake? Click **\"Edit this page\"** to suggest a correction. Your feedback is greatly appreciated!\n\n- Introduction\n- Understanding Log Retention in Loki - Why Log Retention Matters\n- Basic Retention Configuration - Time-Based Retention\n- Size-Based Retention\n- Stream-Based Retention with Labels\n- Implementing Retention in a Microservices Architecture\n- Best Practices for Log Retention - 1. Align with Business Requirements\n- 2. Consider a Tiered Storage Approach\n- 3. Optimize for Query Patterns\n- 4. Monitor Storage Usage\n- Deletion vs. Compaction\n- Implementing a Log Lifecycle Policy\n- Common Challenges and Solutions - Challenge 1: Balancing Performance and Retention\n- Challenge 2: Compliance with Varying Requirements\n- Challenge 3: Cost Management\n- Example: Log Retention in Production\n- Summary\n- Additional Resources\n- Exercises",
      "content_chars": 10758,
      "published_date": "2026-10-06"
    },
    {
      "rank": 8,
      "url": "https://www.reddit.com/r/grafana/comments/1mducu6/working_with_loki_need_help_retaining_logs/",
      "title": "r/grafana on Reddit: Working with Loki: Need help retaining logs locally for 1 day, then ship to S3 (MinIO) and delete after 1 week",
      "content": "# Working with Loki: Need help retaining logs locally for 1 day, then ship to S3 (MinIO) and delete after 1 week\nRight now, I\u2019m able to see logs getting stored in S3. But the problem is\u2014they don\u2019t stick around long. They\u2019re getting deleted much earlier than I expect, and I'm not sure why.\n\n```\nauth_enabled: false\n\nserver:\n  http_listen_port: 3100\n  log_level: info\n\ncommon:\n  path_prefix: /loki\n  storage:\n    s3:\n      endpoint: http://minio:9000\n      region: us-east-1\n      bucketnames: loki-data\n      access_key_id: admin\n      secret_access_key: password123\n      s3forcepathstyle: true\n      insecure: true\n\n  replication_factor: 1\n  ring:\n    kvstore:\n      store: inmemory\n\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/index\n    cache_location: /loki/cache\n    cache_ttl: 30m\n  aws:\n    endpoint: http://minio:9000\n    region: us-east-1\n    bucketnames: loki-data\n    access_key_id: admin\n    secret_access_key: password123\n    s3forcepathstyle: true\n    insecure: true\n\nlimits_config:\n  retention_period: 1h\n\ncompactor:\n  working_directory: /loki/compactor\n  compaction_interval: 30m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  delete_request_store: s3\n\ningester:\n  wal:\n    enabled: true\n  chunk_idle_period: 30m\n  max_chunk_age: 30m\n\nanalytics:\n  reporting_enabled: false\n```\n\n\ud83e\udde9 **Things I\u2019ve noticed / tried:**\n\n* S3 shows data briefly, then it's gone\n\n* `retention_period` under `limits_config` is set to 1h \u2014 maybe that's causing early deletion?\n\n* Not sure how to set *different retention durations* for local disk vs. S3\n\n* I want to make sure logs live:\n\n+ 1 day on local disk\n\n## FaderJockey2600\nYour retention_period indeed dictates the overall retention, so logs get deleted after 1 hour. Loki as far as I\u2019ve experienced does not offer a multi-tiered indexing system like Elastic does. So you cannot configure it to use local and s3 storage for the same data.\n\n## Parley_P_Pratt\nAs others has already pointed out, you cannot configure this. But a work around we do is using memcached to store recent logs in. They still get sent to S3 directly, but recent logs query much faster",
      "content_chars": 2301,
      "published_date": "2025-07-31"
    },
    {
      "rank": 9,
      "url": "https://www.reddit.com/r/grafana/comments/mvrg94/increase_loki_log_retention/",
      "title": "r/grafana on Reddit: Increase loki log retention?",
      "content": "# Increase loki log retention?\n```\nauth_enabled: false\nserver:\n  http_listen_port: 3100\n\ningester:\n  lifecycler:\n    address: 127.0.0.1\n    ring:\n      kvstore:\n        store: inmemory\n      replication_factor: 1\n    final_sleep: 0s\n  chunk_idle_period: 5m\n  chunk_retain_period: 30s\n  max_transfer_retries: 1\n\nschema_config:\n  configs:\n    - from: 2018-04-15\n      store: boltdb-shipper\n      object_store: aws\n      schema: v11\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  aws:\n    s3: s3://us-east-1/mylogbucketname\n\n  boltdb_shipper:\n    active_index_directory: /opt/loki/index\n    shared_store: s3\n    cache_location: /opt/loki/cache\n\ncompactor:\n  working_directory: /opt/loki/compactor\n  shared_store: s3\n\nlimits_config:\n  enforce_metric_name: false\n  reject_old_samples: true\n  reject_old_samples_max_age: 168h\n  ingestion_rate_mb: 8\n  ingestion_burst_size_mb: 16\n\nchunk_store_config:\n  max_look_back_period: 0s\n\ntable_manager:\n  retention_deletes_enabled: false\n  retention_period: 0s\n```\n\n## SelfDestructSep2020\nYour problem is the max lookback setting. Set that equal to or greater than your retention if you want to be able to search the full extent of your logs. I'm not entirely sure what happens when it's set to 0s but it might be jumping to a default value in that case.\n\n Loki doesn't actually delete your log chunks either, you need to configure s3 to expire the objects at the same rate as your retention setting.\n\n### Deleted User\n```\n# Limit how long back data can be queried. Default is disabled. \n# This should always be set to a value less than or equal to \n# what is set in `table_manager.retention_period` .\n```\n\nSo that sounds to me like it being set to 0s should disable any limits to look back; it also mentions that it should be less than or equal to the retention period, not greater than the retention period (the retention period is also disabled anyway as it's set to 0s).\n\n### Deleted User \u203a Deleted User\nOk, I worked out the issue. It seems Grafana limits the amount of logs returned in a query to 1000 lines. I limited my search to a single host and a specific log message and I now get results that are hours old.\n\n EDIT: For the record, the line limit can be adjusted in the Loki datasource config. Increasing it may cause a browser performance hit though.",
      "content_chars": 2330,
      "published_date": "2021-04-21"
    },
    {
      "rank": 10,
      "url": "https://community.grafana.com/t/how-to-set-up-log-retention-correctly/156719",
      "title": "How to set up log retention correctly? - Grafana Loki - Grafana Labs Community Forums",
      "content": "[Documentation](https://grafana.com/docs/?pg=community&plcmt=topnav)\n\n[Ask Grot AI](https://grafana.com/grot/?pg=community&plcmt=topnav)\n\n[Plugins](https://grafana.com/plugins/?pg=community&plcmt=topnav) [Get Grafana](https://grafana.com/get/?pg=community&plcmt=topnav)",
      "content_chars": 269,
      "published_date": "2025-08-25"
    }
  ],
  "answer_text": null,
  "citations": [],
  "raw_response": {
    "success": true,
    "error": "",
    "tx_id": "tx_72a08941-62ff-43ce-9866-6f08a13ad016",
    "query": "In Loki, the log aggregation system from Grafana Labs, which configuration block does retention_enabled have to be set in for old log data to actually be deleted?",
    "results": [
      {
        "title": "Log entry deletion | Grafana Loki documentation",
        "url": "https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/",
        "content": "Menu\n\n[Documentation](https://grafana.com/docs/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Grafana Loki](https://grafana.com/docs/loki/latest/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Manage](https://grafana.com/docs/loki/latest/operations/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Storage](https://grafana.com/docs/loki/latest/operations/storage/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)Log entry deletion\n\nOpen source\n\n# Log entry deletion\n\nGrafana Loki supports the deletion of log entries from a specified stream. Log entries that fall within a specified time window and match an optional line filter are those that will be deleted.\n\nLog entry deletion is supported when the TSDB index is configured as the index store. It is also supported on the deprecated BoltDB Shipper index, but BoltDB Shipper is being removed in Loki 4.0, so new deployments should use TSDB.\n\nThe compactor component exposes REST [endpoints](https://grafana.com/docs/loki/latest/reference/loki-http-api/#compactor) that process delete requests. Hitting the endpoint specifies the streams and the time window. The deletion of the log entries takes place after a configurable cancellation time period expires.\n\nLog entry deletion relies on configuration of the custom logs retention workflow as defined for the [compactor](https://grafana.com/docs/loki/latest/operations/storage/retention/#compactor). The compactor looks at unprocessed requests which are past their cancellation period to decide whether a chunk is to be deleted or not.\n\n## Configuration\n\nEnable log entry deletion by setting `retention_enabled` to `true` in the compactor\u2019s configuration (`-compactor.retention-enabled` on the command line). The tenant\u2019s `deletion_mode` must also be `filter-only` or `filter-and-delete`, which is the case by default. `delete_request_store` also needs to be configured when retention is enabled to process delete requests, this determines the storage bucket that stores the delete requests.\n\n> WarningBe very careful when enabling retention. It is strongly recommended that you also enable versioning on your objects in object storage to allow you to recover from accidental misconfiguration of a retention setting. If you want to enable deletion but do not want to enforce retention, configure the retention_period setting with a value of 0s.\n\n`deletion_mode` is a global and per-tenant setting in [limits_config](https://grafana.com/docs/loki/latest/configure/#limits_config). Its default value is `filter-and-delete`, so log entry deletion is active for every tenant as soon as `retention_enabled` is `true` and `delete_request_store` is configured, unless you explicitly change the mode. Set `deletion_mode` in the main configuration file for a global default, or override it per tenant in the [runtime configuration file](https://grafana.com/docs/loki/latest/configure/#runtime-configuration-file).\n\n`deletion_mode` supports three values:\n\n- `disabled`: Deletion is not allowed. Requests to the deletion API endpoints are rejected with a `403 Forbidden` response.\n- `filter-only`: Log lines matching the query in the delete request are filtered out when querying Loki. They are not removed from storage.\n- `filter-and-delete`: Log lines matching the query in the delete request are filtered out when querying Loki, and they are also removed from storage. This is the default.\n\nA delete request may be canceled within a configurable cancellation period. Set the `delete_request_cancel_period` in the compactor\u2019s YAML configuration or on the command line when invoking Loki. Its default value is 24h. To cancel a delete request that has already started processing, pass the `force=true` query parameter to the cancellation endpoint.\n\nDelete requests that include a line filter are split into smaller requests that each cover no more than `delete_max_interval` (24h by default). You can request a smaller split for a single request with the `max_interval` query parameter, but it cannot be larger than `delete_max_interval`. Delete requests without a line filter are not split. Refer to the [HTTP API reference](https://grafana.com/docs/loki/latest/reference/loki-http-api/#request-log-deletion) for the full set of request parameters.\n\nDelete requests themselves are stored using the database type set by `delete_request_store_db_type`, which defaults to `boltdb`. You can instead use `sqlite`. When you migrate from one database type to another, you can set `backup_delete_request_store_db_type` to `boltdb` so that delete requests are also written to a backup database. Refer to the [compactor configuration reference](https://grafana.com/docs/loki/latest/configure/#compactor) for details.\n\nThe deletion API endpoints are registered only when `compactor.retention_enabled` is `true`. When retention is not enabled, the endpoints are unavailable for every tenant, whatever the `deletion_mode` value is. When retention is enabled, use the `deletion_mode` tenant override to control which tenants can use the deletion API.\n\n> NoteDeletion of log lines with line filters is one of the compactor\u2019s most resource-intensive operations. If you delete large volumes of data with line filters, refer to > Horizontal scaling of Compactor to distribute that work across multiple compactor instances.\n\n## Was this page helpful?\n\n![\ud83d\udc4d](https://grafana.com/media/images/svg/thumbs-up.svg) Yes ![\ud83d\udc4e](https://grafana.com/media/images/svg/thumbs-up.svg) No\n\n[Suggest an edit in GitHub](https://github.com/grafana/loki/edit/release-3.7.x/docs/sources/operations/storage/logs-deletion.md)[Create a GitHub issue](https://github.com/grafana/loki/issues/new?title=Documentation%20feedback:%20/docs/sources/operations/storage/logs-deletion.md)[Email docs@grafana.com](mailto:docs@grafana.com)[Help and support](https://grafana.com/help/)[Community](https://grafana.com/community/)\n\n## Related resources from Grafana Labs\n\nAdditional helpful documentation, links, and articles:\n\n[VideoGetting started with logging and Grafana LokiSee a demo of the updated features in Loki, and how to create metrics from logs and alert on your logs with powerful Prometheus-style alerting rules.](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki/)[VideoEssential Grafana Loki configuration settingsThis webinar focuses on Grafana Loki configuration including agents Promtail and Docker; the Loki server; and Loki storage for popular backends.](https://grafana.com/go/webinar/logging-with-loki-essential-configuration-settings/)[VideoScaling and securing your logs with Grafana LokiThis webinar covers the challenges of scaling and securing logs, and how Grafana Cloud Logs powered by Grafana Loki can help, cost-effectively.](https://grafana.com/go/webinar/scaling-and-securing-your-logs-with-grafana-loki/)",
        "description": "Describes how Loki implements log deletion and deletion configuration options.",
        "source": "web",
        "price": 0.0015,
        "length": 7027,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2022-11-02",
        "crawl_date": "2026-10-06",
        "id": "https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/",
        "image_url": {
          "0": "https://grafana.com/meta-generator/Log+entry+deletion@@@loki@@@4.png"
        },
        "relevance_score": 0.9478415225716799
      },
      {
        "title": "purge - why my Loki log retention is not working? - Stack Overflow",
        "url": "https://stackoverflow.com/questions/73138278/why-my-loki-log-retention-is-not-working",
        "content": "# why my Loki log retention is not working?\n```\nloki:\n  enabled: true\n  isDefault: true\n  table_manager:\n    retention_deletes_enabled: true\n    retention_period: 12h\n```\n\nShare\n\nShare a link to this question\n\nCC BY-SA 4.0\n\nShort permalink to this question\n\nImprove this question\n\nFollow this question to receive notifications\n\n2\n\n- 1 What exactly do you mean by: \"12h\" old logs are not deleting? I am asking because Loki is not supposed to delete any log file. To delete those files, you would use some kind of log rotation provided by your logging framework. (BTW: Getting those files is part of the Promtail configuration not part of Loki.)\n\n## 3 Answers 3\nSorted by: Reset to default\n\nI have the following configurations related to log retention:\n\n```\n...\ncompactor:\n  ...\n  retention_enabled: true\n...\nlimits_config:\n  ...\n  retention_period: 90d\n...\n```\n\nSee more about Loki configuration in the documentation here\n\nShare\n\nShare a link to this answer\n\nCC BY-SA 4.0\n\nShort permalink to this answer\n\nImprove this answer\n\nFollow this answer to receive notifications\n\n## Comments\nGrafana Loki Storage Retention\n\nBy default, when table_manager.retention_deletes_enabled or compactor.retention_enabled *flags are not set*, **then logs sent to Loki live forever.**\n\n**The minimum retention period is 24h**.\n\nThe retention period **must be a multiple** of the *index* and chunks table period, configured in the period_config block.\n\n```yaml\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: boltdb-shipper\n      object_store: filesystem\n      schema: v11\n      index:\n        prefix: index_\n        period: 24h\n\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 24h\n```\n\nThe other way to setup the retention in Grafana Loki is via the Compactor.\n\n```yaml\nlimits_config:\n  retention_period: 168h\n\ncompactor:\n  working_directory: /data/retention\n  compaction_interval: 10m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: boltdb-shipper\n      object_store: filesystem\n      schema: v11\n      index:\n        prefix: index_\n        period: 24h\n```\n\nNote that **retention is only available if the index period is 24h**.\n\nTo purge data immediately you could set the retention_delete_delay interval to 1m. This is a delay after which chunks will be fully deleted during retention.\n\nDo not forget to restart Loki after changing the configuration:\n\n```bash\nsystemctl restart loki\nsystemctl status loki\n```\n\nShare\n\nShare a link to this answer\n\nCC BY-SA 4.0\n\nShort permalink to this answer\n\nImprove this answer\n\nFollow this answer to receive notifications\n\n## 5 Comments\nWith the exception of the filesystem chunk store, Loki will not delete old chunk stores. This is generally handled instead by configuring TTLs (time to live) in the chunk store of your choice (bucket lifecycles in S3/GCS, and TTLs in Cassandra). Neither will Loki currently delete old data when your local disk fills when using the filesystem chunk store \u2013 deletion is only determined by retention duration.\n\n You need to configure bucket lifecycles in your storage provider to be able to delete chunk stores.",
        "description": "# why my Loki log retention is not working?\n```\nloki:\n  enabled: true\n  isDefault: true\n  table_manager:\n    retention_deletes_enabled: true\n    retention_period: 12h\n```\n\nShare\n\nShare a link to this question\n\nCC BY-SA 4.0\n\nShort permalink to this question\n\nImprove this question\n\nFollow this question to receive notifications\n\n2",
        "source": "web",
        "price": 0.0015,
        "length": 3204,
        "data_type": "unstructured",
        "source_type": "forum",
        "publication_date": "2022-07-27",
        "crawl_date": "2026-10-06",
        "id": "https://stackoverflow.com/questions/73138278/why-my-loki-log-retention-is-not-working",
        "image_url": {},
        "relevance_score": 0.924302264738498
      },
      {
        "title": "Medium",
        "url": "https://medium.com/@aditya.hilman_10961/enabling-log-retention-in-loki-with-compactor-fe3de2002b47",
        "content": "[DevOps](https://medium.com/tag/devops?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Logging](https://medium.com/tag/logging?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Monitoring](https://medium.com/tag/monitoring?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Kubernetes](https://medium.com/tag/kubernetes?source=post_page---header_tags--fe3de2002b47-----------------------------------------)[Loki](https://medium.com/tag/loki?source=post_page---header_tags--fe3de2002b47-----------------------------------------)\n\n# Enabling Log Retention in Loki with Compactor\n\n[Aditya Hilman](https://medium.com/@aditya.hilman_10961?source=post_page---byline--fe3de2002b47-----------------------------------------)2 min readDec 3, 2024\n\n--\n\n1\n\n[Listen](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2Fplans%3Fdimension%3Dpost_audio_button%26postId%3Dfe3de2002b47&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40aditya.hilman_10961%2Fenabling-log-retention-in-loki-with-compactor-fe3de2002b47&source=---header_actions--fe3de2002b47---------------------post_audio_button--------------------)\n\nShare\n\nEfficient log management is essential as Kubernetes workloads grow in complexity. In a [previous guide](https://medium.com/@aditya.hilman_10961/centralizing-kubernetes-logs-with-promtail-loki-and-grafana-a-step-by-step-guide-cafffb59a13e), we explored how to centralize logs using Loki, Promtail, and Grafana. But once logs start accumulating, controlling their retention becomes critical to manage storage and maintain performance.\n\nThis article will focus on configuring **log retention in Loki** using the **Compactor**, a core component designed to manage retention policies effectively.\n\n## Understanding Loki\u2019s Compactor\n\nThe **Compactor** in Loki is responsible for managing and enforcing retention periods, cleaning up old logs, and optimizing stored data. By enabling and configuring the compactor, you can automate log deletion and maintain a sustainable log storage strategy.\n\n## Configuring Log Retention in Loki\n\nEnsure the compactor is enabled in your`/etc/loki/config.yml`file:\n\n```python\nlimits_config:\n retention_period: 30d\n\ncompactor:\n working_directory: /tmp/loki/retention\n retention_enabled: true\n retention_delete_delay: 2h\n delete_request_store: filesystem\n```\n\n`limits_config.retention_period: 30d`Sets the global retention period to 30 days. Logs older than 30 days will be marked for deletion by the compactor.\n\n`compactor.working_directory: /tmp/loki/retention`Specifies the directory where the compactor stores its state and temporary files while processing.\n\n`compactor.retention_enabled: true`Enables the retention policy enforcement in Loki. Without this, the compactor will not delete old logs.\n\n`compactor.retention_delete_delay: 2h`Introduces a delay of 2 hours before permanently deleting logs. This ensures a buffer in case of accidental deletion or misconfigurations.\n\n`compactor.delete_request_store: filesystem`Specifies the storage backend for deletion requests.`filesystem`is used here to store delete requests locally on disk. You can switch to other storage backends (e.g.,`s3`,`gcs`) if needed.\n\n## Verify Retention Policy\n\nTo confirm that the retention policy is working, you can inspect the directory where Loki stores its chunks. If you are using the filesystem as your`shared_store`, the chunks are stored in`/tmp/loki/chunks`.\n\n**Check the Directory Contents**: After the compactor runs, check for log chunks in the directory:\n\n```python\nls -lh /tmp/loki/chunks\n```\n\n**Verify Cleanup**:\nEnsure that files corresponding to logs older than the defined retention period (`720h`in this case) are no longer present.\n\nIf old chunks are not deleted, double-check your configuration for errors and ensure the`compactor`is running without issues.\n\n## Conclusion\n\nLog retention is a critical aspect of managing a scalable and efficient log storage system. By enabling Loki\u2019s Compactor and setting retention policies, you can control log storage while ensuring compliance and performance.\n\nIf you\u2019re new to Loki, be sure to check out [step-by-step guide](https://medium.com/@aditya.hilman_10961/centralizing-kubernetes-logs-with-promtail-loki-and-grafana-a-step-by-step-guide-cafffb59a13e) to set up your centralized logging stack.",
        "description": "Enabling Log Retention in Loki with Compactor Efficient log management is essential as Kubernetes workloads grow in complexity. In a previous guide, we explored how to centralize logs using Loki \u2026",
        "source": "web",
        "price": 0.0015,
        "length": 4408,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2024-12-03",
        "crawl_date": "2026-10-06",
        "id": "https://medium.com/@aditya.hilman_10961/enabling-log-retention-in-loki-with-compactor-fe3de2002b47",
        "image_url": {
          "0": "https://miro.medium.com/v2/resize:fit:538/1*TDFVSRecndbaVz1nhu3HuQ.png"
        },
        "relevance_score": 0.920436286633989
      },
      {
        "title": "How to Implement Loki Retention Strategies",
        "url": "https://oneuptime.com/blog/post/2026-01-30-loki-retention-strategies/view",
        "content": "---\n\n## Introduction\n\nManaging log retention in Grafana Loki requires understanding how Loki stores data and which components handle deletion. Unlike traditional databases where you simply set a TTL, Loki's architecture involves chunks, indexes, and a compactor service that work together to enforce retention policies.\n\nThis guide covers the practical implementation of retention strategies in Loki, from basic configuration to advanced per-tenant overrides. We will walk through working configurations, explain the trade-offs between different approaches, and help you build a retention strategy that balances cost with operational requirements.\n\n## Understanding Loki's Storage Model\n\nBefore diving into retention configuration, you need to understand how Loki stores logs.\n\n### Chunks and Indexes\n\nLoki stores log data in two types of objects:\n\n| Component | Purpose | Storage Backend |\n| --- | --- | --- |\n| Chunks | Compressed log data | Object storage (S3, GCS, filesystem) |\n| Index | Metadata for querying chunks | BoltDB, Cassandra, or object storage (TSDB) |\n\nChunks contain the actual log lines, compressed and grouped by stream. The index maps labels and time ranges to specific chunks, enabling efficient queries.\n\n### The TSDB Index (Recommended)\n\nStarting with Loki 2.8, the TSDB index is the recommended schema. It stores indexes in object storage alongside chunks, simplifying operations and enabling better retention handling.\n\nHere is a basic schema configuration using TSDB:\n\n```yaml\n# loki-config.yaml\n\nschema_config:\n  configs:\n    - from: 2024-01-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n```\n\nThe`period: 24h`setting creates daily index tables. This matters for retention because deletion happens at the table level for older index types.\n\n## Compactor-Based Retention\n\nThe compactor is Loki's primary mechanism for enforcing retention. It runs as a separate target (or within the single binary) and handles chunk deletion based on configured policies.\n\n### Enabling the Compactor\n\nFirst, enable the compactor and configure retention:\n\n```yaml\n# loki-config.yaml\ncompactor:\n  # Working directory for compaction\n  working_directory: /loki/compactor\n\n  # How often the compactor runs\n  compaction_interval: 10m\n\n  # Enable retention enforcement\n  retention_enabled: true\n\n  # Store used for delete requests\n  delete_request_store: s3\n\n  # How often to check for chunks to delete\n  retention_delete_delay: 2h\n\n  # Number of workers for deletion\n  retention_delete_worker_count: 150\n\n  # Delete request cancel period\n  delete_request_cancel_period: 24h\n```\n\nThe`retention_delete_delay`parameter adds a safety buffer. Chunks marked for deletion wait this duration before actual removal. This gives you time to cancel accidental deletions.\n\n### Global Retention Period\n\nSet a default retention period that applies to all tenants:\n\n```yaml\n# loki-config.yaml\nlimits_config:\n  # Global retention period - logs older than this are deleted\n  retention_period: 744h  # 31 days\n```\n\nThe`retention_period`value uses Go duration format. Common values:\n\n| Duration | Days | Use Case |\n| --- | --- | --- |\n| 168h | 7 | Development environments |\n| 720h | 30 | Standard production |\n| 2160h | 90 | Compliance requirements |\n| 8760h | 365 | Long-term audit logs |\n\n### Complete Basic Configuration\n\nHere is a complete configuration enabling compactor retention with S3 storage:\n\n```yaml\n# loki-config.yaml\nauth_enabled: false\n\nserver:\n  http_listen_port: 3100\n  grpc_listen_port: 9096\n\ncommon:\n  path_prefix: /loki\n  storage:\n    s3:\n      endpoint: s3.us-east-1.amazonaws.com\n      bucketnames: loki-chunks\n      region: us-east-1\n      access_key_id: ${AWS_ACCESS_KEY_ID}\n      secret_access_key: ${AWS_SECRET_ACCESS_KEY}\n      s3forcepathstyle: false\n  replication_factor: 1\n  ring:\n    kvstore:\n      store: inmemory\n\nschema_config:\n  configs:\n    - from: 2024-01-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/tsdb-index\n    cache_location: /loki/tsdb-cache\n    cache_ttl: 24h\n\ncompactor:\n  working_directory: /loki/compactor\n  compaction_interval: 10m\n  retention_enabled: true\n  delete_request_store: s3\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n\nlimits_config:\n  retention_period: 720h  # 30 days\n  ingestion_rate_mb: 10\n  ingestion_burst_size_mb: 20\n  max_streams_per_user: 10000\n  max_entries_limit_per_query: 5000\n```\n\n## Per-Tenant Retention Overrides\n\nIn multi-tenant deployments, different teams often have different retention requirements. Loki supports per-tenant overrides through a runtime configuration file.\n\n### Configuring Runtime Overrides\n\nFirst, tell Loki where to find the overrides file:\n\n```yaml\n# loki-config.yaml\nruntime_config:\n  file: /loki/runtime-config.yaml\n  period: 10s  # How often to reload\n```\n\nThen create the overrides file:\n\n```yaml\n# runtime-config.yaml\noverrides:\n  # Team with short retention needs\n  team-development:\n    retention_period: 168h  # 7 days\n    ingestion_rate_mb: 5\n    max_streams_per_user: 5000\n\n  # Team requiring longer retention for compliance\n  team-security:\n    retention_period: 8760h  # 365 days\n    ingestion_rate_mb: 20\n    max_streams_per_user: 50000\n\n  # Default tenant with standard settings\n  team-platform:\n    retention_period: 720h  # 30 days\n```\n\n### Stream-Based Retention\n\nYou can apply different retention periods based on log stream selectors. This enables fine-grained control within a single tenant.\n\n```yaml\n# runtime-config.yaml\noverrides:\n  production:\n    retention_period: 720h  # Default: 30 days\n    retention_stream:\n      # Keep error logs longer\n      - selector: '{level=\"error\"}'\n        priority: 1\n        period: 2160h  # 90 days\n\n      # Keep audit logs for compliance\n      - selector: '{log_type=\"audit\"}'\n        priority: 2\n        period: 8760h  # 365 days\n\n      # Delete debug logs quickly\n      - selector: '{level=\"debug\"}'\n        priority: 3\n        period: 72h  # 3 days\n\n      # Short retention for high-volume health checks\n      - selector: '{endpoint=\"/health\"}'\n        priority: 4\n        period: 24h  # 1 day\n```\n\nThe`priority`field determines which rule wins when multiple selectors match. Higher priority values take precedence.\n\n### Dynamic Override Reloading\n\nThe compactor automatically reloads runtime configuration at the interval specified by`period`. To verify overrides are applied, check the compactor logs:\n\n```bash\n# View compactor logs for retention activity\nkubectl logs -l app=loki-compactor | grep -i retention\n```\n\nYou should see entries like:\n\n```\nlevel=info ts=2026-01-30T10:00:00.000Z caller=compactor.go:450 msg=\"applying retention\" tenant=team-security retention=8760h0m0s\n```\n\n## Table Manager vs Compactor Retention\n\nLoki historically used the Table Manager for retention with BoltDB indexes. Understanding the differences helps when migrating or maintaining older deployments.\n\n### Comparison\n\n| Feature | Table Manager | Compactor |\n| --- | --- | --- |\n| Index Support | Legacy chunk/index stores and BoltDB-shipper | TSDB and BoltDB-shipper |\n| Deletion Granularity | Table-level | Chunk-level |\n| Per-Tenant Overrides | Limited | Full support |\n| Stream Selectors | No | Yes |\n| Status | Deprecated | Recommended |\n\n### Table Manager Configuration (Legacy)\n\nIf you are still running BoltDB indexes, here is how Table Manager retention works:\n\n```yaml\n# loki-config.yaml (legacy configuration)\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 720h  # 30 days\n\n  # Table creation settings\n  index_tables_provisioning:\n    inactive_read_throughput: 1\n    inactive_write_throughput: 1\n    provisioned_read_throughput: 100\n    provisioned_write_throughput: 100\n\n  chunk_tables_provisioning:\n    inactive_read_throughput: 1\n    inactive_write_throughput: 1\n    provisioned_read_throughput: 100\n    provisioned_write_throughput: 100\n```\n\n### Migration Path\n\nTo migrate from Table Manager to Compactor retention:\n\n1. Deploy the new schema alongside the old one\n2. Enable compactor retention for new data\n3. Let Table Manager handle old data until it ages out\n4. Remove Table Manager configuration after full migration\n\n```yaml\n# Schema configuration for migration\nschema_config:\n  configs:\n    # Old schema - Table Manager handles retention\n    - from: 2023-01-01\n      store: boltdb-shipper\n      object_store: s3\n      schema: v12\n      index:\n        prefix: loki_index_\n        period: 24h\n\n    # New schema - Compactor handles retention\n    - from: 2024-06-01\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: loki_tsdb_\n        period: 24h\n```\n\n## Deletion Markers and the Deletion API\n\nBeyond automatic retention, Loki supports manual deletion through its API. This is useful for removing sensitive data or clearing specific log streams.\n\n### Enabling the Deletion API\n\n```yaml\n# loki-config.yaml\ncompactor:\n  retention_enabled: true\n  delete_request_cancel_period: 24h\n  delete_request_store: s3\n\n# runtime-config.yaml\noverrides:\n  tenant-a:\n    deletion_mode: filter-and-delete\n```\n\n### Creating Delete Requests\n\nUse the deletion API to mark logs for removal:\n\n```bash\n# Delete all logs matching a selector within a time range\ncurl -X POST \\\n  \"http://loki:3100/loki/api/v1/delete\" \\\n  -H \"Content-Type: application/x-www-form-urlencoded\" \\\n  -d \"query={app=\\\"legacy-service\\\"}\" \\\n  -d \"start=1704067200\" \\\n  -d \"end=1706745600\"\n```\n\nThe timestamps are Unix timestamps in seconds or RFC3339 timestamps. Convert from dates:\n\n```bash\n# Convert dates to Unix timestamps in seconds\nSTART=$(date -d \"2024-01-01\" +%s)\nEND=$(date -d \"2024-02-01\" +%s)\n\ncurl -X POST \\\n  \"http://loki:3100/loki/api/v1/delete\" \\\n  -d \"query={app=\\\"legacy-service\\\"}\" \\\n  -d \"start=${START}\" \\\n  -d \"end=${END}\"\n```\n\n### Listing and Canceling Delete Requests\n\nView pending deletions:\n\n```bash\n# List all delete requests\ncurl \"http://loki:3100/loki/api/v1/delete\"\n```\n\nResponse:\n\n```javascript\n[\n  {\n    \"request_id\": \"abc123\",\n    \"start_time\": 1704067200,\n    \"end_time\": 1706745600,\n    \"query\": \"{app=\\\"legacy-service\\\"}\",\n    \"status\": \"received\",\n    \"created_at\": 1706800000\n  }\n]\n```\n\nCancel a pending deletion (within the cancel period):\n\n```bash\ncurl -X DELETE \\\n  \"http://loki:3100/loki/api/v1/delete?request_id=abc123\"\n```\n\n### Delete Request States\n\n| State | Description |\n| --- | --- |\n| received | Request received, waiting for cancel period |\n| building | Cancel period passed, building deletion plan |\n| deleting | Actively deleting chunks |\n| processed | Deletion complete |\n\n## Storage Considerations\n\nRetention configuration directly impacts storage costs and performance. Here are key considerations for production deployments.\n\n### Object Storage Lifecycle Policies\n\nCombine Loki retention with cloud provider lifecycle policies for defense in depth:\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-chunks-cleanup\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/chunks/\"\n      },\n      \"Expiration\": {\n        \"Days\": 45\n      }\n    },\n    {\n      \"ID\": \"loki-index-cleanup\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/index/\"\n      },\n      \"Expiration\": {\n        \"Days\": 45\n      }\n    }\n  ]\n}\n```\n\nSet the lifecycle policy expiration slightly longer than your Loki retention to catch any orphaned objects.\n\n### Storage Tiering for Cost Optimization\n\nMove older logs to cheaper storage tiers before deletion:\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-tiering\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"loki/\"\n      },\n      \"Transitions\": [\n        {\n          \"Days\": 7,\n          \"StorageClass\": \"STANDARD_IA\"\n        },\n        {\n          \"Days\": 30,\n          \"StorageClass\": \"GLACIER_IR\"\n        }\n      ],\n      \"Expiration\": {\n        \"Days\": 90\n      }\n    }\n  ]\n}\n```\n\n### Chunk Size and Compression\n\nOptimize chunk configuration to reduce storage:\n\n```yaml\n# loki-config.yaml\ningester:\n  chunk_encoding: snappy  # Options: gzip, lz4, snappy, zstd\n  chunk_target_size: 1572864  # 1.5MB target\n  chunk_idle_period: 30m\n  chunk_block_size: 262144  # 256KB\n\n  # Flush chunks before they get too old\n  max_chunk_age: 2h\n\n  # Number of chunks to keep in memory\n  chunk_retain_period: 30s\n```\n\nCompression comparison:\n\n| Algorithm | Compression Ratio | CPU Usage | Best For |\n| --- | --- | --- | --- |\n| snappy | Good | Low | General use |\n| lz4 | Good | Low | High throughput |\n| gzip | Better | Medium | Storage constrained |\n| zstd | Best | Medium-High | Cold storage |\n\n## Monitoring Retention\n\nTrack retention effectiveness with these metrics.\n\n### Key Compactor Metrics\n\n```yaml\n# Prometheus recording rules for retention monitoring\ngroups:\n  - name: loki-retention\n    rules:\n      # Compactor deletion rate\n      - record: loki:compactor:deleted_lines_per_hour\n        expr: |\n          sum(rate(loki_compactor_deleted_lines[1h])) * 3600\n\n      # Pending deletion requests\n      - record: loki:compactor:pending_deletions\n        expr: |\n          sum(loki_compactor_pending_delete_requests_count)\n\n      # Compaction lag\n      - record: loki:compactor:lag_seconds\n        expr: |\n          time() - max(loki_compactor_apply_retention_last_successful_run_timestamp_seconds)\n```\n\n### Alerting Rules\n\n```yaml\n# Prometheus alerting rules\ngroups:\n  - name: loki-retention-alerts\n    rules:\n      # Alert if compactor is not running\n      - alert: LokiCompactorNotRunning\n        expr: |\n          time() - max(loki_compactor_apply_retention_last_successful_run_timestamp_seconds) > 3600\n        for: 15m\n        labels:\n          severity: warning\n        annotations:\n          summary: \"Loki compactor has not run in over an hour\"\n          description: \"Retention policies are not being enforced\"\n\n      # Alert on high deletion backlog\n      - alert: LokiDeletionBacklog\n        expr: |\n          loki_compactor_pending_delete_requests_count > 100\n        for: 1h\n        labels:\n          severity: warning\n        annotations:\n          summary: \"High number of pending deletion requests\"\n\n      # Alert if old delete requests are not being processed\n      - alert: LokiOldestDeleteRequestStalled\n        expr: |\n          loki_compactor_oldest_pending_delete_request_age_seconds > 86400\n        for: 6h\n        labels:\n          severity: info\n        annotations:\n          summary: \"Old Loki delete requests are still pending\"\n```\n\n### Grafana Dashboard Queries\n\nCreate a retention monitoring dashboard:\n\n```sql\n-- Deleted lines over time\nsum(rate(loki_compactor_deleted_lines[$__rate_interval])) by (user)\n\n-- Pending delete requests\nsum(loki_compactor_pending_delete_requests_count)\n\n-- Delete requests processed over time\nsum(rate(loki_compactor_delete_requests_processed_total[$__rate_interval])) by (user)\n\n-- Oldest pending delete request age\nmax(loki_compactor_oldest_pending_delete_request_age_seconds)\n```\n\n## Production Deployment Patterns\n\n### Single Binary with Retention\n\nFor smaller deployments, run everything in one process:\n\n```yaml\n# docker-compose.yml\nservices:\n  loki:\n    image: grafana/loki:3.7.1\n    ports:\n      - \"3100:3100\"\n    volumes:\n      - ./loki-config.yaml:/etc/loki/config.yaml\n      - ./runtime-config.yaml:/loki/runtime-config.yaml\n      - loki-data:/loki\n    command: -config.file=/etc/loki/config.yaml -target=all\n\nvolumes:\n  loki-data:\n```\n\n### Dedicated Compactor in Microservices Mode\n\nFor larger deployments, run the compactor as a dedicated service:\n\n```yaml\n# kubernetes/compactor-deployment.yaml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: loki-compactor\nspec:\n  replicas: 1  # Only one compactor instance\n  selector:\n    matchLabels:\n      app: loki-compactor\n  template:\n    metadata:\n      labels:\n        app: loki-compactor\n    spec:\n      containers:\n        - name: compactor\n          image: grafana/loki:3.7.1\n          args:\n            - -config.file=/etc/loki/config.yaml\n            - -target=compactor\n          ports:\n            - containerPort: 3100\n              name: http\n          volumeMounts:\n            - name: config\n              mountPath: /etc/loki\n            - name: data\n              mountPath: /loki\n          resources:\n            requests:\n              cpu: 500m\n              memory: 1Gi\n            limits:\n              cpu: 2000m\n              memory: 4Gi\n      volumes:\n        - name: config\n          configMap:\n            name: loki-config\n        - name: data\n          persistentVolumeClaim:\n            claimName: loki-compactor-data\n```\n\nThe compactor must run as a singleton. Running multiple instances causes data corruption.\n\n### Helm Chart Configuration\n\nUsing the official Loki Helm chart:\n\n```yaml\n# values.yaml\nloki:\n  auth_enabled: false\n\n  commonConfig:\n    replication_factor: 1\n\n  storage:\n    type: s3\n    bucketNames:\n      chunks: loki-chunks\n      ruler: loki-ruler\n      admin: loki-admin\n    s3:\n      endpoint: s3.us-east-1.amazonaws.com\n      region: us-east-1\n\n  storage_config:\n    aws:\n      region: us-east-1\n      bucketnames: loki-chunks\n      s3forcepathstyle: false\n\n  schemaConfig:\n    configs:\n      - from: 2024-01-01\n        store: tsdb\n        object_store: s3\n        schema: v13\n        index:\n          prefix: loki_index_\n          period: 24h\n\n  limits_config:\n    retention_period: 720h\n\n  compactor:\n    retention_enabled: true\n    delete_request_store: s3\n    retention_delete_delay: 2h\n    retention_delete_worker_count: 150\n\n  # Runtime configuration for per-tenant overrides\n  runtimeConfig:\n    overrides:\n      team-a:\n        retention_period: 168h\n      team-b:\n        retention_period: 2160h\n\ncompactor:\n  replicas: 1\n  persistence:\n    enabled: true\n    claims:\n      - name: data\n        size: 10Gi\n```\n\n## Troubleshooting Retention Issues\n\n### Logs Not Being Deleted\n\nCheck the compactor is running and retention is enabled:\n\n```bash\n# Check compactor status\ncurl http://loki:3100/compactor/ring\n\n# Check if retention is enabled in config\ncurl http://loki:3100/config | grep -A 10 \"compactor\"\n```\n\nCommon causes:\n\n-`retention_enabled: false`in config\n- Compactor not reaching the chunks (network or permission issues)\n- Delete delay has not passed yet\n\n### Debugging Retention Boundaries\n\nVerify what the compactor sees as retention boundaries:\n\n```bash\n# Check compactor logs for retention decisions\nkubectl logs deployment/loki-compactor | grep \"retention boundary\"\n```\n\n### Storage Not Decreasing\n\nIf storage does not decrease after retention runs:\n\n1. Check if lifecycle policies are conflicting\n2. Verify the compactor has write access to delete objects\n3. Look for orphaned chunks from failed compactions\n\n```bash\n# List objects that should have been deleted\naws s3 ls s3://loki-chunks/chunks/ --recursive | \\\n  awk '$1 < \"2024-01-01\" {print $4}'\n```\n\n### Per-Tenant Overrides Not Applied\n\nVerify runtime config is loaded:\n\n```bash\n# Check runtime config hash\ncurl http://loki:3100/runtime_config?mode=diff\n```\n\nEnsure the tenant ID matches exactly (case-sensitive).\n\n## Best Practices Summary\n\n1. **Start with compactor retention** - Table Manager is deprecated and has limited functionality.\n2. **Set conservative defaults** - Start with longer retention and reduce based on observed usage patterns.\n3. **Use stream selectors wisely** - Apply longer retention only to high-value logs like errors and audit trails.\n4. **Monitor compaction lag** - A backlog indicates the compactor cannot keep up with ingestion.\n5. **Combine with lifecycle policies** - Use cloud provider policies as a safety net, not the primary mechanism.\n6. **Test deletion in staging** - Verify retention policies work correctly before applying to production.\n7. **Document retention policies** - Teams need to know how long their logs persist for debugging and compliance.\n8. **Plan for growth** - Ingestion typically grows faster than deletion, so build in headroom.\n\n## Conclusion\n\nImplementing effective retention in Loki requires configuring the compactor correctly and understanding how chunks and indexes interact. Start with the compactor-based approach using TSDB indexes for new deployments. Use per-tenant overrides to give teams flexibility while maintaining cost control. Monitor retention effectiveness with dedicated dashboards and alerts to catch issues before storage costs spiral.\n\nThe configurations in this guide provide a solid foundation. Adjust retention periods based on your compliance requirements, debugging needs, and budget constraints. Remember that retention is a balance - too short and you lose valuable debugging context, too long and storage costs become unsustainable.\n\nShare this article\n\n![Nawaz Dhandala](https://avatars.githubusercontent.com/nawazdhandala)\n\n### Nawaz Dhandala\n\nAuthor\n\n@nawazdhandala \u2022 Jan 30, 2026 \u2022\n\nNawaz is building OneUptime with a passion for engineering reliable systems and improving observability.\n\nTechnically validated\n\n\u00b7 Jun 11, 2026\n\nView report\n\n### Help improve this post\n\nEvery OneUptime blog post is open source. Found a typo, an inaccuracy, or have a clearer way to explain something? Anyone can contribute \u2014 your edits make this post better for everyone who reads it next.",
        "description": "Configure Grafana Loki retention policies with compactor-based deletion, per-tenant limits, and storage tiering for cost-effective log management.",
        "source": "web",
        "price": 0.0015,
        "length": 21354,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2026-01-30",
        "crawl_date": "2026-10-06",
        "id": "https://oneuptime.com/blog/post/2026-01-30-loki-retention-strategies/view",
        "image_url": {
          "0": "https://oneuptime.com/blog/post/2026-01-30-loki-retention-strategies/social-media.png",
          "1": "https://oneuptime.com/img/hou-wb.svg"
        },
        "relevance_score": 0.9127527722060989
      },
      {
        "title": "Logs retention doesn't work \u00b7 Issue #8357 \u00b7 grafana/loki \u00b7 GitHub",
        "url": "https://github.com/grafana/loki/issues/8357",
        "content": "[grafana](https://github.com/grafana)  /  **[loki](https://github.com/grafana/loki)**\n\nPublic\n\n- You must be signed in to change notification settings\n- [4.1k](https://github.com/login?return_to=%2Fgrafana%2Floki)\n- [Star 29k](https://github.com/login?return_to=%2Fgrafana%2Floki)\n\n# Logs retention doesn't work\u00a0#8357\n\nNew issue\n\nNew issue\n\n[Logs retention doesn't work](https://github.com/grafana/loki/issues/8357#top)#8357\n\n## Description\n\n[fpaparoni](https://github.com/fpaparoni)opened[on Jan 31, 2023](https://github.com/grafana/loki/issues/8357#issue-1564748865)\n\n**Describe the bug**\n\nUsing a custom configuration I think the logs should be deleted after 1h, but they are always available\n\n**To Reproduce**\n\nI used the following configuration values for loki helm chart\n\n```\nloki:\n  config:\n    limits_config:\n      retention_period: 1h\n    compactor:\n      retention_enabled: true\n      retention_delete_delay: 1h\n    table_manager:\n      retention_deletes_enabled: true\n      retention_period: 1h\n```\n\nlaunched the creation using the following command\n\n`helm install loki grafana/loki-stack --values loki-stack-values.yaml -n loki --create-namespace`\n\nand if I enter into the loki pod I can see the following configuration from /etc/loki/loki.yaml\n\n```\nauth_enabled: false\nchunk_store_config:\n  max_look_back_period: 0s\ncompactor:\n  retention_delete_delay: 1h\n  retention_enabled: true\n  shared_store: filesystem\n  working_directory: /data/loki/boltdb-shipper-compactor\ningester:\n  chunk_block_size: 262144\n  chunk_idle_period: 3m\n  chunk_retain_period: 1m\n  lifecycler:\n    ring:\n      replication_factor: 1\n  max_transfer_retries: 0\n  wal:\n    dir: /data/loki/wal\nlimits_config:\n  enforce_metric_name: false\n  max_entries_limit_per_query: 5000\n  reject_old_samples: true\n  reject_old_samples_max_age: 168h\n  retention_period: 1h\nmemberlist:\n  join_members:\n  - 'loki-memberlist'\nschema_config:\n  configs:\n  - from: \"2020-10-24\"\n    index:\n      period: 24h\n      prefix: index_\n    object_store: filesystem\n    schema: v11\n    store: boltdb-shipper\nserver:\n  grpc_listen_port: 9095\n  http_listen_port: 3100\nstorage_config:\n  boltdb_shipper:\n    active_index_directory: /data/loki/boltdb-shipper-active\n    cache_location: /data/loki/boltdb-shipper-cache\n    cache_ttl: 24h\n    shared_store: filesystem\n  filesystem:\n    directory: /data/loki/chunks\ntable_manager:\n  retention_deletes_enabled: true\n  retention_period: 1h\n```\n\n**Expected behavior**\n\nAfter one hour logs should be deleted, but the retention doesn't work (of course there is something in the configuration that is wrong)\n\n**Environment:**\n\n- Infrastructure: Kubernetes\n- Deployment tool: helm\n\n## Activity\n\n[Sign up for free](https://github.com/signup?return_to=https://github.com/grafana/loki/issues/8357)[Sign in to comment](https://github.com/login?return_to=https://github.com/grafana/loki/issues/8357)\n\n## Metadata\n\n## Metadata\n\n### Assignees\n\nNo one assigned\n\n### Labels\n\nNo labels\n\nNo labels\n\n### Type\n\nNo type\n\n### Projects\n\nNo projects\n\n### Milestone\n\nNo milestone\n\n### Relationships\n\nNone yet\n\n### Development\n\nNo branches or pull requests\n\n## Issue actions\n\n- ![](https://github.githubassets.com/assets/github-copilot-app-light-15ad5534265eeacd.svg)Open in GitHub Copilot app",
        "description": "Describe the bug Using a custom configuration I think the logs should be deleted after 1h, but they are always available To Reproduce I used the following configuration values for loki helm chart loki: config: limits_config: retention_pe...",
        "source": "web",
        "price": 0.0015,
        "length": 3261,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2023-01-31",
        "crawl_date": "2026-10-06",
        "id": "https://github.com/grafana/loki/issues/8357",
        "image_url": {
          "0": "https://opengraph.githubassets.com/1430d0bf007ff2bd9850045648c8d44c1eaf3a7ab55d92ed04f5b927fb27f653/grafana/loki/issues/8357"
        },
        "relevance_score": 0.8900849291846026
      },
      {
        "title": "Log retention | Grafana Loki documentation",
        "url": "https://grafana.com/docs/loki/latest/operations/storage/retention/",
        "content": "Menu\n\n[Documentation](https://grafana.com/docs/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Grafana Loki](https://grafana.com/docs/loki/latest/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Manage](https://grafana.com/docs/loki/latest/operations/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)[Storage](https://grafana.com/docs/loki/latest/operations/storage/)![breadcrumb arrow](https://grafana.com/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg)Log retention\n\nOpen source\n\n# Log retention\n\nRetention in Grafana Loki is achieved through the Compactor. By default the `compactor.retention-enabled` flag is not set, so the logs sent to Loki live forever.\n\n> NoteIf you have a lifecycle policy configured on the object store, please ensure that it is longer than the retention period.\n\nGranular retention policies to apply retention at per tenant or per stream level are also supported by the Compactor.\n\n> NoteThe Compactor does not support retention on > legacy index types. Please use the > Table Manager when using legacy index types. > Both the Table manager and legacy index types are deprecated and may be removed in future major versions of Loki.\n\n## Compactor\n\nThe Compactor is responsible for compaction of index files and applying log retention.\n\n> NoteRun the Compactor as a singleton (a single instance).\n\nThe Compactor runs compaction and retention on two independent schedules:\n\n- Compaction runs every `compactor.compaction-interval`.\n- Retention runs every `compactor.apply-retention-interval`. If you leave this at its default of `0s`, Loki sets it to the same value as `compactor.compaction-interval`. Whenever the two intervals are equal, Loki extends the retention interval by the smaller of `10m` or half the interval, so that compaction and retention do not run at the same time. For example, with the default `compaction_interval` of `10m`, retention runs every `15m`.\n\nIf the Compactor falls behind on either schedule, it runs the corresponding operation as soon as possible. Both compaction and retention are idempotent, which means once the action has been performed, if the action is performed multiple times, it has no further effect on logs after the first time it is performed. If the Compactor restarts, it continues from where it left off.\n\n> NoteChanges to your retention period are not retroactive, that is, they are not applied to logs that have already been ingested.\n\nThe Compactor\u2019s algorithm to apply retention is as follows:\n\n- For each day or table (one table per day with 24h index period): - Compact multiple index files in the table into per-tenant index files. Result of compaction is a single index file per tenant per day.\n- Traverse the per-tenant index. Use the tenant configuration to identify the chunks that need to be removed.\n- Remove the references to the matching chunks from the index and add the chunk references to a marker file on disk.\n- Upload the new modified index files.\n\nChunks are not deleted while applying the retention algorithm on the index. They are deleted asynchronously by a sweeper process and this delay can be configured by setting `-compactor.retention-delete-delay`. Marker files are used to keep track of the chunks pending for deletion.\n\nChunks cannot be deleted immediately for the following reasons:\n\n- Index Gateway downloads a copy of the index files to serve queries and refreshes them at a regular interval.\nHaving a delay allows the index gateways to pull the modified index file which would not contain any reference to the chunks marked for deletion.\nWithout the delay, index files (that are stale) on the gateways could refer to already deleted chunks leading to query failures.\n- It provides a short window of time in which to cancel chunk deletion in the case of a configuration mistake.\n\nBy default, marker files are written to local disk, under `<working_directory>/retention/<object_store_type>_<period_from_date>/markers/`. Marker files should be stored on a persistent disk to ensure that the chunks pending for deletion are processed even if the Compactor process restarts.\n\n> NoteGrafana Labs recommends running Compactor as a stateful deployment (StatefulSet when using Kubernetes) with a persistent storage for storing marker files.\n\nAlternatively, you can configure the Compactor to store marker files in object storage instead of local disk, by setting `compactor.deletion-marker-object-store-prefix` (`deletion_marker_object_store_prefix`) to a prefix that ends with `/`. When set, the Compactor writes marker files to the same per-period object store bucket used for chunks, under that prefix. On startup it also copies any existing marker files from local disk to that prefix, then removes the local copies. This lets you run the Compactor without a persistent volume for marker files. Leave this setting empty to keep using local disk.\n\n### Retention Configuration\n\nThis Compactor configuration example activates retention.\n\nYAML ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy\n\n```yaml\ncompactor:\n  working_directory: /data/retention\n  compaction_interval: 10m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  retention_delete_worker_count: 150\n  delete_request_store: gcs\nschema_config:\n  configs:\n    - from: \"2020-07-31\"\n      index:\n        period: 24h\n        prefix: index_\n      object_store: gcs\n      schema: v13\n      store: tsdb\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /data/index\n    cache_location: /data/index_cache\n  gcs:\n    bucket_name: loki\n```\n\n> NoteRetention is only available if the index period is 24h. Single store TSDB and single store BoltDB require 24h index period.\n\n`retention_enabled` should be set to true. Without this, the Compactor will only compact tables.\n\n`delete_request_store` should be set to configure the store for delete requests. This is required when retention is enabled.\n\n`working_directory` is the directory where marked chunks and temporary tables will be saved.\n\n`compaction_interval` dictates how often compaction is applied. If the Compactor falls behind, compaction occurs as soon as possible.\n\n`apply_retention_interval` dictates how often retention is applied, independently of `compaction_interval`. Refer to Compactor for how the default value is derived from `compaction_interval`.\n\n`retention_delete_delay` is the delay after which the Compactor will delete marked chunks.\n\n`retention_delete_worker_count` specifies the maximum quantity of goroutine workers instantiated to delete chunks.\n\nA few other options are also relevant when tuning retention:\n\n- `retention_table_timeout` bounds how long the Compactor spends applying retention and processing delete requests on any single table. It defaults to `0s`, meaning no timeout. If the work on a table exceeds this timeout, the Compactor logs a warning and stops early rather than failing, and the remaining work is picked up again on a later run.\n- `retention_backoff_config` controls the retry behavior (minimum and maximum backoff, and number of retries) used by the sweeper when it fails to delete a marked chunk.\n- `delete_request_store_key_prefix` sets the path prefix used to store delete requests within `delete_request_store` (default `index/`).\n- `delete_request_store_db_type` sets the type of database used to store delete requests (default `boltdb`).\n\nRefer to the\n[Configuration reference](https://grafana.com/docs/loki/latest/configure/#compactor) for the full list of `compactor` options and their defaults.\n\n#### Configuring the retention period\n\nRetention period is configured within the\n[limits_config](https://grafana.com/docs/loki/latest/configure/#limits_config) configuration section.\n\nThere are two ways of setting retention policies:\n\n- `retention_period` which is applied globally for all log streams.\n- `retention_stream` which is only applied to log streams matching the selector.\n\n> NoteEach retention_stream rule\u2019s period must be at least 24h. Loki rejects configuration with a shorter value, both in the global retention_stream list and in any per-tenant override. This minimum does not apply to the retention_period setting, either globally or per tenant.\n\nThis example configures global retention that applies to all tenants (unless overridden by configuring per-tenant overrides):\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\n...\nlimits_config:\n  retention_period: 744h\n  retention_stream:\n  - selector: '{namespace=\"dev\"}'\n    priority: 1\n    period: 24h\n  per_tenant_override_config: /etc/overrides.yaml\n...\n```\n\n> NoteYou can only use label matchers in the selector field of a retention_stream definition. Arbitrary LogQL expressions are not supported.\n\nPer tenant retention can be defined by configuring\n[runtime overrides](https://grafana.com/docs/loki/latest/configure/#runtime-configuration-file). For example:\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\noverrides:\n    \"29\":\n        retention_period: 168h\n        retention_stream:\n        - selector: '{namespace=\"prod\"}'\n          priority: 2\n          period: 336h\n        - selector: '{container=\"loki\"}'\n          priority: 1\n          period: 72h\n    \"30\":\n        retention_stream:\n        - selector: '{container=\"nginx\", level=\"debug\"}'\n          priority: 1\n          period: 24h\n```\n\nRetention period for a given stream is decided based on the first match in this list:\n\n1. If multiple per-tenant `retention_stream` selectors match the stream, retention period with the highest priority is picked.\n2. If multiple global `retention_stream` selectors match the stream, retention period with the highest priority is picked. This value is not considered if per-tenant `retention_stream` is set.\n3. If a per-tenant `retention_period` is specified, it will be applied.\n4. The global `retention_period` will be applied if none of the above match.\n5. If no global `retention_period` is specified, the default value of `0s` is used, which means logs are kept indefinitely.\n\n> NoteThe larger the priority value, the higher the priority. If two matching retention_stream rules have the same priority, Loki applies the shorter (lowest) of the two periods.\n\nStream matching uses the same syntax as Prometheus label matching:\n\n- `=`: Select labels that are exactly equal to the provided string.\n- `!=`: Select labels that are not equal to the provided string.\n- `=~`: Select labels that regex-match the provided string.\n- `!~`: Select labels that do not regex-match the provided string.\n\nThe example configurations defined above will result in the following retention periods:\n\n- For tenant `29`: - Streams that have the namespace label `prod` will have a retention period of `336h` (2 weeks), even if the container label is `loki`, since the priority of the `prod` rule is higher.\n- Streams that have the container label `loki` but are not in the namespace `prod` will have a `72h` retention period.\n- For the rest of the streams in this tenant, per-tenant override `retention_period` value of `168h` is applied.\n- For tenant `30`: - Streams that have the label `nginx` and level `debug` will have a retention period of `24h`.\n- For the rest of the streams in this tenant the global retention period of `744h`, since there is no override specified.\n- All tenants except `29` and `30`: - Streams that have the namespace label `dev` will have a retention period of `24h` hours.\n- Streams except those with the namespace label `dev` will have the retention period of `744h`.\n\n> NoteIf you are a Grafana Cloud customer, you can use the config self-serve API to configure your tenant retention.\n\n## Table Manager (deprecated)\n\nRetention through the\n[Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/) is\nachieved by relying on the object store TTL feature, and will work for both\n[boltdb-shipper](https://grafana.com/docs/loki/latest/operations/storage/boltdb-shipper/) store and chunk/index stores.\n\nIn order to enable the retention support, the Table Manager needs to be\nconfigured to enable deletions and a retention period. Please refer to the\n[table_manager](https://grafana.com/docs/loki/latest/configure/#table_manager)\nsection of the Loki configuration reference for all available options.\nAlternatively, the `table-manager.retention-period` and\n`table-manager.retention-deletes-enabled` command line flags can be used. The\nprovided retention period needs to be a duration represented as a string that\ncan be parsed using the Prometheus common model [ParseDuration](https://pkg.go.dev/github.com/prometheus/common/model#ParseDuration). Examples: `7d`, `1w`, `168h`.\n\n> WarningThe retention period must be a multiple of the index and chunks table\n> period, configured in the\n> period_config block.\n> See the\n> Table Manager documentation for\n> more information.\n\n> NoteTo avoid querying of data beyond the retention period,max_query_lookback config in\n> limits_config must be set to a value less than or equal to what is set in table_manager.retention_period.\n\nWhen using S3 or GCS, the bucket storing the chunks needs to have the expiry\npolicy set correctly. For more details check\n[S3\u2019s documentation](https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html)\nor\n[GCS\u2019s documentation](https://cloud.google.com/storage/docs/managing-lifecycles).\n\nIf you must delete ingested logs, you can delete old chunks in your object store. Note,\nhowever, that this only deletes the log content and keeps the label index\nintact; you will still be able to see related labels but will be unable to\nretrieve the deleted log content.\n\nFor further details on the Table Manager internals, refer to the\n[Table Manager](https://grafana.com/docs/loki/latest/operations/storage/table-manager/) documentation.\n\nAlternatively, if the BoltDB Shipper is configured for the index store, you can enable\n[Log entry deletion](https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/) to delete log entries from a specific stream.\n\n## Example Configuration\n\nExample configuration with GCS with a 28 day retention:\n\nYAML\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```yaml\nschema_config:\n  configs:\n    - from: 2018-04-15\n      store: tsdb\n      object_store: gcs\n      schema: v13\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/index\n    cache_location: /loki/index_cache\n  gcs:\n    bucket_name: GCS_BUCKET_NAME\n\nlimits_config:\n  max_query_lookback: 672h # 28 days\n  retention_period: 672h   # 28 days\n\ncompactor:\n  working_directory: /data/retention\n  delete_request_store: gcs\n  retention_enabled: true\n```\n\n## Object store lifecycle policies\n\nLoki stores chunks and indexes as objects in the configured object store (S3, GCS, Azure Blob Storage, and so on). Chunks are deleted by the Compactor according to your configured retention, but the Compactor only removes chunk objects: it never deletes the objects that hold cluster state, such as the index or the cluster seed.\n\nIf you also configure an object store lifecycle policy on the bucket, scope it carefully. A blanket \u201cdelete everything older than N days\u201d rule will eventually remove objects that Loki must keep, which corrupts the store and causes query failures. The note at the top of this page applies here as well: any lifecycle expiration must be longer than your retention period.\n\n> WarningNever apply a lifecycle rule with an empty prefix (one that targets the whole bucket). Always scope lifecycle rules to the per-tenant chunk prefixes described below.\n\n### Bucket object layout\n\nThe following objects are written to the configured bucket. The exact prefixes depend on your `schema_config` and on whether multi-tenancy is enabled, so confirm them against your own configuration before writing a lifecycle rule.\n\nExpand table\n\n| Object or prefix | Content | Safe to expire with a lifecycle policy? |\n| --- | --- | --- |\n| `<tenant_id>/` (for example, `fake/` in single-tenant mode, or your org ID per tenant) | Chunk objects (the log data). | Yes. These are the only objects a lifecycle rule should target. |\n| Index objects (under the index path prefix, `index/` by default, set by `index.path_prefix` in `schema_config`) | TSDB or BoltDB index files, written by ingesters and rewritten by the Compactor. The `index.prefix` value (for example `index_` or `loki_index_`) is the table-name prefix used inside this path, not the path itself, so match a lifecycle rule against the path prefix. | No. Loki manages the index lifecycle internally. Deleting index objects removes references to live chunks and breaks queries. |\n| `loki_cluster_seed.json` | A small file at the bucket root holding the cluster seed used by usage reporting. It is written once and never rotated. | No. A blanket age-based rule will delete it once it ages past the time to live (TTL). |\n| Delete-request store objects (a `delete_requests/` table under `delete_request_store_key_prefix`, `index/` by default, in the bucket configured by `delete_request_store`) | The Compactor\u2019s database of delete requests submitted through the [Log entry deletion](https://grafana.com/docs/loki/latest/operations/storage/logs-deletion/) API, and their processing status. | No. Removing these makes the Compactor lose track of pending delete requests. |\n| Deletion marker objects (only present if you set `deletion_marker_object_store_prefix`; otherwise markers are kept on the Compactor\u2019s local disk and never appear in the bucket) | The Compactor\u2019s record of chunks that are marked for deletion but not yet swept. Written under the configured prefix in the same per-period bucket used for chunks. | No. Removing these makes the Compactor lose track of chunks pending deletion. |\n| Ruler store objects (if the Ruler is backed by the same bucket) | Recording and alerting rule files. | No. These are configuration, not time-series data. |\n\n### Compactor retention versus object store lifecycle policies\n\nCompactor retention and object store lifecycle policies are two independent mechanisms, and they should not be configured in ways that conflict:\n\n- The Compactor is index-aware. It only deletes a chunk after it has removed every reference to that chunk from the index, and it waits `compactor.retention-delete-delay` (default `2h`) before the sweeper deletes the marked chunk, so that index gateways pick up the rewritten index first.\n- An object store lifecycle policy is index-unaware. It deletes objects purely by age and prefix, with no knowledge of whether a chunk is still referenced by the index.\n\nFor most deployments, enabling Compactor retention is sufficient and an object store lifecycle policy is not required. If you do add a lifecycle policy as a cost safety net, set its expiration **longer** than your retention period plus `compactor.retention-delete-delay`, so the Compactor always deletes chunks first and the lifecycle rule only ever catches objects the Compactor has already orphaned.\n\n### Example: Amazon S3 lifecycle policy\n\nThis policy expires only chunk objects under a single tenant prefix. Replace `fake/` with your tenant prefix (use one rule per tenant, or your org ID prefix, when multi-tenancy is enabled). The `Days` value must be larger than your retention period plus the delete delay.\n\nJSON ![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg) Copy\n\n```javascript\n{\n  \"Rules\": [\n    {\n      \"ID\": \"loki-chunk-expiration\",\n      \"Status\": \"Enabled\",\n      \"Filter\": {\n        \"Prefix\": \"fake/\"\n      },\n      \"Expiration\": {\n        \"Days\": 395\n      }\n    }\n  ]\n}\n```\n\nApply it with the AWS CLI:\n\nBash\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```bash\naws s3api put-bucket-lifecycle-configuration \\\n  --bucket YOUR_LOKI_BUCKET \\\n  --lifecycle-configuration file://lifecycle.json\n```\n\n### Example: Google Cloud Storage lifecycle rule\n\nThe equivalent GCS rule expires objects by age and prefix. As with S3, scope `matchesPrefix` to the tenant chunk prefix and keep `age` larger than your retention period plus the delete delay.\n\nJSON\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```javascript\n{\n  \"rule\": [\n    {\n      \"action\": {\"type\": \"Delete\"},\n      \"condition\": {\n        \"age\": 395,\n        \"matchesPrefix\": [\"fake/\"]\n      }\n    }\n  ]\n}\n```\n\nApply it with the Google Cloud CLI:\n\nBash\n![Copy code to clipboard](https://grafana.com/media/images/icons/icon-copy-small-2.svg)\nCopy\n\n```bash\ngcloud storage buckets update gs://YOUR_LOKI_BUCKET --lifecycle-file=lifecycle.json\n```\n\nFor Azure Blob Storage, use a management policy with a `prefixMatch` set to the tenant chunk prefix and the same \u201cexpiration must exceed retention plus delete delay\u201d rule.\n\n## Was this page helpful?\n\n![\ud83d\udc4d](https://grafana.com/media/images/svg/thumbs-up.svg) Yes\n![\ud83d\udc4e](https://grafana.com/media/images/svg/thumbs-up.svg) No\n\n[Suggest an edit in GitHub](https://github.com/grafana/loki/edit/release-3.7.x/docs/sources/operations/storage/retention.md)[Create a GitHub issue](https://github.com/grafana/loki/issues/new?title=Documentation%20feedback:%20/docs/sources/operations/storage/retention.md)[Email docs@grafana.com](mailto:docs@grafana.com)[Help and support](https://grafana.com/help/)[Community](https://grafana.com/community/)\n\n## Related resources from Grafana Labs\n\nAdditional helpful documentation, links, and articles:\n\n[VideoGetting started with logging and Grafana LokiSee a demo of the updated features in Loki, and how to create metrics from logs and alert on your logs with powerful Prometheus-style alerting rules.](https://grafana.com/go/webinar/getting-started-with-logging-and-grafana-loki/)[VideoEssential Grafana Loki configuration settingsThis webinar focuses on Grafana Loki configuration including agents Promtail and Docker; the Loki server; and Loki storage for popular backends.](https://grafana.com/go/webinar/logging-with-loki-essential-configuration-settings/)[VideoScaling and securing your logs with Grafana LokiThis webinar covers the challenges of scaling and securing logs, and how Grafana Cloud Logs powered by Grafana Loki can help, cost-effectively.](https://grafana.com/go/webinar/scaling-and-securing-your-logs-with-grafana-loki/)",
        "description": "Describes how Loki implements storage retention and retention configuration options.",
        "source": "web",
        "price": 0.0015,
        "length": 22620,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2023-04-12",
        "crawl_date": "2026-10-06",
        "id": "https://grafana.com/docs/loki/latest/operations/storage/retention/",
        "image_url": {
          "0": "https://grafana.com/meta-generator/Log+retention@@@loki@@@13.png"
        },
        "relevance_score": 0.8826547784020347
      },
      {
        "title": "Log Retention | Compile N Run",
        "url": "https://www.compilenrun.com/docs/observability/loki/advanced-features/log-retention/",
        "content": "On this page\n\n# Log Retention\n\n## Introduction\u200b\n\nLog retention is a critical aspect of any logging system, including Grafana Loki. It refers to how long your logs are stored before being automatically deleted or archived. Understanding and configuring log retention properly helps you:\n\n- Control storage costs\n- Maintain system performance\n- Comply with data retention regulations\n- Ensure you have access to logs when you need them\n\nIn this guide, we'll explore how log retention works in Grafana Loki, how to configure retention policies, and best practices for managing your log data effectively.\n\n## Understanding Log Retention in Loki\u200b\n\nLoki's approach to log retention is centered around the concept of \"chunks\" - compressed batches of log data that are stored and managed together. Log retention in Loki involves deciding when these chunks should be deleted or moved to different storage tiers.\n\n### Why Log Retention Matters\u200b\n\nWithout proper retention policies:\n\n1. **Storage costs can skyrocket**: Logs accumulate quickly in production environments\n2. **Query performance degrades**: More data means slower queries\n3. **Compliance risks increase**: Many industries have specific requirements for how long logs must be kept\n\n## Basic Retention Configuration\u200b\n\nLoki supports several methods for configuring retention:\n\n1. **Time-based retention**: Delete logs older than a specified period\n2. **Size-based retention**: Limit the total storage used for logs\n3. **Stream-based retention**: Set different retention policies for different log streams\n\nLet's look at how to configure each approach.\n\n### Time-Based Retention\u200b\n\nTime-based retention is the most common approach. Here's how to configure it in your Loki configuration file:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # 31 days\n```\n\nThis configuration tells Loki to keep logs for 31 days (744 hours) before deleting them.\n\nYou can also configure retention periods for specific tenants:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # Default retention period\n\n  per_tenant_override_config: /etc/loki/retention-overrides.yaml\n```\n\nWith a`retention-overrides.yaml`file that looks like:\n\n```yaml\noverrides:\n\n  \"tenant1\":\n\n    retention_period: 1440h # 60 days\n\n  \"tenant2\":\n\n    retention_period: 168h  # 7 days\n```\n\n### Size-Based Retention\u200b\n\nTo limit storage based on size:\n\n```yaml\ncompactor:\n\n  retention_enabled: true\n\n\n\nlimits_config:\n\n  retention_period: 0   # 0 means no time-based deletion\n\n  retention_stream_size_mb: 100  # Max size per stream\n\n  retention_total_size_mb: 10000  # Max total size\n```\n\nIn this example, logs will be deleted when:\n\n- A single stream exceeds 100MB\n- Total log storage exceeds 10GB\n\n### Stream-Based Retention with Labels\u200b\n\nYou can set different retention periods for different log streams using Loki's stream selector syntax:\n\n```yaml\nlimits_config:\n\n  retention_period: 744h # Default retention period\n\n  retention_stream_selectors:\n\n    - '{app=\"critical-service\"}': 8760h  # 1 year\n\n    - '{env=\"development\"}': 72h        # 3 days\n\n    - '{namespace=\"monitoring\"}': 2160h  # 90 days\n```\n\n## Implementing Retention in a Microservices Architecture\u200b\n\nLet's look at a practical example of configuring retention in a microservices environment:\n\nIn this architecture, we want to configure different retention policies:\n\n1. Frontend logs (high volume, less critical): 7 days\n2. Backend logs (medium volume, important): 30 days\n3. Database logs (low volume, critical): 1 year\n\nHere's the configuration:\n\n```yaml\nlimits_config:\n\n  retention_period: 168h # Default: 7 days\n\n  retention_stream_selectors:\n\n    - '{component=\"backend\"}': 720h     # 30 days\n\n    - '{component=\"database\"}': 8760h   # 365 days\n```\n\n## Best Practices for Log Retention\u200b\n\n### 1. Align with Business Requirements\u200b\n\nConfigure retention based on:\n\n- Compliance requirements (GDPR, HIPAA, SOX, etc.)\n- Incident investigation needs\n- Audit requirements\n- Debugging needs\n\n### 2. Consider a Tiered Storage Approach\u200b\n\nLoki supports moving older logs to cheaper storage:\n\n```yaml\nschema_config:\n\n  configs:\n\n    - from: 2022-01-01\n\n      store: boltdb-shipper\n\n      object_store: filesystem\n\n      schema: v11\n\n      index:\n\n        prefix: index_\n\n        period: 24h\n\nstorage_config:\n\n  boltdb_shipper:\n\n    active_index_directory: /loki/index\n\n    cache_location: /loki/cache\n\n    cache_ttl: 24h\n\n    shared_store: s3\n\n\n\n  filesystem:\n\n    directory: /loki/chunks\n\n\n\n  aws:\n\n    s3: s3://bucket/prefix\n\n    region: us-west-2\n```\n\nThis configuration stores recent logs in the filesystem for fast access and moves older logs to S3 for cost-effective long-term storage.\n\n### 3. Optimize for Query Patterns\u200b\n\nIf you frequently query only recent logs, set a shorter retention period for fast queries and keep longer retention for specific critical components.\n\n### 4. Monitor Storage Usage\u200b\n\nSet up alerts for storage usage to avoid unexpected costs:\n\n```yaml\n# In your Prometheus alerting rules\n\n- alert: LokiHighStorageUsage\n\n  expr: sum(loki_ingester_memory_chunks) > 1000000\n\n  for: 10m\n\n  labels:\n\n    severity: warning\n\n  annotations:\n\n    summary: \"High number of chunks in memory\"\n\n    description: \"Loki has a high number of chunks in memory, which might indicate retention policies need adjustment.\"\n```\n\n## Deletion vs. Compaction\u200b\n\nLoki supports two main approaches to managing old data:\n\n1. **Deletion**: Completely removes old logs\n2. **Compaction**: Combines and compresses old logs to reduce storage needs\n\nTo enable compaction:\n\n```yaml\ncompactor:\n\n  working_directory: /loki/compactor\n\n  shared_store: s3\n\n  compaction_interval: 24h\n```\n\n## Implementing a Log Lifecycle Policy\u200b\n\nA complete log management strategy often involves multiple stages:\n\n1. **Hot storage** (0-7 days): Kept in memory/local storage for fast queries\n2. **Warm storage** (8-30 days): Moved to object storage but still indexed\n3. **Cold storage** (31+ days): Archived with limited indexing\n4. **Deletion**: Removed after retention period\n\nHere's how to implement this in Loki:\n\n```yaml\nschema_config:\n\n  configs:\n\n    - from: 2022-01-01\n\n      store: boltdb-shipper\n\n      object_store: filesystem\n\n      schema: v11\n\n      index:\n\n        prefix: index_\n\n        period: 24h\n\nstorage_config:\n\n  boltdb_shipper:\n\n    active_index_directory: /loki/index/hot\n\n    cache_location: /loki/cache\n\n    cache_ttl: 24h\n\n    shared_store: s3\n\n\n\n  aws:\n\n    s3: s3://loki-bucket\n\n    s3forcepathstyle: true\n\n\n\nstorage_schema:\n\n  - regex: \".*\"\n\n    prefix: loki/\n\n    object_store: s3_hot\n\n    schema: v11\n\n    index:\n\n      enabled: true\n\n      store: boltdb-shipper\n\n      prefix: loki/index/\n\n      period: 24h\n\n  - regex: \".*\"\n\n    prefix: loki_cold/\n\n    object_store: s3_cold\n\n    schema: v11\n\n    index:\n\n      enabled: false\n\nruler:\n\n  storage:\n\n    type: local\n\n    local:\n\n      directory: /loki/rules\n\n  rule_path: /loki/rules-temp\n\n  alertmanager_url: http://alertmanager:9093\n\n  ring:\n\n    kvstore:\n\n      store: inmemory\n\n  enable_api: true\n```\n\n## Common Challenges and Solutions\u200b\n\n### Challenge 1: Balancing Performance and Retention\u200b\n\n**Solution**: Use Loki's stream labels to retain important logs longer while keeping retention shorter for high-volume, less critical logs.\n\n### Challenge 2: Compliance with Varying Requirements\u200b\n\n**Solution**: Configure tenant-specific retention policies that align with each department's compliance needs.\n\n### Challenge 3: Cost Management\u200b\n\n**Solution**: Implement a tiered storage approach with automated movement from expensive to cheaper storage.\n\n## Example: Log Retention in Production\u200b\n\nLet's examine a complete production example with a combination of strategies:\n\n```yaml\nlimits_config:\n\n  # Global defaults\n\n  retention_period: 168h # 7 days default\n\n  # Per-stream retention based on importance\n\n  retention_stream_selectors:\n\n    - '{app=\"payment-service\"}': 2160h    # 90 days for payment logs\n\n    - '{app=\"auth-service\"}': 720h        # 30 days for auth logs\n\n    - '{level=\"error\"}': 720h             # 30 days for all errors\n\n    - '{env=\"development\"}': 72h          # 3 days for dev environment\n\ncompactor:\n\n  working_directory: /loki/compactor\n\n  shared_store: s3\n\n  retention_enabled: true\n\n  retention_delete_delay: 2h\n\n  compaction_interval: 10m\n\n\n\nstorage_config:\n\n  aws:\n\n    s3: s3://logs-bucket\n\n    region: us-west-2\n\n    bucketnames: logs-bucket\n\n    access_key_id: ${AWS_ACCESS_KEY_ID}\n\n    secret_access_key: ${AWS_SECRET_ACCESS_KEY}\n\n    s3forcepathstyle: true\n\n    insecure: false\n```\n\n## Summary\u200b\n\nLog retention in Grafana Loki is a powerful feature that helps you balance storage costs, performance, and compliance requirements. Key takeaways include:\n\n- Configure retention based on time, size, or stream characteristics\n- Consider a tiered storage approach for cost optimization\n- Set different retention periods for different log types based on importance\n- Monitor storage usage to avoid unexpected costs\n- Implement compaction to optimize storage while retaining valuable logs\n\nBy implementing a thoughtful log retention strategy, you can ensure your Loki deployment remains performant, cost-effective, and compliant with your organization's requirements.\n\n## Additional Resources\u200b\n\n- [Official Loki Documentation on Retention](https://grafana.com/docs/loki/latest/operations/storage/retention/)\n- [Storage Best Practices](https://grafana.com/docs/loki/latest/operations/storage/)\n- [Compactor Documentation](https://grafana.com/docs/loki/latest/operations/compactor/)\n\n## Exercises\u200b\n\n1. Configure a basic time-based retention policy for Loki that keeps logs for 14 days.\n2. Create a multi-tier retention policy that keeps error logs for 60 days and all other logs for 7 days.\n3. Set up a monitoring dashboard in Grafana to track Loki storage usage over time.\n4. Design a retention policy that complies with GDPR requirements for a web application.\n\n\ud83d\udca1 Found a typo or mistake? Click **\"Edit this page\"** to suggest a correction. Your feedback is greatly appreciated!\n\n- Introduction\n- Understanding Log Retention in Loki - Why Log Retention Matters\n- Basic Retention Configuration - Time-Based Retention\n- Size-Based Retention\n- Stream-Based Retention with Labels\n- Implementing Retention in a Microservices Architecture\n- Best Practices for Log Retention - 1. Align with Business Requirements\n- 2. Consider a Tiered Storage Approach\n- 3. Optimize for Query Patterns\n- 4. Monitor Storage Usage\n- Deletion vs. Compaction\n- Implementing a Log Lifecycle Policy\n- Common Challenges and Solutions - Challenge 1: Balancing Performance and Retention\n- Challenge 2: Compliance with Varying Requirements\n- Challenge 3: Cost Management\n- Example: Log Retention in Production\n- Summary\n- Additional Resources\n- Exercises",
        "description": "Learn how to configure and manage log retention policies in Grafana Loki to optimize storage and performance",
        "source": "web",
        "price": 0.0015,
        "length": 10758,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2026-10-06",
        "crawl_date": "2026-10-06",
        "id": "https://www.compilenrun.com/docs/observability/loki/advanced-features/log-retention/",
        "image_url": {},
        "relevance_score": 0.8499792177299244
      },
      {
        "title": "r/grafana on Reddit: Working with Loki: Need help retaining logs locally for 1 day, then ship to S3 (MinIO) and delete after 1 week",
        "url": "https://www.reddit.com/r/grafana/comments/1mducu6/working_with_loki_need_help_retaining_logs/",
        "content": "# Working with Loki: Need help retaining logs locally for 1 day, then ship to S3 (MinIO) and delete after 1 week\nRight now, I\u2019m able to see logs getting stored in S3. But the problem is\u2014they don\u2019t stick around long. They\u2019re getting deleted much earlier than I expect, and I'm not sure why.\n\n```\nauth_enabled: false\n\nserver:\n  http_listen_port: 3100\n  log_level: info\n\ncommon:\n  path_prefix: /loki\n  storage:\n    s3:\n      endpoint: http://minio:9000\n      region: us-east-1\n      bucketnames: loki-data\n      access_key_id: admin\n      secret_access_key: password123\n      s3forcepathstyle: true\n      insecure: true\n\n  replication_factor: 1\n  ring:\n    kvstore:\n      store: inmemory\n\nschema_config:\n  configs:\n    - from: 2020-10-24\n      store: tsdb\n      object_store: s3\n      schema: v13\n      index:\n        prefix: index_\n        period: 24h\n\nstorage_config:\n  tsdb_shipper:\n    active_index_directory: /loki/index\n    cache_location: /loki/cache\n    cache_ttl: 30m\n  aws:\n    endpoint: http://minio:9000\n    region: us-east-1\n    bucketnames: loki-data\n    access_key_id: admin\n    secret_access_key: password123\n    s3forcepathstyle: true\n    insecure: true\n\nlimits_config:\n  retention_period: 1h\n\ncompactor:\n  working_directory: /loki/compactor\n  compaction_interval: 30m\n  retention_enabled: true\n  retention_delete_delay: 2h\n  delete_request_store: s3\n\ningester:\n  wal:\n    enabled: true\n  chunk_idle_period: 30m\n  max_chunk_age: 30m\n\nanalytics:\n  reporting_enabled: false\n```\n\n\ud83e\udde9 **Things I\u2019ve noticed / tried:**\n\n* S3 shows data briefly, then it's gone\n\n* `retention_period` under `limits_config` is set to 1h \u2014 maybe that's causing early deletion?\n\n* Not sure how to set *different retention durations* for local disk vs. S3\n\n* I want to make sure logs live:\n\n+ 1 day on local disk\n\n## FaderJockey2600\nYour retention_period indeed dictates the overall retention, so logs get deleted after 1 hour. Loki as far as I\u2019ve experienced does not offer a multi-tiered indexing system like Elastic does. So you cannot configure it to use local and s3 storage for the same data.\n\n## Parley_P_Pratt\nAs others has already pointed out, you cannot configure this. But a work around we do is using memcached to store recent logs in. They still get sent to S3 directly, but recent logs query much faster",
        "description": "# Working with Loki: Need help retaining logs locally for 1 day, then ship to S3 (MinIO) and delete after 1 week\nRight now, I\u2019m able to see logs getting stored in S3. But the problem is\u2014they don\u2019t stick around long. They\u2019re getting deleted much earlier than I expect, and I'm not sure why.",
        "source": "web",
        "price": 0.0015,
        "length": 2301,
        "data_type": "unstructured",
        "source_type": "forum",
        "publication_date": "2025-07-31",
        "crawl_date": "2026-07-28",
        "id": "https://www.reddit.com/r/grafana/comments/1mducu6/working_with_loki_need_help_retaining_logs/",
        "image_url": {},
        "relevance_score": 0.8116806013496
      },
      {
        "title": "r/grafana on Reddit: Increase loki log retention?",
        "url": "https://www.reddit.com/r/grafana/comments/mvrg94/increase_loki_log_retention/",
        "content": "# Increase loki log retention?\n```\nauth_enabled: false\nserver:\n  http_listen_port: 3100\n\ningester:\n  lifecycler:\n    address: 127.0.0.1\n    ring:\n      kvstore:\n        store: inmemory\n      replication_factor: 1\n    final_sleep: 0s\n  chunk_idle_period: 5m\n  chunk_retain_period: 30s\n  max_transfer_retries: 1\n\nschema_config:\n  configs:\n    - from: 2018-04-15\n      store: boltdb-shipper\n      object_store: aws\n      schema: v11\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_config:\n  aws:\n    s3: s3://us-east-1/mylogbucketname\n\n  boltdb_shipper:\n    active_index_directory: /opt/loki/index\n    shared_store: s3\n    cache_location: /opt/loki/cache\n\ncompactor:\n  working_directory: /opt/loki/compactor\n  shared_store: s3\n\nlimits_config:\n  enforce_metric_name: false\n  reject_old_samples: true\n  reject_old_samples_max_age: 168h\n  ingestion_rate_mb: 8\n  ingestion_burst_size_mb: 16\n\nchunk_store_config:\n  max_look_back_period: 0s\n\ntable_manager:\n  retention_deletes_enabled: false\n  retention_period: 0s\n```\n\n## SelfDestructSep2020\nYour problem is the max lookback setting. Set that equal to or greater than your retention if you want to be able to search the full extent of your logs. I'm not entirely sure what happens when it's set to 0s but it might be jumping to a default value in that case.\n\n Loki doesn't actually delete your log chunks either, you need to configure s3 to expire the objects at the same rate as your retention setting.\n\n### Deleted User\n```\n# Limit how long back data can be queried. Default is disabled. \n# This should always be set to a value less than or equal to \n# what is set in `table_manager.retention_period` .\n```\n\nSo that sounds to me like it being set to 0s should disable any limits to look back; it also mentions that it should be less than or equal to the retention period, not greater than the retention period (the retention period is also disabled anyway as it's set to 0s).\n\n### Deleted User \u203a Deleted User\nOk, I worked out the issue. It seems Grafana limits the amount of logs returned in a query to 1000 lines. I limited my search to a single host and a specific log message and I now get results that are hours old.\n\n EDIT: For the record, the line limit can be adjusted in the Loki datasource config. Increasing it may cause a browser performance hit though.",
        "description": "# Increase loki log retention?\n```\nauth_enabled: false\nserver:\n  http_listen_port: 3100\n\ningester:\n  lifecycler:\n    address: 127.0.0.1\n    ring:\n      kvstore:\n        store: inmemory\n      replication_factor: 1\n    final_sleep: 0s\n  chunk_idle_period: 5m\n  chunk_retain_period: 30s\n  max_transfer_retries: 1\n\nschema_config:\n  configs:\n    - from: 2018-04-15\n      store: boltdb-shipper\n      object_store: aws\n      schema: v11\n      index:\n        prefix: loki_index_\n        period: 24h\n\nstorage_",
        "source": "web",
        "price": 0.0015,
        "length": 2330,
        "data_type": "unstructured",
        "source_type": "forum",
        "publication_date": "2021-04-21",
        "crawl_date": "2026-08-06",
        "id": "https://www.reddit.com/r/grafana/comments/mvrg94/increase_loki_log_retention/",
        "image_url": {},
        "relevance_score": 0.8015383574088337
      },
      {
        "title": "How to set up log retention correctly? - Grafana Loki - Grafana Labs Community Forums",
        "url": "https://community.grafana.com/t/how-to-set-up-log-retention-correctly/156719",
        "content": "[Documentation](https://grafana.com/docs/?pg=community&plcmt=topnav)\n\n[Ask Grot AI](https://grafana.com/grot/?pg=community&plcmt=topnav)\n\n[Plugins](https://grafana.com/plugins/?pg=community&plcmt=topnav) [Get Grafana](https://grafana.com/get/?pg=community&plcmt=topnav)",
        "description": "Loki is installed in the Kubernetes cluster using the official Helm chart. \nI am trying to configure automatic log deletion and expect that space will be automatically freed in the chunks bucket on MinIO. \nHowever, with \u2026",
        "source": "web",
        "price": 0.0015,
        "length": 269,
        "data_type": "unstructured",
        "source_type": "website",
        "publication_date": "2025-08-25",
        "crawl_date": "2026-10-06",
        "id": "https://community.grafana.com/t/how-to-set-up-log-retention-correctly/156719",
        "image_url": {
          "0": "https://us1.discourse-cdn.com/grafana/original/2X/a/a7f38198d3aa26d70bae13c3379e5b93a010e7d7.png",
          "1": "https://grafana.com/meta-generator/Log+retention@@@loki@@@7.jpg"
        },
        "relevance_score": 0.6171075383218466
      }
    ],
    "results_by_source": {
      "web": 10,
      "proprietary": 0
    },
    "total_deduction_dollars": 0.015,
    "total_characters": 77532
  }
}