Cloud storage is cheap enough that the default answer to “how long do we keep this?” becomes “forever.” That default has costs that do not show up on the storage bill: more data to secure, more data exposed in a breach, more to search through when a legal or privacy request arrives, and more places for stale, wrong information to keep feeding decisions.

A retention policy is the deliberate alternative. It states how long each kind of data is kept, why, and what happens at the end. The writing is the easy part. The hard part is making deletion actually happen across every copy.

This article gives general guidance. Retention requirements differ by jurisdiction, industry and data type, so confirm specifics with your legal or compliance advisers.

Why Keep Data at All, and for How Long

Retention periods usually come from three sources, and each pulls in a different direction:

• Obligations to keep: financial, tax, employment or sector rules that require records for a minimum period.
• Obligations to delete: privacy rules and contracts that require personal data not be held longer than needed for its purpose, or deleted on request.
• Business value: how long the data remains useful for analysis, model training or support.

Where these conflict, the legal and contractual requirements generally win, and the business-value period fills the gaps. Do not guess these; ask the owners of the compliance obligations and write down the source of each period so that later reviewers know why it is what it is.

Build a Retention Schedule by Data Category

Do not attempt a rule for every table. Group data into categories and set a rule for each. A workable schedule has columns for the category, example datasets, the retention period, the trigger that starts the clock (creation date, end of contract, account closure), the end-of-life action (delete, anonymize, archive), and the owner.

The trigger matters more than people expect. “Seven years” means very different things if the clock starts at record creation versus at account closure.

Deletion Is Harder Than It Sounds

The place most programs fail is the gap between the policy and the technology. A deleted record may still exist in:

• Derived tables, extracts and caches built from it.
• Raw landing zones and archive tiers.
• Object versions retained by versioning.
• Backups and snapshots.
• Logs and event streams.
• Notebooks, local exports and shared drives.
• Copies held by vendors and processors.

To delete reliably, you need to know where data lives, which is where lineage and a data catalog earn their keep. Start by mapping, for each category, the systems where it appears. Without that map, a deletion request becomes a scramble.

Practical Mechanisms in the Cloud

Lifecycle rules on object storage can automatically expire objects after a set age and clean up old versions. They are the simplest enforcement for file-based data and worth using wherever the age of the object matches the retention clock.

Time-based partitions make table-level retention easy: dropping a date partition is far simpler and more complete than deleting scattered rows. Designing tables to partition by the date that drives retention pays off here.

Row-level deletion for individual requests, such as “delete this person’s data,” is harder in columnar systems and lakes. Check how your platform handles deletes, whether physical removal happens immediately or only after compaction or vacuum operations, and how long deleted data remains recoverable.

Anonymization or aggregation can be an alternative to deletion when you want to keep analytical value. Be cautious: weakly anonymized data can sometimes be re-identified, so have it reviewed against your privacy requirements.

Handle Backups Honestly

Backups usually cannot have individual records removed. Common approaches are to keep backup retention short enough that deleted data ages out within an acceptable period, and to record the approach in your policy so it is a documented decision, not a silent gap. Whether that approach satisfies a given regulation is a question for your compliance advisers.

Retention Plan Table

Layer	Retention mechanism	Watch out for
Raw landing files	Lifecycle expiry by object age	Old versions kept after delete
Curated tables	Drop partitions by retention date	Row-level requests need separate handling
Derived marts and extracts	Rebuild from source after upstream deletion	Stale copies in BI tools and spreadsheets
Backups and snapshots	Short retention windows	Records surviving until backup expiry
Logs and events	Expiry rules, minimize personal data	Identifiers logged unintentionally
Third-party processors	Contractual deletion terms	No confirmation of deletion

A Realistic Example

Imagine a team that stores customer support transcripts. The policy says transcripts are kept for a set number of months after a ticket closes, because after that they lose analytical value and carry personal information. The team partitions the transcripts table by ticket close month, adds a lifecycle rule on the raw files, and schedules a monthly job that drops expired partitions and reruns dependent summary tables. They also document that backups age out on their own schedule. When a customer requests deletion, a documented procedure removes their rows from the curated table, triggers compaction, and logs the action with a date and the person who ran it.

Prove It Worked

A deletion nobody can demonstrate is hard to defend. Keep a log of scheduled deletions and individual requests: what was deleted, when, from which systems, and who verified it. Periodically test by picking a record that should be gone and searching every layer for it.

Review the Policy Regularly

Business needs, systems and laws change. Review the schedule at least annually and whenever you add a major new data source. Treat new datasets as incomplete until they have a category and an owner.

Frequently Asked Questions

Can’t we just keep everything in cheap archive storage? You can, but stored data is still data you are responsible for. Archive does not remove security, privacy or legal obligations.

What is the difference between deleting and anonymizing? Deleting removes the data. Anonymizing removes the link to a person while keeping some content. Anonymization is only as good as its resistance to re-identification, so it needs careful review.

Who decides retention periods? Typically legal or compliance sets minimums and maximums, data owners confirm business needs, and the data team implements. Document who approved each rule.

How do we handle deletion requests across many systems? Maintain a data map showing where each category of personal data lives, and a runbook listing the steps per system. Lineage tools help keep the map current.

What if the platform cannot physically delete quickly? Document the actual behavior and time to physical removal, and have compliance advisers confirm it is acceptable for your obligations.

Next Step

Choose one category of personal data you hold. Write down where it appears across storage, warehouse, extracts and backups, and what the retention period is. If you cannot fill in all of those, that gap is your first governance task.