Design a Content Moderation System

Catch spam, nudity, hate and illegal media in 4 billion daily posts and comments before they spread, with 15,000 reviewers for the hard cases.

Last updated 2026-10-27. Difficulty: hard. Patterns: moderation, perceptual-hashing, review-queues, policy-engine, audit-log. Reported at Meta, TikTok, Google, Reddit, Discord, Snap.

Functional requirements

• Check every post, comment, image and video. Spam, adult nudity, hate, harassment, graphic violence, violent extremism and known illegal imagery. Cheap checks inside the write; heavier image, video and multimodal models after the post is accepted and before it reaches a wide audience.
• Block known violating media at upload. Match against hash lists (industry lists of known child sexual abuse material and terrorist content, plus our own lists of media removed before), surviving re-encoding, resizing and small edits.
• Gate distribution on confidence. An item whose scan is pending or whose score is uncertain reaches existing followers only (no recommendations, search or trending) until it is cleared. A slow check costs reach, not safety.
• User reports. 2 M a day, folded into one case per item and per cluster of copies, weighted by how often each reporter’s past reports were upheld. Notices from trusted flaggers get priority.
• Human review queues. Ordered by expected harm, routed by policy and language, with deadlines by severity. One decision per cluster of copies, applied to all of them. Reviewer protections and quality control are part of the tool.
• Graduated actions. Remove, withhold in a country, limit distribution, age-gate, warning screen, strike, feature limits, suspension. Each comes with a notice and a statement of reasons.
• Appeals. By the author, and by reporters when content is left up. Decided by someone other than the first reviewer, and by a person when the first decision was automated. A reversal restores the content and removes the strike.
• Policy changes without model changes. Thresholds and actions are versioned per policy, surface and country, replayed over stored scores and shadowed before they take effect.
• Re-scans, legal reports and transparency. Re-scan existing content when a hash list or policy changes; file the reports the law requires (for example to NCMEC in the US); publish transparency reports from the same records.
• Out of scope. Model architectures and training code, fake-account and bot detection (used here only as reputation signals), ads review, live streams, end-to-end encrypted messages, fact-checking programmes, and reviewer workforce scheduling.

Non-functional requirements

• Write-time checks (p99 < 50 ms added). They run on every one of ~120 k items a second at peak, so only cheap checks go inline: limits, reputation, hash lookups and a small text model. Anything that needs a GPU for longer runs after the ack.
• Reach before review (limited until cleared · images p95 < 1 min · video p95 < 10 min). This drives the hardest trade-off in the design: the models that catch the most are too slow and costly to run inline, so uncertain items wait at reduced reach instead of blocking the post or spreading unchecked.
• Known illegal imagery (0 views of a hash-list match). Blocked inside the write, preserved and reported where the law requires. General reviewers never see it; a specialist team handles confirmations.
• Time to action by severity (S0 < 15 min · S1 < 1 h · S2 < 24 h). Measured from the first signal (report or detection) to the action, and as views before action. Exposure is what the user feels, so it is the number reported alongside latency.
• Precision of automated actions (≥ 98 % per policy). Measured weekly from blind second reviews and appeal reversals. A policy whose automated removals fall below it routes to review instead of acting.
• Every decision recorded (100 % in the ledger · version + evidence). Appeals, strikes, statements of reasons, legal reports and transparency counts all come from one append-only record, so they cannot disagree.
• Fail to limited, not open or shut (posting up when checks are down). A moderation outage must neither stop posting nor let unscanned items into recommendations. Missing answers mean limited reach.
• Reviewer wellbeing and privacy (exposure caps enforced in the tool). Blurred, greyscale previews by default and per-shift caps on the most severe queues. Reviewers see only the personal data a decision needs, and every access is logged.

Back-of-envelope estimates

• Items written per second: ~46 k/s avg · ~120 k/s peak. 1 B posts + 3 B comments = 4 B items/day. 4 B ÷ 86 400 ≈ 46 k/s; evenings and big events run ~2.5× that, ~120 k/s. Every one passes the inline checks, so that tier is sized for the peak with a 50 ms budget.
• Inline text model at peak: ~30 GPUs. A small distilled multilingual classifier batched every few milliseconds does ~4 k short texts a second per GPU. 120 k/s ÷ 4 k = 30 GPUs at peak, ~12 on average. Hash lookups add ~6 k/s at peak, a few cores. The inline tier is cheap; its constraint is the 50 ms deadline, not money.
• GPUs for video scanning: ~350 busy · ~700 provisioned. 10 M videos × ~1 min average × ~3 GPU-seconds per minute (frames at 1 fps, speech to text, a multimodal model on flagged segments) = 30 M GPU-s/day ≈ 8,300 GPU-hours ≈ 350 GPUs busy on average, ~700 to absorb peaks. At ~$2 per GPU-hour that is ~$17 k a day. Images (100 M × 20 ms ≈ 23 GPUs) and text are small next to it: video is 0.25 % of items and ~85 % of the GPU bill.
• Hash lists in memory: ~1.6 GB raw · ~29 GB indexed. ~50 M list entries × 32 B (a 256-bit PDQ hash) = 1.6 GB. The multi-index keeps 16 chunk tables, each entry stored with its full hash for a sequential scan: 16 × 50 M × 36 B ≈ 29 GB, held in RAM on every matcher. A lookup reads 16 buckets of ~760 entries, ~0.1 ms, so RAM rather than CPU is the cost.
• Review capacity vs demand: ~2.6 M decisions/day vs ~20 M candidates. 15,000 reviewers × 5/7 on shift ≈ 11,000 a day × 6 h on queue × ~40 decisions an hour ≈ 2.6 M/day. Candidates: ~0.5 % of 4 B items land in an uncertain band (~20 M), plus 2 M reports on ~0.8 M items and ~250 k appeals. People can look at about one candidate in eight, so the queue is a ranking and the rest stay limited.
• Enforcement ledger: ~45 M rows/day · ~16 TB/year. ~20 M automated actions a day (0.5 % of items, mostly spam), ~20 M holds kept because of a score, and ~2.6 M human decisions ≈ 45 M decisions × ~1 KB = 45 GB/day, ~16 TB/year. Kept for years because strikes, appeals, legal requests and transparency reports read it. Evidence (preserved copies) is stored apart, with its own retention.
• Stored hashes for re-scans: ~600 B hashes · ~19 TB. Every upload’s hashes are stored: 100 M images + 10 M videos × ~10 keyframes = 200 M hashes/day. Over 8 years ≈ 584 B × 32 B ≈ 19 TB. A list update streams these through an index of only the new entries: one sequential pass, under an hour on a few hundred cores, instead of re-reading exabytes of media.
• Re-scoring with a new image model: ~12 days on 200 GPUs. ~10 B images were viewed in the last 30 days and carry nearly all views. 10 B × 20 ms = 200 M GPU-s ≈ 55 k GPU-hours ≈ 12 days on 200 GPUs, ~$110 k. All ~300 B stored images would take 30× longer, so backfills run by exposure, newest and most viewed first.

Components

• Inline checks (limits · reputation · text · hashes): The cheap tier, run inside the write: rate limits scaled by account reputation (age, verified phone, prior strikes, upheld reports), text normalised (Unicode NFKC, look-alike characters, zero-width characters) and scored by a small multilingual model in ~5 ms on batched GPUs, and a hash lookup per media file. ~120 k items a second at peak on ~30 GPUs. It returns signals and scores, never actions: the policy engine decides.
• Apps (post · report · appeal): The mobile and web apps. Media goes to object storage through a presigned upload before the post is sent, and the upload pipeline computes each file’s SHA-256 and perceptual hashes, so a post carries media ids rather than bytes. The apps also file reports and appeals and show notices with the statement of reasons for any action.
• API gateway (auth · per-IP limits): Authenticates users and applies coarse per-IP and per-device limits against floods, then routes posts and comments to the content service, and reports and appeals to the review platform. It holds no moderation logic, so policy never depends on which client sent the request.
• Content service (sharded by item id · outbox): Source of truth for posts, comments and media metadata, including each media file’s stored hashes and every item’s visibility. On a write it calls the inline checks with a 50 ms deadline, then commits the item, its reach state and an item.created event in one transaction. If the checks time out, the item is accepted as limited rather than rejected, except for authors with low reputation, whose items are held until checked.
• Policy engine (versioned rules · score store): Turns calibrated per-label scores plus context (surface, country, viewer age, account signals, reports, hash matches) into a decision under an immutable, versioned policy: an action and its scope, a reach state, and whether a person must look, with severity and queue. It keeps each item’s latest scores and model versions (~150 B per item for 90 days), so a threshold change is replayed over stored scores instead of rerunning models. Models never contain thresholds.
• Hash matcher (PDQ multi-index · ~50 M in RAM): Every list of known violating media in memory: industry lists of known child sexual abuse material and terrorist content, and our own lists of media removed before, ~50 M 256-bit PDQ hashes plus SHA-256 and video signatures. Each hash is split into 16 chunks of 16 bits with one table per chunk, so a near-match within 15 bits takes ~0.1 ms and within 31 bits about a millisecond. Each list carries its own match radius and action, and hashes of near-blank images are never matched.
• Event log (Kafka · key = item id): item.created and item.edited events from the content service’s outbox, keyed by item id so an item’s versions stay in order. Scanners consume it in separate lanes for hashing, images, video and backfill, so a re-scan never delays fresh uploads. Everything slow sits behind it, so posting latency never depends on a GPU queue.
• Async scanners (GPU · image, video, OCR, multimodal): The heavy tier, after the ack and before wide reach: image models (~20 ms of GPU each), OCR so text drawn into pictures is scored, video at ~3 GPU-seconds per minute (a cheap per-frame model at 1 fps, speech to text, then a multimodal model on flagged segments), and a model over image and caption together. Video is 0.25 % of items and ~85 % of the ~400 busy GPUs. It also places each item in a near-duplicate cluster using an index of the last 7 days of uploads.
• Review platform (queues · routing · tool · QC): Folds reports and policy referrals into one case per item or cluster, weights reports by reporter reliability, and orders each (policy, language) queue by expected harm with deadlines by severity. Serves cases to reviewers certified for that policy and language in a tool with blurred, greyscale previews and per-shift exposure caps, and mixes in known-answer items and blind second reviews. Appeals are cases too, never given to anyone who decided the item before.
• Enforcement service (ledger first · idempotent effects): The only writer of enforcement. It records each decision in the ledger first, then applies its effects keyed by decision id: visibility in the content service, reach state, account standing, notices with a statement of reasons, banked hashes and legal reports. A reconciler replays the ledger to repair any effect that failed, and the service serves decision records to the review platform for appeals.
• NCMEC & regulators (reports · lists · statements): Outside bodies: NCMEC, to which US providers must report apparent child sexual abuse material they become aware of; industry hash-sharing lists; the EU Commission’s database of statements of reasons under the Digital Services Act; and authorities that send removal orders. Their rules set deadlines, preservation periods and what the ledger must be able to prove.
• Training & re-scans (labels · active learning · backfills): Turns ledger decisions into policy-versioned labels, picks the active-learning slice and the random prevalence sample for review, retrains and calibrates models, and runs re-scans: new hash list entries against ~600 B stored hashes (~19 TB, under an hour), and new models over recently viewed media first within a GPU budget. Transparency counts are computed here from the ledger.
• Enforcement ledger (append-only · by day · hash-chained): One row per enforcement decision: subject, action and scope, policy id and version, source, model versions and scores, hash list entry and distance, pseudonymous reviewer id, evidence pointer, and the parent decision for appeals and reversals. ~45 M rows a day, ~16 TB a year, never updated: a reversal is a new row. Strikes, statements of reasons, transparency counts and training labels are all computed from it.
• Reach gate (in-memory KV · per item version): Per item version: eligible, limited (followers and direct links only, with a 10 k impression cap for authors over 100 k followers), restricted (age-gated or behind a warning screen, never recommended), withheld in listed countries, or removed. Recommendation and search indexes follow its change stream, and every recommended page is checked against it in one batched read before serving. A missing or unreadable key reads as limited, so a failure costs reach, not safety.
• Feeds, search, recs (outside this design): Every surface that hands an item an audience: followers’ feeds, recommendations, search, trending and hashtag pages. They run no moderation of their own: they index only eligible items and check each page of candidates against the reach gate, dropping anything not eligible for this viewer’s country and age.

User flows

1. A photo post, checked inline and cleared. The main write path. Cheap checks run inside the request and decide whether the post is accepted and at what reach; the image models run after the ack, and the post becomes eligible for recommendations only when they clear it. The author never waits for a GPU, and nothing unscanned reaches a stranger’s feed.
   1. The app posts a photo it has already uploaded, with a caption; the gateway authenticates the author and forwards the post to the content service. The photo went to object storage through a presigned upload before the post was sent (see Design Instagram), and that step computed its SHA-256 and its PDQ hash with a quality score. So the post carries a media id, and the inline checks never touch pixels. Hashing once at upload is also what makes re-scans cheap later.
   2. The content service calls the inline checks with the author, the text and the media hashes, under a 50 ms deadline. Cheapest first: rate limits and the author’s reputation (account age, verified phone, prior strikes, report history) from an in-memory store, in microseconds. A day-old account posting its thirtieth link in an hour stops here, before any model runs. Limits scale with reputation, so a new account gets a few posts an hour with links and an established one far more.
   3. The checks normalise and score the caption, and look up the photo’s hashes in the hash lists. Normalisation comes before the model: Unicode NFKC, look-alike characters mapped to one form, zero-width characters removed, repeated letters collapsed, links resolved to their final domain. A small multilingual model then scores spam, hate and harassment in ~5 ms on batched GPUs. The hash matcher answers an exact SHA-256 hit or a PDQ near-match in about a millisecond; a match on a list of known child sexual abuse material stops the post here, never visible, preserved and reported (the re-scan flow shows that path).
   4. The policy engine turns the scores into a write-time decision for this surface and country. Scores say how likely each label is; the policy says what to do about it. Every score here is under its action line, but the image models have not seen the photo yet, so the decision is accept at limited reach. A text-only post with clean scores from an author in good standing gets eligible at once from the same engine.
   5. The content service commits the post with its reach state and an outbox event, then acknowledges the author. The item row, its reach state and the item.created event commit in one transaction, so a crash cannot leave a post that is public and never scanned. Inline checks added ~20 ms typically and 50 ms at worst. If they time out, the post is still accepted as limited, unless the author’s reputation is low, in which case it is held until checked.
   6. The image scanners classify the photo, read any text in it, and score the photo and caption together. Nudity, graphic violence and hate-symbol models share one decode of the image. OCR pulls out drawn-in text so the text model sees it, and a multimodal model scores the image and caption together, because a harmless photo with a hateful caption is exactly what each model alone misses. About 20 ms of GPU per image; p95 under a minute end to end, mostly queueing.
   7. The scanners send the scores to the policy engine, which decides the post is clear. Every score is under its clear line for this surface and country, so the decision is eligible. The scores are stored with their model versions, so a later threshold change can be replayed over them without running a model again.
   8. The enforcement service sets the post’s reach to eligible, and recommendations and search can now serve it. Recommendation and search indexes pick up the change from the gate’s change stream, and the gate is also the final filter at serve time: each page of candidates is checked in one batched read, so a stale index cannot serve something that was limited a second ago. A clearance is not an enforcement action and is not written to the ledger; a hold kept because of a score is.
2. A reported comment reaches the right reviewer. The main human path: a hateful comment in Portuguese under a popular post is reported 40 times. The reports become one case, ordered by expected harm, routed to a reviewer certified for hate speech in Portuguese, decided once, and recorded before anything changes.
   1. Users report the comment; the gateway sends each report to the review platform, which folds them into one case. Forty reports on one item are one case, not forty. Each report is weighted by its reporter’s record (the share of their past reports that were upheld), so a coordinated mass report moves priority less than one notice from a trusted flagger, which the EU Digital Services Act requires platforms to handle with priority.
   2. The review platform gets the comment’s stored scores and the policy’s view of severity and queue. The comment scored 0.55 for hate at write time, inside the uncertain band, so it was already limited; the reports raise the estimated probability of a violation to about 0.7. If the comment’s cluster had already been decided, the platform would apply that decision without a person.
   3. The case is placed in the Portuguese hate-speech queue, ordered by expected harm. Priority is P(violation) × severity weight × predicted views in the next 24 hours. Here 0.7 × 10 (S2) × ~40 k views from a busy thread ≈ 280 k, far ahead of a likely-spam comment at 0.9 × 1 × 50 ≈ 45. Priorities are recomputed every few minutes from view velocity, and as a case nears its severity deadline the deadline takes over.
   4. A reviewer certified for hate speech in Portuguese opens the case in the review tool. The tool shows the comment, the thread around it, the author’s recent history in summary, the policy text for the version in force, and the model’s scores. Media in a case would be blurred and greyscale until the reviewer chooses to look. The tool counts each reviewer’s exposure to the most severe queues and stops serving those cases once the shift’s cap is reached.
   5. The reviewer decides it violates the hate policy, and the verdict goes to the enforcement service. The verdict names the exact rule, not just "hate", because the statement of reasons, the strike weight and the training label all depend on it. About 5 % of verdicts are picked at random for a blind second review before they are final, and 3 % of the queue is known-answer items the reviewer cannot tell apart.
   6. The enforcement service records the decision in the ledger before applying anything. Ledger first, then effects: an effect without a record can never be explained, appealed or counted, while a record whose effect failed is repaired by retrying. The row holds the rule and policy version, the scores, the reports, the reviewer’s pseudonymous id and a pointer to a preserved copy.
   7. It removes the comment, records a strike against the author, and sends a notice with a statement of reasons. Every effect is keyed by the decision id, so a retry cannot apply it twice. The author’s standing is recomputed from upheld decisions in the last 90 days, weighted by severity. The notice says what was restricted, which rule and facts it rests on, whether automated means were used, and how to appeal, which is what a statement of reasons under the DSA needs. Reporters are told the outcome.
   8. The decision flows from the ledger to training as a label tagged with its policy version. A label is only as valid as the definition it was made under, so it carries the rule and the policy version, plus whether it was a second review or later reversed on appeal. The comment’s text, normalised, also joins the near-duplicate index, so copies posted under other threads join its cluster and are decided without another review.
3. One violating video, 100,000 copies. The scale-breaking case: a graphic video is uploaded, then re-uploaded by thousands of accounts, some edited to evade matching. Reviewing each copy would take more reviewer hours than exist. The design reviews it once, applies the decision to every copy, and blocks future copies at upload.
   1. A new video passes the inline checks, is accepted at limited reach and queued for the video scanners. It matches no list yet, the caption is clean and the account is not new, so nothing inline objects. Limited reach means its followers can watch but no recommendation, search or trending surface can, so for the next few minutes its audience is bounded by the uploader’s followers.
   2. The video scanners score its frames, speech and segments, and find likely graphic violence. A cheap frame model at 1 fps finds the segments worth a closer look, and only those go to the expensive multimodal model, which is how video stays near 3 GPU-seconds per minute. The scanners also store the video’s TMK+PDQF signature and keyframe PDQ hashes, and its cluster already has 3,000 members from other accounts in 20 minutes.
   3. The policy engine keeps every copy limited behind a warning screen and sends the cluster to review at top priority. Predicted reach is the cluster’s, not one copy’s: 3,000 copies gaining views is a viral item, so it lands at the top of a specialist queue with the S1 deadline of an hour. The warning screen is applied to the whole cluster meanwhile, because 0.91 is high but not above the line for automatic removal in this policy.
   4. A specialist reviewer decides the cluster once: it violates the graphic violence policy. The reviewer sees one representative copy as a strip of blurred, greyscale keyframes with the flagged segments marked and audio off, not as playback. The verdict applies to the cluster. Members far from the representative (a wider hash distance or only an embedding match) are listed as a grid for a quick spot check rather than reviewed one by one.
   5. The enforcement service records a decision for every copy, removes them all, and banks the video’s hashes. One verdict fans out to one ledger row per copy, each pointing to the parent decision, because each author gets their own notice, strike and right to appeal. At ~10 k items a second the fan-out takes seconds even for 100 k copies. The signatures go into an internal list with action block at a tight radius and review at a wider one.
   6. New uploads of the video now match the banked hashes inside the write and are blocked before anyone sees them. Blocking at upload is the main lever in a re-upload wave: Facebook said it removed about 1.5 million copies of the Christchurch attack video in the first 24 hours, more than 1.2 million of them at upload. Video hashes come from the upload pipeline within seconds; if they are not ready when the post arrives, the hash lane runs first in the scanners, so a match lands within seconds either way.
   7. Edited copies that miss the tight radius join the cluster through the wider radius or the embedding, and go to review as members. Crops, mirroring, borders, overlaid text and speed changes move a perceptual hash further than re-encoding does. Mirrored and rotated variants are hashed too, and a learned copy-detection embedding catches heavier edits. Those copies are confirmed in batches of twenty as members of a decided cluster, seconds each, instead of opening new cases.
4. A wrong removal, appealed and reversed. The failure path: the image model removes a breast cancer awareness post as adult nudity. The author appeals; a person who was not part of the first decision reviews it, the reversal is a new ledger row, and everything derived from the first decision (visibility, reach, the strike, the counts, the label) follows it.
   1. The image model scores a post 0.985 for adult nudity, above the automatic removal line, and the post is removed. Adult nudity is removed automatically at 0.98 and above on this surface, where measured precision is about 99 %. At ~100 k such removals a day, 1 % is still ~1,000 wrong removals daily, which is what appeals exist for. The ledger row records the model version, the score, the threshold and the policy version, so the error can be traced exactly.
   2. The author gets a notice with the statement of reasons and appeals; the appeal reaches the review platform. The notice says what was removed, under which rule, that the decision was automated, and how to appeal. Under the DSA, an internal complaint system must be available for at least six months after a decision, and complaints must not be decided solely by automated means.
   3. The review platform builds an appeal case from the decision record and excludes everyone who decided the item before. The enforcement service returns the whole decision chain from the ledger: scores, rule, version, evidence. Every reviewer in that chain is excluded, and the case goes to a more senior certification at another site where possible. The first decision was automated, so this is the first time a person sees the post.
   4. A senior reviewer finds the post allowed under the health-awareness exception and reverses the decision. The reviewer sees the rule as it was when the post was removed and as it is today; either one would be enough to restore it. The verdict names the exception, which matters later: it tells the policy team and the model that this is a known context, not a one-off.
   5. The enforcement service appends a reversal row, then restores the post, its reach and the author’s standing. The first row is never edited; the reversal points to it as its parent. Standing is recomputed without the strike, the post goes back to the reach it would have had, and the author is told. Transparency counts will show the item as removed and restored on appeal, both dated by their own decisions.
   6. The reversal becomes a high-weight training label, and repeated reversals in one context flag the threshold. An appeal that overturns a confident automated decision is the most valuable label there is: a measured model error. Reversal rate per rule and per model version is a headline metric, and if health-awareness posts keep being reversed, the policy team adds a rule that routes posts with that context signal to review instead of removing them, while the next model learns the case.
5. A hash list update reaches old content. The background path: an industry list adds 20,000 hashes of known illegal imagery. Copies already on the platform must be found without re-reading exabytes of media; the hashes stored at upload turn it into one pass over ~19 TB. Matches are removed, preserved and reported, and the reports and counts come from the ledger.
   1. The enforcement service pulls a list update from an industry partner and loads the new entries into the hash matcher. Each list has an owner, a version and a policy: here, block at a tight radius and route a wider one to a specialist queue. New entries are appended to the matcher’s chunk tables on every replica, so new uploads match from the moment the load finishes, within a minute.
   2. The re-scan job streams every stored media hash through an index built from only the new entries. ~600 B stored hashes, ~19 TB, are read sequentially in the scanners’ batch lane. The 20,000 new entries make a small 16-chunk index that sits in cache, so each stored hash costs 16 lookups. One pass takes under an hour on a few hundred cores and touches no media at all.
   3. Candidates are confirmed against the original files and the policy engine decides. Stored hashes came from the upload; the batch lane re-hashes the original file at full resolution and runs the image model as a second signal, because a severe action on an old account deserves more than one stored number. Tight matches confirmed this way are removed and reported; the rest go to the specialist queue.
   4. The enforcement service records each match, removes the media everywhere, and preserves a copy for the report. Removal means every surface and every cached copy: visibility in the content service, the reach gate, and a CDN purge of the media’s URLs. The preserved copy goes to a restricted evidence store with every access logged, separate from the ledger, which holds only a pointer to it.
   5. The enforcement service files the report the law requires and records that it did. US providers must report apparent child sexual abuse material they become aware of to NCMEC’s CyberTipline and preserve it for a period the law sets, which a 2024 law extended to a year. Whether a person at the provider views a hash-matched image before it is reported has legal consequences (US courts have split on whether investigators may then open a file nobody at the provider viewed without a warrant), so the design supports both and counsel decides. The report id goes into the ledger.
   6. Each period, the transparency job counts decisions from the ledger by policy, source and outcome. Proactive rate (found by us before any report), appeals, restorations, legal reports and authority orders are all queries over the same rows that produced each user’s notice, dated by decision time, so the aggregate and the individual answers cannot disagree. The same rows feed the statements of reasons the DSA asks platforms to send to the Commission’s public database.

Deep dives

The theory behind it

• Trust and safety, moderation and abuse prevention: Designing defences against spam, fake accounts, fraud and harmful content: layered checks at signup, posting and payment, rules plus machine learning, review queues, hashing known-bad media, reputation, rate limits and feedback loops.
• Audit logs and tamper-evident records: Designing audit trails that answer who did what and when: what to record, append-only storage, immutability and retention, hash chains and Merkle trees for tamper evidence, querying and exporting, privacy in audit data, and the difference between audit logs, application logs and event sourcing.
• Media uploads and processing pipelines: How photo and video uploads work at scale: direct-to-storage uploads with presigned URLs, resumable and chunked uploads, processing pipelines for thumbnails and transcoding, metadata and status tracking, deduplication, moderation hooks and serving variants through a CDN.

Related

• Design Live Comments
• All 83 system design interview questions
• Run this question for a friend: the interviewer kit