How to Scrape Reddit with an API

By Nikhil Kumar. Last updated August 2026.

Reddit is the best public dataset on the internet and the most annoying one to collect. Every method that used to be easy now fights back.

To scrape Reddit with an API, you have four real options in 2026: the undocumented .json endpoint, the official API through PRAW, HTML scraping of old.reddit.com, or a managed data API. The .json trick is now rate-limited, PRAW caps you at roughly 60 requests a minute and needs an OAuth app, and HTML scraping breaks on every redesign. A managed data API is the one path that hands you clean Reddit data without any of that.

This is the practical companion to what the Reddit API actually costs. That post covers the money; this one covers the mechanics.

How do you scrape Reddit with an API?

You send a request for a subreddit, a keyword, or a post URL, and get structured Reddit data back. The honest version is that “with an API” can mean Reddit’s official API, which you drive yourself with PRAW and OAuth, or a third-party data API, which drives the collection for you. Both return JSON. The difference is who owns the rate limits, the retries, and the blocks: you, or the service.

“reddit scraper” pulls about 480 US searches a month, “reddit scraping” another 110, and “how to scrape reddit” 70, according to DataforSEO keyword data pulled in August 2026. Most of the people searching have already tried the free tricks and hit a wall.

What are the four ways to scrape Reddit in 2026?

There are four methods that still function, and each trades effort against reliability. The .json endpoint is easiest and least reliable, PRAW is official but capped and needs setup, HTML scraping is flexible and fragile, and a managed data API is the least work and a paid service. Pick by how much maintenance you are willing to own.

The four, ranked by how little they leave you to maintain:

1. Managed data API. You send a request, you get Reddit data. Collection, rate limits, and blocks are handled for you, billed per item. Least work.
2. PRAW on Reddit’s free tier. The official Python Reddit API Wrapper. Reliable and free under about 60 requests a minute, but you register an OAuth app and manage the limits.
3. HTML scraping of old.reddit.com. Parse the old UI with BeautifulSoup. Flexible, but it breaks whenever Reddit changes markup, and a 2026 developer benchmark clocks it at roughly 10 to 30 posts a minute with polite delays.
4. The .json endpoint. Append .json to any URL. Fastest to start, but now rate-limited and blocked, so it is a one-off tool, not a pipeline.

For historical bulk data, there is a fifth path: archive dumps like Arctic Shift, which ship full datasets by torrent but run a month or two behind live.

Does the Reddit .json trick still work?

Barely, and not for anything real. Appending .json to a Reddit URL, like reddit.com/r/python/hot.json, used to return clean structured data with no authentication. In 2026 that endpoint is heavily rate-limited and frequently blocked without a descriptive User-Agent, and it returns 429 or 403 errors under any sustained load. It still works for a single manual pull. It fails the moment you loop.

import requests
# Works once, throttles fast, needs a real User-Agent
r = requests.get("https://www.reddit.com/r/python/hot.json",
                 headers={"User-Agent": "my-research-script/1.0"})

The trap is that it looks like it works in testing. You pull one page, it returns, you build a loop, and by request fifty you are staring at 429s. It was never a supported endpoint, and Reddit has quietly tightened it.

How do you scrape Reddit with PRAW?

PRAW is the official, maintained Python wrapper for Reddit’s API, and it is the correct DIY path for small projects. You register an app to get a client id and secret, pass those with a descriptive user agent, and call methods like subreddit.hot() or submission.comments. It respects Reddit’s rate limits automatically, which in practice means around 60 requests a minute for legitimate use. Reddit’s own Data API terms keep that free tier for non-commercial work.

import praw
reddit = praw.Reddit(client_id="...", client_secret="...",
                     user_agent="my-app/1.0 by u/you")
for post in reddit.subreddit("python").hot(limit=50):
    print(post.title, post.score)

PRAW is genuinely good for a bot or a small dashboard. Its ceiling is Reddit’s ceiling: the free tier is for non-commercial use, and above it you are into the $0.24-per-1,000-calls metering covered in the pricing post.

How do you avoid getting blocked while scraping Reddit?

Send a descriptive User-Agent, pace your requests, and treat 429 and 403 as different problems, because they are. A 429 means you hit a rate limit, so back off with exponential delay plus jitter and retry. A 403 means Reddit flagged your behavior, so stop retrying, because pounding a 403 deepens the block. Mixing them up is the single most common way people get their IP banned.

import time, random
def get(url, headers, tries=4):
    for i in range(tries):
        r = requests.get(url, headers=headers)
        if r.status_code == 429:                 # rate limit: back off + retry
            time.sleep(2 ** i + random.random())
            continue
        if r.status_code == 403:                 # behavioral block: stop
            return None
        return r
    return None

Reddit tracks IPs and serves CAPTCHAs to traffic that looks automated. Rotating proxies helps, but it is a maintenance job of its own. This is the exact work a managed API takes off your plate.

How do you scrape Reddit comments?

You point the scraper at a post’s URL and read its comment tree. With PRAW, you load submission.comments and flatten the nested forest, handling the “load more comments” placeholders yourself. With a data API, you send the post URL to a comments endpoint and get a flat list of comments back, no tree-walking required.

With ScraperSocial, comments and the rest of Reddit map to one request shape:

import requests
r = requests.get(
    "https://api.scrapersocial.com/v1/reddit/post-comments",
    params={"url": "https://www.reddit.com/r/python/comments/abc123/", "limit": 100},
    headers={"Authorization": "Bearer sk_live_..."},
)
print(r.json()["data"])

Comments are 2 credits each, search results and subreddit posts are 2 credits, and a full post is 4. Credits are $0.005 on the monthly plan and $0.0045 on annual, so comments run about a cent each and the 100 free signup credits cover roughly 50 to test with.

How do you scrape Reddit without managing any of this?

A managed data API owns the parts you keep tripping on: the OAuth, the rate limits, the 429 and 403 handling, the proxies. You hand it a target and the same envelope comes back every time. You give up fine-grained control. In return, you never write a rate-limit loop again.

That trade is worth it when you want data, not infrastructure. It is the wrong trade when scraping Reddit is your product and you need to tune every call. For most teams pulling posts and comments to feed an analysis or a model, it is the fastest path from zero to data.

Is it legal to scrape Reddit?

Reading Reddit’s public posts and comments is collecting public data, which US courts have generally allowed, though Reddit’s own terms are a separate question from the law. In hiQ Labs v. LinkedIn, the Ninth Circuit reaffirmed in April 2022 that scraping publicly available data does not violate the Computer Fraud and Abuse Act. That is the federal-law question, not Reddit’s contract terms.

So treat it as a compliance decision. Stick to public data, do not log in with a personal account to get around limits, and cap how long you keep anything tied to a username.

None of this is legal advice. If your use is commercial or high-volume, run it past counsel before you build on it.

Pick the method, then count the calls

Start by matching the method to your job. A one-off pull can live on the .json trick, a small bot belongs on PRAW, and a production pipeline that needs posts and comments across many subreddits belongs on a managed API where the blocks are someone else’s problem. Whatever you pick, add the User-Agent, respect the 429, and never retry a 403.

Frequently asked questions

How do I scrape Reddit?

You can scrape Reddit four ways: the undocumented .json endpoint, the official API via PRAW, HTML scraping of old.reddit.com, or a managed data API. The .json trick is now rate-limited, PRAW needs an OAuth app and stays under about 60 requests a minute, and HTML scraping breaks on layout changes. A data API removes all three problems by handling collection for you.

Does the Reddit .json trick still work?

Barely. Appending .json to a Reddit URL used to return clean JSON with no auth, but in 2026 the endpoint is heavily rate-limited and often blocked without a descriptive User-Agent, and it 429s or 403s under any real volume. It is fine for a one-off manual pull and unreliable for anything you need to run on a schedule.

How do I scrape Reddit comments?

Point a scraper at a post’s URL and read its comment tree. With PRAW you call submission.comments and flatten the forest; with a data API you send the post URL to a comments endpoint and get a flat list of comments back. The data API path skips OAuth and rate-limit code, and bills per comment returned instead of per API call.

How do I avoid getting blocked while scraping Reddit?

Send a descriptive User-Agent, add delays, and treat 429 and 403 differently. A 429 is a rate limit, so back off with exponential delay plus jitter and retry. A 403 is a behavioral block, so stop retrying, because hammering it makes the block worse. A managed API absorbs both by pacing requests and rotating infrastructure for you.

Is it legal to scrape Reddit?

Reading Reddit’s public posts and comments is collecting public data, which US courts have generally allowed, though Reddit’s terms are a separate question from the law. In hiQ Labs v. LinkedIn the Ninth Circuit reaffirmed that scraping public data does not violate the Computer Fraud and Abuse Act. Stick to public data, do not log in to bypass limits, and cap retention of anything tied to a username.

What is the best Reddit scraper?

There is no single best one; it depends on the job. For a small non-commercial project, PRAW on Reddit’s free tier is the cheapest official route. For historical bulk analysis, an archive like Arctic Shift. For hands-off collection across subreddits and comments without OAuth or block-handling, a managed data API is the least work. Match the tool to volume and maintenance tolerance.