If you last scraped Reddit before 2026, the method you used is probably dead. In late May 2026, Reddit started returning HTTP 403 on unauthenticated requests — including the .json URL trick that powered a decade of tutorials, scripts, and no-code workflows. Most guides on this topic were written before that change and will walk you into an error.
Quick answer: there are four ways to scrape Reddit that still work in 2026. For scripts and research, use the official API with PRAW — free up to ~100 requests/minute. For no-code one-off pulls, use a cloud scraper like Apify. For bulk volume, use scraping infrastructure (Bright Data) or build your own crawler — a real engineering project. And for ongoing monitoring or AI-agent use, skip scraping and use a hosted data layer like Prowlo that serves clean Reddit data over MCP, REST, and webhooks. What no longer works: unauthenticated .json endpoints, and any script built on them.
Can you still scrape Reddit in 2026?
Yes — but the free-by-default era is over, and it helps to know what changed and when:
- June 2023: Reddit introduced commercial API pricing (~$0.24 per 1,000 calls above the free tier), killing third-party apps like Apollo and, later, tools like GummySearch.
- Late May 2026: Reddit blocked unauthenticated access outright.
www.reddit.com/*.json,old.reddit.com, and unauthenticated API paths all return 403, and Reddit's Public Content Policy plus a deny-allrobots.txtformalize it.
The context behind both moves: Reddit now sells its data. It's the most-cited source in AI-generated answers (40.1% of citations in a 150,000-citation Semrush study), and it has paid licensing deals with Google and OpenAI. Free bulk access undercuts that business, so Reddit keeps closing the free paths.
Here's each method that still works, and when to use it.
Method 1: The official Reddit API with PRAW (free, for developers)
The sanctioned route. Register a developer app in your Reddit account preferences (instant, no approval for personal apps), then use PRAW — the Python Reddit API Wrapper — to pull posts, comments, and subreddit data:
import praw
reddit = praw.Reddit(
client_id="YOUR_CLIENT_ID",
client_secret="YOUR_CLIENT_SECRET",
user_agent="myapp:v1.0 (by /u/yourusername)",
)
for post in reddit.subreddit("SaaS").new(limit=100):
print(post.title, post.score, post.url)
The free tier allows about 100 requests per minute per OAuth token for non-commercial use. The User-Agent format matters — Reddit rate-limits generic ones.
Where it breaks: continuous monitoring. Checking 50 subreddits every 5 minutes needs ~600 calls per window — 6× the free limit — and commercial use of API data requires a separately negotiated license. For a full breakdown of the tiers and costs, see Reddit API pricing.
Use it for: research scripts, prototypes, personal tools, one-off datasets.
Method 2: Cloud scraper tools (no code)
If you don't write code, cloud scrapers do the extraction for you. The Apify Reddit Scraper is the most-used: paste subreddit or post URLs, click Start, export JSON/CSV. Usage-based pricing with free starter credits, no Reddit account needed. General-purpose visual tools like Octoparse also still handle Reddit, with more fragility.
These platforms absorb the anti-bot arms race for you — that's what you're paying for. You still get raw output: unfiltered posts, spam included, re-run manually or on a schedule you manage.
Use it for: one-off or occasional bulk pulls without code. We compared the options in the best Reddit scrapers.
Method 3: Build your own crawler (the honest cost)
You can still crawl Reddit's public pages yourself with headless browsers — but understand what you're signing up for. Logged-out automated access gets 403'd, so working crawlers need rotating residential proxies, browser fingerprinting countermeasures, CAPTCHA handling, and constant maintenance as Reddit adjusts its defenses. The r/webscraping consensus on "how to scrape Reddit now" is a stack of proxies and retries, not a weekend script.
Add the parsing layer (Reddit's HTML changes), storage, and deduplication, and the "free" DIY route costs real engineering time every month. It also runs against Reddit's user agreement — see the legality section below.
Use it for: teams with scraping expertise who need full control and accept the maintenance and the terms-of-service risk.
Method 4: Skip the scraper — use a hosted data layer
Most people searching "how to scrape Reddit" don't want Reddit HTML. They want the outcome: mentions of their brand, threads in their niche, buying-intent signals, data their AI agent can search. For that, scraping is the hard way.
A hosted data layer like Prowlo maintains the Reddit (and X) data pipeline on its own infrastructure — it kept working through the May 2026 lockout — and does the parts a scraper leaves to you: it filters spam and noise, structures every record as typed JSON, and embeds everything for semantic search. You define Watchers (subreddits, keywords, accounts); your tools consume the results over MCP (for Claude, Cursor, or any agent), REST, or webhooks. Flat $19/mo after a free 14-day trial — no proxies, no parsers, no Reddit credentials.
Is it legal to scrape Reddit?
Two separate questions hide in there. Is it a crime? US courts have generally held that scraping publicly accessible web data is not criminal hacking (the hiQ v. LinkedIn line of cases). Is it allowed? No — Reddit's user agreement prohibits unauthorized scraping, and since May 2026 Reddit enforces this technically with the 403 lockout. Violating terms of service is a contract issue rather than a criminal one, but it can mean blocked infrastructure, banned accounts, and legal exposure at commercial scale — especially now that Reddit sells data licenses and has an incentive to protect them.
The clean options: the official API within its terms, or a data provider that holds the access relationship so you don't. This isn't legal advice; if you're scraping commercially, get some.
How do you scrape Reddit comments specifically?
Comments follow the same four paths. In PRAW, submission.comments walks a thread's full comment tree within the same free-tier limits. Apify's scraper extracts comments alongside posts. For a single thread with zero setup, a browser extension exports the comments of the page you're viewing. And if you're monitoring comments continuously — brand mentions, competitor complaints — that's the hosted-layer use case again: Prowlo records include full comment threads your agent can pull with one tool call.
Related reading
- The best Reddit scrapers in 2026, compared
- Reddit API pricing: cost, limits & alternatives
- The Reddit Data API, explained
- The best Reddit MCP servers for AI agents
FAQ
How do I scrape Reddit for free? Use the official API with PRAW: register a developer app (instant), authenticate over OAuth, and you get about 100 requests/minute at no cost for non-commercial use. It's the only free method that's both stable and sanctioned — the free .json endpoint trick stopped working in May 2026.
Is Reddit scraping allowed? Reddit's user agreement prohibits unauthorized scraping, and Reddit enforces it technically — unauthenticated automated requests return 403 since May 2026. The allowed paths are the official Data API within its terms and licensed data access. Public-data scraping hasn't generally been treated as criminal in US courts, but ToS and platform countermeasures still apply.
Why does my Reddit scraper return 403 errors? Reddit blocked unauthenticated access in late May 2026. Any script or tool that fetched reddit.com/*.json or other endpoints without OAuth credentials now gets HTTP 403. Fix: authenticate against the official API (100 req/min free), or switch to a tool that doesn't depend on those endpoints.
Does Reddit have a scraping API? Reddit's official Data API is the sanctioned interface — free for non-commercial use up to ~100 requests/minute, with commercial pricing (~$0.24/1,000 calls) and licensing above that. Commercial "scraper APIs" from Bright Data and similar vendors return structured Reddit data as a paid service.
Can Reddit detect and block scraping? Yes. Rate limits, User-Agent checks, IP reputation, and behavioral detection all apply, and the May 2026 lockout blocks unauthenticated automation outright. Sustained scraping without detection requires rotating residential proxies and constant adaptation — which is why most teams buy that capability or skip scraping entirely.
What's the best way to get Reddit data into an AI agent? Not a scraper — an MCP server. Your agent gets typed tools (search, fetch thread, list posts) instead of raw HTML. Prowlo serves filtered, semantically searchable Reddit data over MCP at $19/mo; the best Reddit MCP servers guide compares it with the open-source options.
Scraping Reddit to feed an agent or a monitoring workflow? Prowlo replaces the scraper: filtered, typed, semantically searchable Reddit data over MCP, REST, and webhooks. Start your free 14-day trial →