Blog

Notes from the data layer

Engineering notes and practical guides on building with social media data. Written by the people running the pipeline, so the limitations are in here too.

Instagram, Facebook and X Video Transcripts: What Works

Instagram reel transcripts now work through an API, priced per minute. Facebook and X don't yet. What each returns, what it costs, and what to use instead.

Read →

Monitor a TikTok Creator and Summarize New Videos

Watch any public TikTok creator for new uploads, summarize each video with AI and post a daily digest to Slack. Python script, pinned-video fix, cost math.

Read →

Social Video Transcripts for Research: Methods and Limits

How to use TikTok and YouTube transcripts in research: sampling you can defend, caption gaps, accuracy checks, reproducible collection, ethics and GDPR.

Read →

15 Business Ideas Built on Social Media Data (2026)

15 business ideas you can build on public social media data, each with the endpoints it needs, what one unit of output costs, and the limits to plan for.

Read →

Supadata Alternatives for Transcripts and Social Data (2026)

Supadata alternatives compared: which API wins on TikTok transcripts, AI fallback, comments and social data, with prices checked on each vendor's page.

Read →

Build a RAG Pipeline on TikTok Transcripts (TypeScript)

Build a RAG pipeline on TikTok transcripts in TypeScript: load a creator's videos, embed with Voyage, retrieve, and get Claude answers that cite the video.

Read →

TikTok Transcript API in JavaScript and Node.js

Get TikTok transcripts in JavaScript and Node.js: typed fetch code, retries with backoff, a concurrency pool, a server route for your frontend, and cost.

Read →

n8n Social Media Scraping with the HTTP Request Node

Scrape TikTok and other platforms in n8n with one HTTP Request node: the credential, two working workflows, de-duping, 429 retries and real credit math.

Read →

How to Get Instagram Location Data with an API

Instagram's official API has no location search. Here's how to find places, read their coordinates and post counts, and pull tagged posts as JSON.

Read →

Build a Social Media Research Agent with Claude

Three ways to build a social media research agent with Claude: the MCP connector, the Tool Runner, and the Agent SDK. With a credit cap and source checks.

Read →

How to Scrape TikTok Hashtags with an API

TikTok hid hashtag view totals in 2024 and its official API is academics-only. Here's how to pull videos under any hashtag as JSON, and what you can't get.

Read →

How to Get Bluesky Data with an API

Bluesky's own API is free and needs no login for public data. Here's how to pull profiles, posts, and keyword search, and when a unified API is worth it.

Read →

How to Search Instagram by Keyword with an API

Instagram's official API has no keyword search. Here's how to find public posts that mention any term, not just a hashtag, in one GET request.

Read →

How to Search Twitter (X) with an API

X's official search API meters every read and stops at 7 days. Here's how to search public tweets with a data API, what comes back, and what it costs.

Read →

How to Find Facebook Events with an API

Meta's Graph API returns almost no event data. How to search public Facebook events by keyword, pull a page's events and read attendance as JSON.

Read →

How to Scrape a Facebook Group with an API

Meta killed the Facebook Groups API in 2024. How to pull public group posts, reactions and comments as JSON, and what you can no longer get at all.

Read →

Best Pinterest Data APIs in 2026

Pinterest's official API posts pins but won't read them at scale. The best Pinterest data APIs for public pins, search and save counts, compared.

Read →

How to Build an Amazon Price Tracker with an API

Poll any Amazon product's current price with one GET request, store the history yourself, and alert on drops. Real fields, honest cost, no PA-API wall.

Read →

Google Maps API: How to Pull Business Data and Reviews

Get Google Maps listings, reviews, and contacts with one GET request. Real fields, honest pricing, and what the official API's terms won't let you do.

Read →

How to Get a YouTube Transcript with an API

Get the full text of any public YouTube video with one GET request: real fields, timestamps, and why the official API and free libraries break on servers.

Read →

Twitter (X) Profile Viewer: See Any Public Account with an API

X walls off profiles behind a login. How to view a public X account's bio, counts and join date without one, why private accounts stay locked, and the API.

Read →

Instagram Profile Viewer: See Any Public Account with an API

View any public Instagram profile's stats and posts while logged out. What a profile viewer can and can't show, why private accounts stay locked, the API.

Read →

How to Download Facebook Videos with an API

Pass a public Facebook video or Reel URL and get a direct MP4 link back. How to download Facebook videos by API, why the link expires, and the legal line.

Read →

How to Get Facebook Page Posts & Comments

The Facebook Graph API gates public page data behind app review. How to pull any page's posts and their comment threads as JSON with a data API instead.

Read →

How to Pull LinkedIn Company Data with an API

LinkedIn's official API returns company data only for pages you own. How to pull any company's firmographics, funding, jobs, and people as JSON instead.

Read →

How to Pull LinkedIn Posts with an API

LinkedIn's API publishes posts but will not read anyone else's. How to pull a person's or company's public posts as JSON, the fields, and the legal line.

Read →

How to Get Twitch Data with an API

Twitch's official API is free but wrapped in OAuth. How to pull a channel's followers, live status, videos and clips as JSON with a data API instead.

Read →

Best TikTok Data APIs in 2026

TikTok's official API is free but academics-only, so commercial work runs on third-party APIs. Six TikTok data APIs compared on coverage, cost, and access.

Read →

How to Scrape Facebook Marketplace with an API

Facebook Marketplace has no official API. How to pull listings, prices and locations as JSON with a data API, what fields you get, and where the line is.

Read →

Best Facebook Data APIs in 2026

There is no single Facebook API. Data lives across pages, ads, marketplace and groups, each behind its own wall. Six data APIs, ranked by surface.

Read →

Best Twitter (X) Data APIs in 2026

X killed the free tier and now meters every read. The best Twitter (X) API in 2026 is the one that fits your risk and volume. Six options ranked honestly.

Read →

Best Reddit Data APIs in 2026

Reddit data APIs in 2026, ranked by the question most lists skip: can you get approved at all? Six options compared on access, coverage, and cost.

Read →

How to Get Pinterest Data with an API

Pinterest's official API only reads your own account and bars caching. How to search public pins by keyword and get engagement and destination domains.

Read →

How to Scrape Instagram Hashtags with an API

Instagram's official API caps hashtag search at 30 tags per 7 days, Business accounts only. How to pull posts, reels and related tags for any hashtag.

Read →

TikTok Profile Viewer: See Any Account Anonymously

View any public TikTok account's stats and videos without logging in. What a profile viewer shows, what stays private, and how to do it at scale by API.

Read →

How to Use the Reddit API in Python (PRAW & Beyond)

Set up PRAW, pull Reddit posts and comments in Python, and handle the 2026 walls: the 100 QPM limit, app approval, and no commercial use without a deal.

Read →

Facebook Ad Library API: Pull Competitor Ads as JSON

Meta's Ad Library API only returns political ads outside the EU. How to pull any advertiser's Facebook and Instagram ads as JSON, and what you get.

Read →

How to Use the Twitter (X) API in Python (2026)

Four ways to get X data in Python: the official API via Tweepy, dead free scrapers, DIY browser automation, and a data API. What each one costs.

Read →

TikTok Research API vs a Data API: What Each Gives You

TikTok's Research API is free but academic-only and bans commercial use. A data API is commercial and self-serve. What each gives you, and who it's for.

Read →

Instagram Follower Tracker: Track Any Account with an API

Instagram hides the follower list, even from its own API. What a follower tracker can actually do: track follower count and growth for any public account.

Read →

How to Get LinkedIn Jobs Data with an API

LinkedIn has no self-serve jobs API. Here's how to pull LinkedIn job postings as JSON by keyword or company, what fields you get, and what it costs.

Read →

How to Download Instagram Videos with an API

Pass a public Instagram Reel URL and get a direct MP4 link back. How to download Instagram videos with an API, why the link expires, and what it costs.

Read →

Proxycurl Alternatives for LinkedIn Data (2026)

Proxycurl shut down in 2025. Its replacements are real-time scrapers, dataset providers, and account-based APIs, compared by cost, freshness, and risk.

Read →

How to Search Threads by Keyword with an API

Two ways to search public Threads posts by keyword: Meta's gated official API, or a data API with no app review. Endpoints, fields, limits and code.

Read →

How to Scrape YouTube Comments with an API

Three ways to scrape YouTube comments in 2026: the official Data API, a DIY scraper, or a data API. What each costs, the quota math, and a CSV export.

Read →

Instagram Comments Viewer: See and Export Every Comment

See every comment on any public Instagram post without scrolling, and export to CSV, Excel or JSON. Read one post free, or pull comments at scale by API.

Read →

Best Social Media APIs in 2026

The best social media API depends on one question: do you want to post, or to read? Posting, data and analytics APIs, compared honestly.

Read →

TikTok Comment Viewer: How to Read and Export Comments

How to view every comment on a public TikTok video and export them to CSV or JSON. Read them in seconds, or pull them at scale with an API for analysis.

Read →

LinkedIn Ad Library API: Pull Competitor Ads as JSON

LinkedIn's Ad Library is a website with no usable API. How to pull competitor LinkedIn ads as JSON by keyword, and what the data does and doesn't include.

Read →

Best LinkedIn Scraping APIs in 2026

The biggest LinkedIn API shut down in 2025. Here are the LinkedIn scraping APIs still standing in 2026, compared by data, price, method, and compliance.

Read →

Threads Has No Public API. Here's What You Can Still Get

Meta's Threads API is publish-only and scoped to your own account. Why there's no public read API, and how to still pull public Threads data as JSON.

Read →

ScrapeCreators Alternatives for Social Media Data (2026)

ScrapeCreators is fast and broad, but returns raw per-endpoint JSON you parse yourself. Alternatives compared by schema, price, and who each is for.

Read →

Best Social Search APIs in 2026

The best social search API depends on whether you need one platform, many in one call, or ongoing monitoring. Compared on coverage, cost, and fit.

Read →

Best Instagram Data APIs in 2026

The best Instagram data API depends on the job. Seven providers compared by how they bill, what it takes to start, coverage, and who each one fits.

Read →

How to Search Reddit by Keyword with an API

Search Reddit by keyword with an API: the 1,000-result cap, how to search comments and old posts, getting past the limit, and monitoring mentions.

Read →

How to Scrape Reddit with an API

How to scrape Reddit in 2026: the four methods, why the .json trick keeps breaking, how to handle 429 and 403 blocks, and the no-maintenance API path.

Read →

Reddit API Pricing: What It Costs and What to Use Instead

What the Reddit API costs in 2026 ($0.24 per 1,000 calls), what the free tier gets you, why Reddit started charging, and the alternatives for Reddit data.

Read →

Apify Alternatives for Social Media Data (2026)

Apify is a great general scraping platform, but pricey and fiddly for social data. Honest alternatives compared by price, coverage, and who each is for.

Read →

How to Scrape Instagram Comments with an API

Pull every comment on a public Instagram post or reel with one GET request. Real response shape, pagination, credit cost, and the replies caveat.

Read →

Social Listening APIs: What You Need to Build One

The data layer behind a social listening tool: search, comments, stats and profiles across 30+ platforms. What to build, what to buy, and what it costs.

Read →

LinkedIn API Pricing: What It Costs and What You Can't Get

What LinkedIn's official API actually costs, why most developers can't get profile or company data from it, and a priced alternative for the data you need.

Read →

How to Get a TikTok Transcript (Free Tool and API)

Get the full text of any public TikTok in seconds. Paste a URL into a free tool, or pull transcripts at scale with one GET request. Real data, real limits.

Read →

How to Get a TikTok Video Transcript in Python

Pull the spoken text from any public TikTok in Python with one GET request. Real response shape, why segments is null, and error handling.

Read →

Why Your Video Transcript Has No Timestamps

Transcripts built from platform caption files come back as flat text with segments null. Here is why, which platforms do it, and how to code for it.

Read →

Prefer a reader? Subscribe via RSS.