# How to Extract Metadata from Web Pages (Open Graph, Schema.org, and Twitter Cards)
Web pages carry structured metadata in Open Graph tags, Schema.org JSON-LD, and Twitter Card elements. This guide walks through extracting all three protocols with Python and JavaScript, handling JavaScript-rendered pages, and building a pipeline that pulls consistent data from any URL.
Source: https://fast.io/resources/metadata-extraction-from-web-pages-open-graph/
Last reviewed: 2026-04-23
## How Web Page Metadata Works
Every web page carries hidden metadata that controls how it appears when shared on social media, how search engines understand its content, and what information crawlers can extract programmatically. Three protocols dominate this space, and understanding how they differ is the first step toward extracting them reliably.
**Open Graph (OG) tags** were introduced by Facebook in 2010 to control how URLs render in social feeds. They live in the HTML `` as `` tags with a `property` attribute prefixed by `og:`. According to W3Techs, 70.5% of all websites now include Open Graph tags, making them the most widely adopted structured metadata format on the web.
```html
```
**Schema.org markup** serves a different purpose. Created jointly by Google, Microsoft, Yahoo, and Yandex, Schema.org provides a vocabulary for describing entities like articles, products, events, and organizations. The most common implementation is JSON-LD (JavaScript Object Notation for Linked Data), embedded in a `
```
**Twitter Cards** use `` tags with a `name` attribute prefixed by `twitter:`. They control how links appear in posts on X (formerly Twitter) and follow similar conventions to Open Graph, though they support card-specific features like `twitter:card` types (`summary`, `summary_large_image`, `player`).
```html
```
Most well-maintained websites include all three. When building an extraction pipeline, you want to pull from each protocol and merge the results, since different platforms populate different fields.
Helpful references: [Fastio Workspaces](/product/workspaces/), [Fastio Collaboration](/product/collaboration/), and [Fastio AI](/product/ai/).
## Extracting Open Graph Tags
Open Graph tags are the easiest metadata to extract because they follow a consistent pattern: ``. Here are working examples in both Python and JavaScript.
### Python with BeautifulSoup
```python
import requests
from bs4 import BeautifulSoup
def extract_og_tags(url):
response = requests.get(url, headers={"User-Agent": "MetadataBot/1.0"})
soup = BeautifulSoup(response.text, "html.parser")
og_tags = {}
for tag in soup.find_all("meta", attrs={"property": True}):
prop = tag.get("property", "")
if prop.startswith("og:"):
og_tags[prop] = tag.get("content", "")
return og_tags
### Example usage
metadata = extract_og_tags("https://example.com/article")
print(metadata)
### {'og:title': 'Project Management Guide',
### 'og:description': 'A practical guide...',
### 'og:image': 'https://example.com/cover.jpg'}
```
A few practical notes on this approach. Always send a `User-Agent` header, because many sites return different content (or block requests entirely) when the user agent is missing. Use `html.parser` for speed, but switch to `lxml` if you need to handle malformed HTML more gracefully. Check for `response.status_code` before parsing, since a 403 or 404 response will waste time in the parser.
### Node.js with open-graph-scraper
The `open-graph-scraper` npm package handles fetching, parsing, and normalizing OG tags in a single call:
```javascript
import ogs from "open-graph-scraper";
async function extractOG(url) {
const { result } = await ogs({ url });
return {
title: result.ogTitle.
description: result.ogDescription.
image: result.ogImage?.[0]?.url.
type: result.ogType.
siteName: result.ogSiteName.
};
}
const metadata = await extractOG("https://example.com/article");
console.log(metadata);
```
The `open-graph-scraper` library also extracts Twitter Card data and standard meta tags as fallbacks, which saves you from writing separate parsers for each protocol. For batch extraction, the `metascraper` package from Microlink is another strong option. It uses a plugin architecture where you compose rule sets for different metadata sources.
## Parsing Schema.org JSON-LD from HTML
JSON-LD extraction follows a different pattern from OG tags. Instead of scanning meta elements, you look for `