Data model
Extralt separates source observations from normalized records and cross-store entities. That separation keeps a page that was actually observed distinct from a Product or Variant inferred across several sources.
Lineage
Build
Create the customer dataset
Extract
Collect source product pages and catalogs
Enrich
Normalize Captures and publish the analytical model
Explore
Use the customer dataset
Inspect and query the data built by Extract and Enrich.
Data follows the same Extract and Enrich stages shown in the dashboard. Operational records coordinate each stage; the resulting Captures, Items, relationships, and observations belong to the organization's analytical dataset. Explore reads that dataset.
Entity grains
| Entity | One record represents | Primary lineage or relationship |
|---|---|---|
| Capture | One successfully extracted or imported page | Run or Import, source URL, observation time |
| Item | One successfully enriched Capture | capture_id, enrichment_id |
| Product | One normalized product family | Parent of one or more Variants |
| Variant | One exact canonical configuration | Product and equivalent Listings |
| Listing | One source SKU identity in one Store and market | Variant, Store, source URL |
| Offer | One seller, price, and availability observation | Listing, Capture, Run, observation time |
| Review | One normalized review aggregate observation | Listing, Store, Run, observation time |
| Store | One host and country pair | Referenced by Items, Listings, Offers, and Reviews |
| Change | One observed change to one entity aspect | Entity, Run, observation time |
Observations and current entities
Captures and Items are page-grain observations with source URL and time lineage. Offers and Changes are also observations: they record a fact at a point in time.
Products, Variants, Listings, and Stores represent current dimensions. Reviews are append-only observations; current Review views select the latest known aggregate for each active Listing. The API and Explore views own that current-state contract; callers do not need to reason about ClickHouse merge behavior or storage engines.
Operational and analytical names
A Robot, Run, or Enrichment belongs to the control plane and configures or tracks processing work. A Capture, Item, Product, Variant, Listing, Offer, Review, Store, or Change belongs to the analytical customer dataset.
A Robot contains extraction logic for a host and country. A Store currently represents a host and country in one organization's analytical dataset. They may describe the same website and market, but they do not share an identity or lifecycle. Creating or using a Robot does not create a globally shared retailer entity.
Scope boundaries
The dataset contains observed public product evidence. It does not contain sales, transactions, revenue, turnover, traffic, market share, demand, margin, or promotion codes that were not visible in the extracted data.
Coverage is conditional on the stores, countries, URLs, and observation times the organization selected. “Current” means the latest observation in that dataset, not a continuously synchronized view of the whole market.
Continue with the access-surface matrix, the semantic model, and query semantics.