Semantic model

This page defines the stable meaning of Extralt data. It is not a description of the physical ClickHouse layout.

Use the OpenAPI reference for exact request and response schemas. Use this page when you need to understand what one record represents, how records relate, and which facts can be compared safely.

Two data layers

Extralt exposes two related layers:

1. Queryable records preserve page observations and processing lineage. The constrained SQL surface currently supports captures, items, and changes.
2. Extend entities connect observations into Product, Variant, Listing, Offer, Review, and Store relationships. They are currently exposed through purpose-built Explore views and APIs, not as arbitrary public SQL tables.

Do not assume that a ClickHouse table exists, or that it can be queried, because an entity appears in this model. See the access-surface matrix for the current contract.

Queryable records

The types in this section are the types visible to the constrained SQL query surface. API payloads can represent the same values differently; OpenAPI is the authority for JSON payloads.

Only the fields listed below are part of the documented query contract. Select columns explicitly rather than depending on SELECT *: service-scoping, search-index, or reserved columns can be present and can change without becoming customer metrics.

captures

Grain: one successfully extracted or imported page record.
Identity: id.
Lineage: Run or Import -> Capture.
Use it for: source evidence, extraction output, exports, and Enrich input.

Identity, source, and time

Source product fields

Commerce summary and collections

JSON paths vary with the source evidence. Inspect representative values before depending on a nested path.

items

Grain: one successfully enriched Capture.
Identity: id.
Lineage: Capture -> Enrichment -> Item.
Use it for: normalized, page-grain product data and Extend input.

The Item API also returns derived summaries such as offer counts and price ranges. Those values are computed from skus; they are not columns available to SQL queries. Use the OpenAPI Item schema for the current response fields.

An Item is not a canonical Product, and Enrich does not publish final Offers. Extend resolves Item evidence into the entities below.

changes

Grain: one observed change to one entity aspect.
Identity: id.
Lineage: Run -> observed entity state -> Change.
Use it for: explaining what changed, when it changed, and which state was replaced.

Extend entity model

Extend connects page-grain Items into comparable ecommerce entities:

Product -> Variant -> Listing -> Offer observations
                         |-> Review aggregate
Store -------------------+

Extend and Explore are in beta. The relationships below are useful product semantics, not a promise of unrestricted SQL access to physical tables.

Product

Grain: one normalized product family in an organization dataset.
Identity: Extralt Product id.
Contains: canonical family title, brand, description, taxonomy, attributes, signals, option axes, and selected media.
Relationship: parent of one or more Variants.

Variant

Grain: one exact canonical configuration under a Product.
Identity: Extralt Variant id; identifiers such as GTIN or MPN when resolved are evidence, not guaranteed primary keys.
Contains: selected option values, variant media, identifiers, and first-known time.
Relationship: belongs to one Product and connects equivalent Listings.

Listing

Grain: one source SKU identity in one Store and market.
Identity: Extralt Listing id plus its source SKU evidence.
Contains: source URL, source title, identifiers, media, market, and first, last, removal, and invalid-status observation times.
Relationship: belongs to one currently resolved Variant and one Store.

Offer

Grain: one observed seller, price, availability, condition, and stock state for a Listing at one observation time.
Identity: Extralt Offer observation id.
Contains: price, compare-at price, currency, seller, availability, condition, and categorical stock level.
Relationship: belongs to a Listing and retains Capture and Run lineage.

An Offer is observed evidence, not a continuously synchronized price. A compare-at price can show a visible markdown; it does not prove that a coupon or checkout-only promotion was available.

Review

Grain: latest normalized review aggregate for one Listing and Store.
Contains: review count and score normalized to 0-100.
Does not contain: review text, sentiment, or review history.

Store

Grain: one host and country pair in an organization dataset.
Identity: deterministic Store id referenced by Items, Listings, Offers, and Reviews.
Contains today: host and country.

A Store is an analytical entity. A Robot is operational extraction tooling for a host and country. They should not be treated as the same object. The current Store model does not claim retailer ownership, company hierarchy, DTC status, traffic, sales, rank, revenue, or turnover.

Interpretation rules

• Identifiers are scoped to the organization's dataset unless the API says otherwise.
• Empty or zero-valued source fields can mean that the source did not expose a fact. Do not turn absence into a measured zero without checking coverage.
• “Current” means the latest observation represented by the product surface, not a continuous market snapshot.
• Prices should be compared within a country and currency unless the calling workflow supplies an explicit conversion source and timestamp.
• Product, Variant, and Listing relationships depend on Extend matching and its available identifiers and evidence.

Continue with query semantics for time, coverage, availability, and current-state rules.