Products - Elasticsearch Search Documentation
Location: api/nextango/apps/products/documents.py, services.py (ProductSearchService), views.py (ProductSearchAPIView, ProductSmartFilterAPIView), smart_filter_service.py
Last Updated: 2026-07-06
Overview
The Products app uses Elasticsearch for product search:
- Full-text search across product names, descriptions, and variant data
- Fuzzy matching for typo tolerance
- Nested variant search for SKUs, attributes, and per-store inventory
- Aggregated pricing and inventory snapshot data
- Auto-completion support via a completion sub-field on
name - Custom analyzer configuration on the index
The ProductDocument class defines the Elasticsearch mapping for Product indexing. Indexing reads the canonical ProductVariant records (product.variant_records) and each variant's prefetched InventoryLevel rows, not the Product.variants JSONField; the JSONField remains the Sanity sync structure consumed elsewhere (serializers, webhook sync) but no longer feeds the index.
ProductDocument
Location: documents.py:11-327
Index Name: products
Elasticsearch document for the Product model with embedded variant data.
Document Structure
Product Fields
| Field | ES Type | Description | Features |
|---|---|---|---|
sanity_id | KeywordField | Sanity CMS document ID | Exact match |
name | TextField | Product name | Standard analyzer, keyword + suggest sub-fields |
slug | KeywordField | URL-friendly identifier | Exact match |
hidden | BooleanField | Hide from storefront | Filter |
shortDescription | TextField | Short product description | Standard analyzer |
body | ObjectField | Rich text content (Portable Text) | Nested structure |
highlights | KeywordField | Array of product highlights | Multi-value |
baseDimensions | ObjectField | Base dimensions (width, height, depth, weight) | Structured data |
productNotes | TextField | Internal team notes | Standard analyzer |
basePromoCode | KeywordField | Sanity promo code reference | Exact match |
created_at | Date | Mapped from the Django model | Sort (smart filter newest) |
updated_at | Date | Mapped from the Django model |
Organization Fields
Nested within organization ObjectField:
| Field | ES Type | Description |
|---|---|---|
productType | KeywordField | ProductType sanity_id, extracted from the _ref |
category | KeywordField | Category sanity_id, derived from the ProductType's category |
brand | KeywordField | Brand sanity_id, extracted from the _ref |
collections | KeywordField | Array of Collection sanity_ids |
tags | KeywordField | Tags array |
SEO Fields
Nested within seo ObjectField:
| Field | ES Type | Description |
|---|---|---|
title | TextField | SEO title |
description | TextField | SEO description |
Aggregated Variant Fields
Computed from the canonical ProductVariant records at index time:
| Field | ES Type | Description | Computation |
|---|---|---|---|
variant_names | TextField | All variant names | Array from variants[].name |
variant_skus | KeywordField | All variant SKUs | Array from variants[].sku |
total_inventory | IntegerField | Initial-stock snapshot total | Sum of variants[].stockInitial, excluding digital-inventory variants |
has_inventory | BooleanField | Snapshot availability | True if any variant is digital or has stockInitial > 0 |
featured_variants_count | IntegerField | Count of featured variants | Count where featured=True |
on_sale_variants_count | IntegerField | Count of sale variants | Count where onSale=True |
total_inventory and has_inventory are document-level snapshots built from the immutable stockInitial value. They are not authoritative for store-scoped availability. Store-scoped truth lives in the nested variants[].inventory_by_store entries, which index live InventoryLevel data.
Price Range Fields
Nested within price_range ObjectField:
| Field | ES Type | Description | Computation |
|---|---|---|---|
min_price | FloatField | Minimum price | Min of variants[].price |
max_price | FloatField | Maximum price | Max of variants[].price |
min_sale_price | FloatField | Minimum sale price | Min of variants[].salePrice where onSale=True |
max_sale_price | FloatField | Maximum sale price | Max of variants[].salePrice where onSale=True |
Nested Variants
Array of variant objects stored as NestedField:
| Field | ES Type | Description |
|---|---|---|
name | TextField | Variant name |
sku | KeywordField | Stock Keeping Unit |
featured | BooleanField | Featured flag |
onSale | BooleanField | On sale flag |
price | FloatField | Regular price |
salePrice | FloatField | Sale price |
stockInitial | IntegerField | Immutable initial-stock snapshot (camelCase, matches the Sanity variant shape) |
digitalInventory | BooleanField | Unlimited-inventory sentinel for digital variants |
attributes | NestedField | Variant attributes (attribute keyword, value keyword) |
dimensions | ObjectField | Variant dimensions (width, height, depth, weight) |
inventory_by_store | NestedField | Live per-store inventory (see below) |
prepare_variants also includes image_urls (from ProductVariant.images) in each indexed variant entry.
Nested inventory_by_store
Each entry mirrors a live InventoryLevel row for the variant:
| Field | ES Type | Description |
|---|---|---|
store_id | KeywordField | Django store UUID |
store_sanity_id | KeywordField | Store Sanity ID |
store_name | KeywordField | Store name |
stock_on_hand | IntegerField | Live on-hand count |
stock_reserved | IntegerField | Live reserved count |
is_available | BooleanField | True when available quantity is positive (or the variant is digital) and the inventory row is active |
Index Configuration
Location: documents.py:104-129
Index Settings
{
'number_of_shards': 1,
'number_of_replicas': 0,
'analysis': {
'analyzer': {
'product_analyzer': {
'type': 'custom',
'tokenizer': 'standard',
'filter': [
'lowercase',
'asciifolding',
'stop',
'stemmer'
]
}
},
'filter': {
'stemmer': {
'type': 'stemmer',
'language': 'english'
}
}
}
}
Custom Analyzer Features
product_analyzer:
- Tokenizer: Standard (Unicode Text Segmentation)
- Filters:
lowercaseconverts to lowercaseasciifoldingconverts unicode to ASCII (café → cafe)stopremoves common stop wordsstemmerapplies English stemming (running → run)
Field-Specific Analyzers
name field:
name = fields.TextField(
analyzer='standard',
fields={
'keyword': fields.KeywordField(), # Exact match sorting
'suggest': fields.CompletionField(), # Auto-completion
}
)
- Main field: Standard analyzer for full-text search
- name.keyword: Exact match for sorting/aggregations
- name.suggest: Completion suggester for autocomplete
Document Preparation Methods
get_queryset()
Location: documents.py:139-151
Filters out hidden products at indexing time and prefetches the canonical variant and inventory graph, so every prepare_* method below reads ProductVariant and InventoryLevel records instead of running one query per variant.
def get_queryset(self):
return (
super().get_queryset()
.filter(hidden=False)
.prefetch_related('variant_records__inventory_levels__store')
)
Only visible products are indexed in Elasticsearch.
prepare_variant_names(instance)
Location: documents.py:153-155
Aggregates variant names from instance.variant_records.all() for text search.
Returns: ['Small - Red', 'Medium - Blue', 'Large - Green']
prepare_variant_skus(instance)
Location: documents.py:157-159
Aggregates variant SKUs from instance.variant_records.all() for exact matching, skipping variants without a SKU.
Returns: ['SHIRT-SM-RED', 'SHIRT-MD-BLU', 'SHIRT-LG-GRN']
prepare_price_range(instance)
Location: documents.py:161-180
Calculates price ranges from the canonical ProductVariant records. Sale prices count only where the variant has on_sale=True. Returns None when the product has no variant records.
Returns:
{
'min_price': 29.99,
'max_price': 59.99,
'min_sale_price': 24.99,
'max_sale_price': 49.99
}
prepare_organization(instance)
Location: documents.py:182-237
Transforms the organization JSONField for indexing. Extracts the _ref string from the productType and brand reference objects, collects collection _ref values into an array, and derives category by looking up the ProductType record and reading its category's sanity_id. Missing or unresolvable references index as empty strings. organization is the one part of this document that still reads a Product JSONField directly; it holds Sanity taxonomy references, not variant data.
prepare_total_inventory(instance)
Location: documents.py:239-246
Sums stock_initial across the product's non-digital ProductVariant records. This is the document-level initial-stock snapshot, not live on-hand inventory. Store-scoped reads use the nested inventory_by_store data instead.
def prepare_total_inventory(self, instance):
return sum(
pv.stock_initial for pv in instance.variant_records.all()
if not pv.digital_inventory
)
prepare_has_inventory(instance)
Location: documents.py:248-255
Returns True if any variant record is digital or has stock_initial > 0. A snapshot signal used only for non-store-scoped browsing. Store-scoped availability comes from inventory_by_store.is_available.
def prepare_has_inventory(self, instance):
return any(
pv.digital_inventory or pv.stock_initial > 0
for pv in instance.variant_records.all()
)
prepare_featured_variants_count(instance)
Location: documents.py:257-259
Counts ProductVariant records with featured=True.
prepare_on_sale_variants_count(instance)
Location: documents.py:261-263
Counts ProductVariant records with on_sale=True.
prepare_variants(instance)
Location: documents.py:265-327
Builds the nested variant entries from the canonical ProductVariant records (instance.variant_records.all()), enriching each with its prefetched InventoryLevel rows rather than the derived Product.variants JSONField, which is empty for Django-authored variants and cannot back store-scoped availability:
- Transforms Sanity attribute reference objects to plain
_refstrings for Elasticsearch compatibility - Copies
ProductVariant.imagesintoimage_urls - Builds
inventory_by_storeentries from each variant's prefetchedinventory_levels, withstore_id,store_sanity_id,store_name,stock_on_hand,stock_reserved, andis_available - Computes
is_availableas positive available quantity (on hand minus reserved) ordigital_inventory, gated on the inventory row being active - Logs and continues if the inventory lookup fails for a variant, so a single bad record does not break indexing
Each entry carries name, sku, featured, onSale, price, salePrice, stockInitial, digitalInventory, attributes, dimensions, inventory_by_store, and image_urls. The field names are unchanged from before this rewrite; only the source of the data changed, from Product.variants to the ProductVariant model.
Search API Endpoint
GET /search/
View: ProductSearchAPIView (views.py:987-1065)
Auth: AllowAny
Validates query parameters with ProductSearchSerializer, runs the search through ProductSearchService.search, and serializes hits with ProductDocumentSerializer.
Query Parameters (serializers.py:906-917):
| Parameter | Type | Description |
|---|---|---|
q | string | Full-text search query |
category | string | Category/type filter, matched against organization.productType |
brand | string | Accepted and echoed in filters; the service applies no brand filter to the ES query |
min_price | decimal | Minimum price filter (nested variant range) |
max_price | decimal | Maximum price filter (nested variant range) |
in_stock | boolean | Snapshot availability filter; ignored when store_id is present |
store_id | string | Store scope, accepts a Django UUID or a Sanity store ID |
featured | boolean | Restrict to products with a featured variant |
page | int | 1-based page number (default 1) |
page_size | int | Results per page (default 20, min 1, max 100) |
Response Shape:
{
"results": [
{
"id": "product-sanity-id",
"name": "Canvas Print",
"slug": "canvas-print",
"short_description": "A short description",
"hidden": false,
"variants": [
{
"name": "16x16",
"sku": "CNV-16",
"featured": true,
"onSale": false,
"price": 25.0,
"salePrice": null,
"stockInitial": 50,
"digitalInventory": false,
"attributes": [{"attribute": "attr-sanity-id", "value": "Red"}],
"dimensions": {},
"image_urls": [],
"inventory_by_store": [
{
"store_id": "store-uuid",
"store_sanity_id": "store-sanity-id",
"store_name": "Main Store",
"stock_on_hand": 12,
"stock_reserved": 2,
"is_available": true
}
]
}
],
"price_range": {"min_price": 25.0, "max_price": 60.0, "min_sale_price": null, "max_sale_price": null},
"has_inventory": true,
"total_inventory": 50
}
],
"total": 156,
"page": 1,
"page_size": 20,
"total_pages": 8,
"took": 12,
"query": "canvas",
"filters": {"store_id": "store-uuid"}
}
ProductDocumentSerializer (serializers.py:920-947) exposes id, name, slug, short_description (sourced from the ES shortDescription field), hidden, variants, price_range, has_inventory, and total_inventory.
Status codes: 400 for invalid parameters, 500 with "Search service temporarily unavailable" when the ES call fails.
Inventory Visibility (Capability-Keyed Strip)
Raw per-store quantities are visible only to privileged callers. _caller_sees_raw_inventory (views.py:84-94) returns true for POS-session principals and for authenticated users holding the employee or manager role. Everyone else, including anonymous callers, customers, and the storefront service principal, receives a stripped response.
For /search/, the strip removes stock_on_hand and stock_reserved from every variants[].inventory_by_store[] entry, keeping store_id, store_sanity_id, store_name, and is_available as the binary availability signal. The ES query shape is identical for all callers; only the response payload differs.
ProductSearchService
Location: services.py:597-746
search(query=None, filters=None, page=1, page_size=20)
Builds and executes the ES query for /search/. Returns the raw elasticsearch_dsl response carrying exactly the requested page of hits; hits.total.value is the full match count regardless of page.
Query building:
- With a query string:
multi_matchacrossname^3,shortDescription^2,variants.name^2,variants.sku^2, andorganization.tags, typebest_fields, fuzzinessAUTO - Without a query string:
match_all - Always filters
hidden=False
Filters:
store_idresolves throughresolve_store_id_param, accepting a Django UUID or Sanity ID. A resolved store adds a nested filter onvariants.inventory_by_storerequiring bothstore_sanity_idto match andis_available=True, so a zero-stock or fully reserved entry never matches. An unresolvablestore_idapplies no store filter; the search degrades to the unfiltered result rather than returning empty.in_stockapplies ahas_inventoryterm filter only when no store scope is present. Withstore_id, store availability is answered by the nestedis_availablefilter and the document-level snapshot term is dropped.featuredadds a nested filter onvariants.featured=True.categoryapplies a term filter onorganization.productType.min_price/max_priceadd nested range filters onvariants.price.
Sorting: _score first, then name.keyword ascending.
Pagination contract (ES from/size):
The 1-based page and page_size route directly to ES from/size, with from = (page - 1) * page_size. There is no post-execution re-slice. ES rejects from + size beyond max_result_window (10000), so the service clamps:
max_result_window = 10000
start = (page - 1) * page_size
if start >= max_result_window:
search = search[:0]
else:
end = min(start + page_size, max_result_window)
search = search[start:end]
A page starting at or beyond 10000 fetches zero hits but still reports the accurate total. A page that straddles the window is clamped to end at 10000.
Search Query Examples
Basic Full-Text Search
from elasticsearch_dsl import Search
# Search by product name
search = Search(index='products')
search = search.query('match', name='blue shirt')
response = search.execute()
for hit in response:
print(f"{hit.name}: {hit.slug}")
Multi-Field Search with Boosting
from elasticsearch_dsl import Q
# The field weights ProductSearchService.search uses
search = Search(index='products')
query = Q('multi_match', {
'query': 'nike running shoes',
'fields': [
'name^3', # Boost product name 3x
'shortDescription^2', # Boost description 2x
'variants.name^2', # Boost variant names 2x
'variants.sku^2', # Boost variant SKUs 2x
'organization.tags' # Tags standard weight
],
'type': 'best_fields',
'fuzziness': 'AUTO' # Typo tolerance
})
search = search.query(query)
Nested Variant Query
from elasticsearch_dsl import Q
# Search for products with featured variants
search = Search(index='products')
nested_query = Q('nested', {
'path': 'variants',
'query': Q('term', **{'variants.featured': True})
})
search = search.query(nested_query)
Store-Scoped Availability Query
from elasticsearch_dsl import Q
# Products available at a specific store (live inventory, not the snapshot)
search = Search(index='products')
store_filter = Q('nested', path='variants', query=Q(
'nested',
path='variants.inventory_by_store',
query=Q('bool', must=[
Q('term', **{'variants.inventory_by_store.store_sanity_id': 'store-sanity-id'}),
Q('term', **{'variants.inventory_by_store.is_available': True}),
]),
))
search = search.query('bool', filter=[store_filter])
Price Range Filter
from elasticsearch_dsl import Q
# Find products in price range $20-$50
search = Search(index='products')
price_filter = Q('nested', {
'path': 'variants',
'query': Q('bool', {
'must': [
Q('range', **{'variants.price': {'gte': 20.0}}),
Q('range', **{'variants.price': {'lte': 50.0}})
]
})
})
search = search.query(price_filter)
Combined Search with Filters
from elasticsearch_dsl import Search, Q
# Full-text search + filters
search = Search(index='products')
# Text query
text_query = Q('multi_match', {
'query': 'shirt',
'fields': ['name^3', 'shortDescription^2', 'variants.sku^2']
})
# Filters
filters = [
Q('term', hidden=False), # Not hidden
Q('term', has_inventory=True), # Snapshot availability
Q('nested', {
'path': 'variants',
'query': Q('term', **{'variants.onSale': True})
}) # On sale
]
# Combine
search = search.query('bool', must=[text_query], filter=filters)
search = search.sort('_score', {'name.keyword': {'order': 'asc'}})
Auto-Completion
from elasticsearch_dsl import Search
# Suggest products as user types
search = Search(index='products')
search = search.suggest('product_suggest', 'blu', completion={'field': 'name.suggest'})
response = search.execute()
for option in response.suggest.product_suggest[0].options:
print(option.text) # "Blue Shirt", "Blue Jeans", etc.
Indexing Operations
Manual Index Rebuild
from nextango.apps.products.documents import ProductDocument
from nextango.apps.products.models import Product
# Rebuild entire index
ProductDocument().update(Product.objects.filter(hidden=False))
Index Single Product
from nextango.apps.products.models import Product
from nextango.apps.products.documents import ProductDocument
product = Product.objects.get(slug='blue-shirt')
doc = ProductDocument()
doc.update(product)
Bulk Indexing
from django.core.management import call_command
# Rebuild search index (Django command)
call_command('search_index', '--rebuild', '-f')
Performance Considerations
Index Size
- Shards: 1 (single shard for small-medium catalogs)
- Replicas: 0 (no replicas for development; increase for production)
Query Patterns
- Use
filtercontext for exact matches (cached, no scoring) - Use
mustcontext for full-text search (scored) - Limit results with slicing to reduce data transfer
- Use
_sourcefiltering to return only needed fields
Indexing Behavior
- Hidden products never enter the index (
get_querysetfilter) - Reindex requests from model signals coalesce: many changes in one transaction collapse to a single Celery dispatch on commit
InventoryLevelsaves and deletes trigger reindexing soinventory_by_storestays current
See Signals for the reindex triggers.
Troubleshooting
Index Not Found
# Create index
python manage.py search_index --create
# Rebuild index
python manage.py search_index --rebuild -f
Stale Data
# Update specific products
python manage.py search_index --update
# Full rebuild
python manage.py search_index --rebuild -f
Mapping Changes
After changing field mappings:
# Delete old index
python manage.py search_index --delete -f
# Recreate with new mapping
python manage.py search_index --create
# Populate
python manage.py search_index --populate
Smart Product Filtering with Elasticsearch
Source: api/nextango/apps/products/smart_filter_service.py, views.py (ProductSmartFilterAPIView)
The smart filtering system serves the storefront product grid and carousels through a hybrid architecture: pre-generated manifest files for browsing, real-time Elasticsearch aggregations for search. It returns filter options with counts and available flags so the UI can disable options that would yield zero results.
Hybrid Routing
Manifest mode runs when there is no search query, a store_id is present, none of brand, category, productType, collection, attributes, on_sale, or low_stock is active (_MANIFEST_BYPASS_FILTERS, views.py:1312-1313), and a manifest file exists for that store. The view loads the manifest JSON, runs the product query with skip_aggregations=True, injects the manifest aggregations, and always recalculates the available flags live through get_available_options, so a stale manifest never shows an unavailable option as available.
Real-time mode runs in every other case (a search query is present, no store_id, any bypass filter is active, or no manifest file). execute_smart_search runs the aggregations and the product query against Elasticsearch in the same request.
Request: GET /search/smart-filter/
|
v
ProductSmartFilterAPIView.get
parse filters, page, page_size
|
no search query AND store_id present AND
no bypass filter active AND manifest exists?
| |
yes no
| |
Manifest mode Real-time mode
- load manifest JSON - ES aggregations
- product query only - product query
- inject manifest aggs - transform results
- recalc available flags
| |
+-----------+--------------+
|
capability-keyed strip
(drop stock_available for
non-privileged callers)
|
Response
SmartFilterService
Core methods:
-
build_search_query(query, filters, exclude_filter_types=None): Converts filter state into an ES query. Supports excluding specific filter types for availability calculation. Accepts Django UUIDs (converted to sanity_ids) for category, productType, and collection filters; brand values are sanity_ids. -
add_aggregations(search, filters): Adds aggregation buckets for filter options (brands, categories, productTypes, collections, attributes). -
execute_smart_search(query, filters, page, page_size, skip_aggregations=False): Executes the aggregation query (unpaginated, unless skipped) and the paginated product query, then flattens product documents into grid rows. Returnsproducts,total,total_variants,aggregations,page,page_size, and afiltersecho. -
get_available_options(filters): Calculates which filter options would yield results by running a separate query per filter type, excluding only the target type while keeping all other active filters. Returns sets of available sanity_ids per filter category. -
apply_availability_flags(aggregations, available_options): Appliesavailable: true/falseflags to manifest aggregations. Used in manifest mode.
API Endpoint
URL: GET /search/smart-filter/
Auth: AllowAny
Query Parameters:
| Parameter | Type | Description |
|---|---|---|
search | string | Full-text search query |
store_id | string | Store UUID; scopes inventory and enables manifest mode |
brand[] | array | Brand IDs to filter by |
category[] | array | Category IDs to filter by |
productType[] | array | ProductType IDs to filter by |
collection[] | array | Collection IDs to filter by |
attributes[Name][] | array | Attribute values (e.g., attributes[Color][]=Red) |
sort | string | featured (default), price-asc, price-desc, newest, name-asc, name-desc |
on_sale | boolean | true restricts to products with at least one on-sale variant |
low_stock | boolean | true restricts to products with a low on-hand variant (store-scoped when store_id is present) |
collapse_variants | boolean | true collapses each product to one card row (the /products/ grid); default emits one row per available variant (carousels) |
page | int | Page number (default 1) |
page_size | int | Results per page (default 20, max 100) |
Sort behavior (execute_smart_search): price-asc sorts on price_range.min_price, price-desc on price_range.max_price, newest on created_at descending, name-asc/name-desc on name.keyword. The default featured sorts on featured_variants_count descending, then name.keyword ascending.
Example Request:
GET /search/smart-filter/?search=canvas&brand[]=uuid-1&brand[]=uuid-2&category[]=uuid-3&attributes[Color][]=Red&attributes[Color][]=Blue&sort=price-asc&page=1&page_size=20
Response Structure:
{
"products": [
{
"id": "product-1_CNV-16-RED",
"product_name": "Canvas Print 16x16",
"product_slug": "canvas-print-16x16",
"name": "Red",
"product_short_description": "A short description",
"sku": "CNV-16-RED",
"price": 25.00,
"sale_price": 20.00,
"effective_price": 20.00,
"on_sale": true,
"is_in_stock": true,
"stock_available": 15,
"featured": true,
"store_name": "Main Store",
"image_urls": []
}
],
"total": 156,
"total_variants": 20,
"aggregations": {
"brands": [
{
"_id": "uuid-1",
"sanity_id": "brand-sanity-id-1",
"name": "Artisan Studio",
"count": 42,
"available": true
}
],
"categories": [
{
"_id": "uuid-3",
"name": "Wall Art",
"count": 156,
"available": true
}
],
"productTypes": [
{
"_id": "uuid-4",
"sanity_id": "pt-sanity-id-1",
"name": "Canvas",
"count": 89,
"available": true
}
],
"collections": [
{
"_id": "uuid-6",
"sanity_id": "coll-sanity-id-1",
"name": "Summer 2024",
"count": 23,
"available": true
}
],
"attributes": {
"featured": [
{
"name": "Color",
"attribute_id": "attr-uuid",
"featured": true,
"display_order": 1,
"colors": [
{"name": "Red", "hex": "#FF0000", "count": 12, "available": true},
{"name": "Blue", "hex": "#0000FF", "count": 8, "available": false}
]
}
],
"regular": []
}
},
"page": 1,
"page_size": 20,
"filters": {
"brand": ["uuid-1", "uuid-2"],
"category": ["uuid-3"],
"attributes": {"Color": ["Red", "Blue"]},
"sort": "price-asc"
}
}
The capability-keyed strip applies here too: non-privileged callers (anonymous, customers, the storefront service principal) receive product rows without stock_available; is_in_stock and store_name remain. Staff (employee/manager) and POS sessions keep the full shape.
Row shapes:
- Default mode (carousels): one row per store-available variant, no
card_kind. collapse_variants=true(the grid): one row per product. A single matching variant emits that variant row withcard_kind: "variant". Multiple matching variants emit a product card withcard_kind: "product"and a price range. Variant-discriminating filters (attributes,on_sale,low_stock) narrow the matched variants before collapsing.
Smart Availability Flags:
The available boolean on each aggregation item indicates whether selecting that option would yield results given current filters. When available: false, the UI should disable that option to prevent incompatible filter combinations.
Manifest Generation System
Celery Task: nextango.apps.products.tasks.generate_filter_manifests
Schedule: Every 15 minutes via Celery Beat (crontab(minute='*/15'))
Management Command: python manage.py generate_filter_manifests
The Celery task is a chord dispatcher: it fans active store IDs out to generate_filter_manifests_chunk tasks and finalizes with generate_filter_manifests_finalize. The management command runs the same per-store helper (generate_store_manifest) serially. Each manifest is built by running execute_smart_search with only the store_id filter and capturing the aggregations:
{
"store_id": "74c0cd56-f6f8-40c6-87f9-73e587155fa3",
"store_name": "Main Store",
"generated_at": "2026-06-12T14:00:00",
"aggregations": {
"brands": [
{"_id": "...", "sanity_id": "...", "name": "Brand A", "count": 42}
],
"productTypes": [],
"collections": [],
"categories": [],
"attributes": {"featured": [], "regular": []}
}
}
Storage: {MEDIA_ROOT}/manifests/store_{store_id}.json
Manifest aggregations can go stale between regenerations. The view compensates by recalculating available flags on every manifest-mode request.
Smart Availability Calculation
When the view serves a manifest, it calculates which options would yield results:
For each filter type (brand, category, productType, collection, attributes):
- Build an ES query with all filters except the target type
- Execute the aggregation for that type only
- Extract available option IDs from the results
- Apply
available: true/falseflags to the manifest data
Example: the user selects an office-furniture brand. The productTypes query excludes the productType filter and returns only types that brand carries, so incompatible types render disabled. This prevents selections that would return zero products.
Elasticsearch Aggregations
Aggregations work like SQL GROUP BY with COUNT, calculated after filters are applied, so counts reflect what would be available given current selections:
// Elasticsearch aggregation
{
"aggs": {
"brands": {
"terms": {
"field": "organization.brand",
"size": 100
}
}
}
}
// Result
{
"aggregations": {
"brands": {
"buckets": [
{"key": "brand-sanity-id-1", "doc_count": 42},
{"key": "brand-sanity-id-2", "doc_count": 15}
]
}
}
}
Frontend Integration
The storefront consumes this endpoint through two modules:
-
storefront/web/src/lib/stores/manifestStore.ts: Zustand store that caches manifests per store.loadManifest(storeId)fetches/api/media/manifests/store_{storeId}.json(a Next.js proxy route to Django's/media/manifests/), skips reloads when cached or in flight, and retries after a prior error. -
storefront/web/src/lib/hooks/useSmartFiltersQuery.ts: React Query hook for the/products/grid. Sends the filter store state pluscollapse_variants=true, keyed by store. It is store-aware: SSR server data and React Query placeholder data are shown only for the store they were fetched for, so a store switch shows loading rather than another store's products.staleTimeis 30 seconds; the query is enabled only when a store ID is resolved.
Rendering availability flags:
{data.aggregations.brands.map(brand => (
<FilterOption
key={brand._id}
label={brand.name}
count={brand.count}
disabled={!brand.available}
selected={selectedBrands.includes(brand._id)}
onChange={() => toggleBrand(brand._id)}
/>
))}
Related Documentation
- Models - Product model structure
- Serializers - ProductSearchSerializer and ProductDocumentSerializer
- Services - ProductSearchService implementation
- Views - Search API endpoints
- Smart Filter Service - Smart filter internals
- Signals - Reindex triggers
- Examples - Complete search examples
File Reference
Backend:
Elasticsearch document definition: api/nextango/apps/products/documents.py
- ProductDocument: documents.py:11-327
- Index configuration: documents.py:104-129
- Preparation methods: documents.py:139-327
Search service: api/nextango/apps/products/services.py
- ProductSearchService.search: services.py:597-746
Search views: api/nextango/apps/products/views.py
- ProductSearchAPIView: views.py:987-1065
- ProductSmartFilterAPIView: get(), _try_manifest_mode(), _parse_filters()
- Capability seam
_caller_sees_raw_inventory: views.py:84-94 - Response strip helpers: views.py:49-69
Smart Filter Service: api/nextango/apps/products/smart_filter_service.py
- generate_store_manifest(store, manifests_dir, dry_run=False)
- SmartFilterService: build_search_query, add_aggregations, get_available_options, execute_smart_search, apply_availability_flags
Manifest Generation:
- Task:
api/nextango/apps/products/tasks.py(generate_filter_manifests chord dispatcher, _chunk, _finalize) - Management Command:
api/nextango/apps/products/management/commands/generate_filter_manifests.py - Celery Beat Schedule:
api/nextango/celery.py(every 15 minutes)
Frontend:
Manifest Store: storefront/web/src/lib/stores/manifestStore.ts
React Query Hook: storefront/web/src/lib/hooks/useSmartFiltersQuery.ts
Manifest Files:
- Location:
{MEDIA_ROOT}/manifests/store_{store_id}.json - Regenerated every 15 minutes by Celery Beat
- Contain pre-computed aggregations per store; availability flags recalculated at request time