What the data for this category looks like
Data sources for glass investment research include monthly production capacity reports from industry associations, spot price data from futures exchanges, public announcements from manufacturing enterprises, and downstream real estate and construction material bidding information. Update frequencies vary by data type: spot prices are updated daily, industry research reports weekly, and production capacity and bidding data monthly. Document structures mix structured tables and long-form text analysis, with dedicated fields including glass thickness (mm), unit price (yuan/square meter), annual production capacity (10,000 tons), and more. Some CSV files contain bulk spot price entries.
What constraints do these characteristics impose on vector models and indexing?
Mixed structured and unstructured data requires vector models to adapt to both field semantics and long-text analysis. Daily updated spot data needs incremental indexing to avoid wasting resources on full reindexing. Glass-specific units and model fields require vector encoding to retain semantic details, avoiding encoding bias from general-purpose models. Bulk CSV data with over 100,000 entries needs batch processing to prevent overloading a single indexing batch. Long-form research reports require proper segmentation to avoid truncating key terms like glass models and thickness.
Configuration Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
EMBEDDING_MODEL | text-embedding-3-small | Balances semantic accuracy for glass industry terminology and computational cost, and supports mixed structured and unstructured data |
CHUNK_SIZE | 800–1200 characters | Preserves complete semantics of key fields including glass models, thickness, and unit prices, avoiding cross-chunk truncation |
RECALL_TOP_K | Top 8–12 results | Balances recall needs for multi-source data including spot prices, research reports, and bidding information, avoiding missed critical investment research data |
INDEX_INCREMENTAL_ENABLE | Enabled | Adapts to the daily update rhythm of spot price data, reducing resource overhead from full reindexing |
UPLOAD_CSV_MAX_ROWS | 10,000 rows per batch | Splits 100,000+ row CSV data for upload, preventing timeouts caused by overly large single batches |
VECTOR_SCORE_THRESHOLD | 0.75–0.85 | Distinguishes semantic similarity between different glass thicknesses and models, filtering invalid recall results |
The parameter values provided on this page are common starting points for configuration. Actual values are affected by material forms, data volume, and business rules. Specific issues require case-by-case analysis, and it is recommended to test against your own samples before finalizing settings.
Three Common Mistakes
- Phenomenon: Vector calculation scores are abnormally high, and all results have identical scores. Cause: No reasonable range is configured for
VECTOR_SCORE_THRESHOLD, or a general-purpose vector model not adapted to construction industry terminology is used, leading to semantic encoding bias. - Phenomenon: Vectorization takes too long when uploading CSV files to the knowledge base. Cause: The batch upload parameter
UPLOAD_CSV_MAX_ROWSis not set, and overly large single batch data triggers server resource bottlenecks. - Phenomenon: After vectorization of 100,000-row CSV data, the total indexed entries only reach 90,000, with thousands missing. Cause: The CSV field validation switch is not enabled, or rows with formatting errors or null values are not filtered, leading to automatic skipping of invalid data.
How to Confirm Correct Configuration
- Upload a single test text containing glass models and thickness, verify the semantic similarity of the vector encoding results, and confirm that
CHUNK_SIZEdoes not truncate key terms. - Upload 100,000-row structured CSV data, use batch uploads, and verify that the final number of indexed entries matches the original data volume, confirming that
UPLOAD_CSV_MAX_ROWSis properly configured. - Run an incremental indexing task, verify that only newly updated daily data is correctly written to the index, confirming that
INDEX_INCREMENTAL_ENABLEis active. - Search for keywords related to glass thickness, verify that the distribution of
VECTOR_SCOREvalues matches the preset range, confirming that the similarity threshold configuration is correct.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.