What the Data for This Category Looks Like
TCM research report data comes primarily from brokerages’ pharmaceutical industry research reports, public reports from TCM industry associations, R&D and sales documents from listed pharmaceutical companies, and academic journal papers on TCM. Updates follow a quarterly and annual cycle, with temporary documents released during sudden industry policy changes or herbal material price fluctuations. Document structures include sections such as abstract, herbal material origin, nature, flavor, and meridian tropism, clinical applications, pharmacological research, quality standards, and industrial data. These documents contain extensive tables and mathematical formulas. Fields include herbal material name, origin, content units (such as mg/g, %), price units (such as yuan per kilogram), processing parameters (such as minutes, multiple ratios) and other specialized content.
What Constraints These Characteristics Impose on Document Parsing and Chunking
The multi-source formats and specialized content features of TCM research reports impose multiple constraints on the parsing and chunking process. Documents exist in multiple formats including scanned PDFs, Word files, and plain text. Parsing logic must adapt to these different formats. Embedded tables and mathematical formulas must be fully identified, otherwise core quantitative data will be lost. Specialized terminology and cross-chapter related content require that chunking does not break semantic integrity. Splitting different research modules for the same herbal material into separate chunks must be avoided. During batch parsing, balance must be maintained between parsing time for long documents and content completeness. Timeouts or broken chunks must be prevented.
Configuration Settings
| Configuration Key | Recommended Value | Rationale |
|---|---|---|
PARSE_ENGINE | marker | This engine’s parsing performance for scanned PDFs, mathematical formulas, and embedded tables aligns with the complex formatting of TCM research reports |
PARSE_TABLE_ENABLE | enabled | TCM research reports contain extensive tables of herbal component and industrial data; disabling this will lose structured core information |
MAX_CHUNK_SIZE | 1200–1500 characters | The complete content for single herbal material research or clinical cases typically falls within this range. Values that are too long will cause redundant context, while values that are too short will break semantic coherence |
PARSE_MATH_FORMULA | enabled | Reports include formulas for active ingredient ratios, pharmacological dose calculations, and similar content; disabling this will lose critical quantitative data |
UPLOAD_FILE_MAX_SIZE | 200 MB | Some large industry research report collections have significant file sizes; a smaller limit will prevent uploading complete documents |
PARSE_FILE_TIMEOUT_SECONDS | 900 seconds | Long documents with multiple chapters and complex charts require extended parsing time; this setting prevents mid-process timeout failures |
The parameter values provided on this page are common starting points for configuration. Actual values are affected by material format, data volume, and business rules. Specific issues require case-by-case analysis. It is recommended to test on your own samples before finalizing settings.
Three Common Mistakes
- Symptom: Clicking the chunk preview button for a TCM research report PDF in a knowledge base returns an error stating "Unable to read the file content". Cause: Table and formula parsing configurations are not enabled, or the parsing engine does not support complex formatting.
- Symptom: The 4.9.0 online version correctly returns formula results from research reports, but the local 4.8.12 version cannot parse formulas. Cause: The local parsing engine does not include a built-in formula parsing module, or the
PARSE_MATH_FORMULAconfiguration item is not enabled. - Symptom: An error stating container image startup failure appears when executing the docker deployment command for the parsing engine. Cause: An outdated image version (v0.1) is used, or GPU mounting parameters are not configured correctly.
How to Verify Proper Configuration
- Upload a sample TCM research report that includes ingredient tables and pharmacological formulas, and confirm that the parsed text retains structured tables and formula content in full.
- Navigate to the file management page of the knowledge base, select the uploaded research report, click chunk preview, and confirm that the chunked content is visible with no obvious semantic breaks.
- Check the parsing engine’s runtime logs to confirm no timeout or format parsing failure error messages appear.
- Test retrieval of specialized terminology from the research report, and confirm that chunked content can be correctly recalled with no critical information lost.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.