index_diff #

Description #

The index_diff processor is used to compare differences between two result sets.

Configuration Example #

A simple example is as follows:

- name: bulk_request_ingest
  auto_start: true
  keep_running: true
    - index_diff:
        diff_queue: "diff_result"
        buffer_size: 1
        text_report: true #If data needs to be saved to Elasticsearch, disable the function and start the diff_result_ingest task of the pipeline.
        source_queue: 'source_docs'
        target_queue: 'target_docs'

Parameter Description #

source_queuestringName of source data
target_queuestringName of target data
diff_queuestringQueue that stores difference results
buffer_sizeintMemory buffer size
keep_sourceboolWhether difference results contain document source information
text_reportboolWhether to output results in text form