版本发布日志 #

这里是极限网关历史版本发布的相关说明。

1.9.0 #

Breaking changes #

• Refactoring config for ip access control
• Disable metadata refresh and node availability check by default
• Update default config path from configs to config

Features #

• Support listen on IPv6 address
• Add general health api
• Add request_ip to context
• Add badger filter plugin
• Allow to split produce and consume messages from s3
• Add bulk_request_throttle filter
• Support access request context and more output options in echo filter
• Add body_json to response context
• Add cert config to API module, support mTLS
• Add api to clear scroll context
• Floating_ip support stick by priority

Bug fix #

• Fix user was removed in logging filter
• Fix incorrect message size issue, reload when files changed in disk_queue
• Fix issue that index_diff could not finished automatically
• Fix hostname was not well updated in filter set_request_header or set_hostname
• Fix to check consumer’s lag instead of queue’s lag in flow_runner processor

Improvements #

• Remove newline in indexing_merge and json_indexing processor
• Improve instance check, add config to disable
• Add option skip_insecure_verify to s3 module
• Improve instance check, enable config to disable
• Update the way to get ctx process info, optimize memory usage
• Improve indexing performance for bulk_indexing processor
• Refactoring disk_queue, speedup message consumption
• Enable segment compress for disk_queue by default
• Skip download s3 files when s3 was not enabled
• Add option to log warning messages for throttle filters
• Optimize hash performance for getting primary shardID and partitionID
• Add cache for get index routing table
• Optimize performance for bulk response processing
• Refactoring bulk_processor, pass meta info to payload func
• Don’t call payload func for delete action
• Improve queue consumer’s lag check

1.8.1 #

Bug fix #

• Remove newline in document for processor es_scroll and dump_hash

1.8.0 #

Breaking changes #

• Remove config compress_on_message_payload from disk_queue
• Rename parameter consumer to name in consumer_has_lag condition
• Remove redundancy prefix name of the disk_queue files

Features #

• Add segment level based disk_queue file compression

Bug fix #

• Fix nil host in bulk_indexing processor
• Fix nil body in bulk_response_process filter
• Fix sliced consume in bulk_indexing processor

Improvements #

• Handle bulk stats to bulk_response_process and used in logging filter

1.7.0 #

Breaking changes #

Features #

• Add prometheus format to stats API
• Add redirect filter
• Add context_flow to flow filter
• Add permitted_client_ip_list to router
• Add Centos based docker image

Bug fix #

• Fix date_range_precision filter failed to parse on specify field
• Fix disk usage status in windows platform

Improvements #

• Merge events during config change, prevent unnecessary reload
• Handle templates when loading config
• Add cache to ldap_auth filter

1.6.0 #

Breaking changes #

• Update disk_queue folder structure, use UUID as folder name instead of the queue name
• Parameter mode was removed from bulk_reshuffle filter, only async was supported
• Rename filter bulk_response_validate to bulk_response_process

Features #

• Add metadata to queue
• Support subscribe queue by specify labels
• Support concurrent worker control for bulk_indexing processor
• Auto detect new queues for bulk_indexing processor
• Allow to consume queue messages over disk queue
• Auto sync disk_queue files to remote s3 in background
• Add api to operate gateway entry
• Support plugin auto discovery
• Add API to operate gateway entities
• Filter bulk_request_mutate support remove _type in bulk requests for es v8.0+
• Add elasticsearch adapter for version 8.0+
• Add http filter for general reverse proxy usage, like proxy Kibana
• Add consumer_has_lag condition to check queue status
• Add record filter to play requests easier
• Add zstd compress to disk_queue, disabled by default
• Add disorder_bulk_indexing processor
• Add javascript filter
• Add prefix and suffix to when conditions
• Add indexing_merge processor

Bug fix #

• Fix date_range_precision_tuning filter for complex range query
• Fix node availability initially check
• Fix basic_auth filter not asking user to input auth info in browser
• Fix null id not fixed in filter bulk_request_mutate and bulk_reshuffle
• Fix switch filter not forwarding when remove_prefix was disabled
• Fix buffer was not proper reset in flow_runner processor
• Fix entry not loading the pre-defined TLS certificates
• Fix set_basic_auth not proper reset previous user information
• Fix elapsed in request logging not correct
• Fix switch filter, use strings.TrimPrefix instead of strings.TrimLeft
• Fix the last query_string args can’t be deleted, parameter no_cache in get_cache filter fixed
• Fix s3 downloaded file corrupted

Improvements #

• Handle http public address, remove prefix if that exists
• Refactor bulk_reshuffle filter and bulk_indexing processor
• Should not fetch nodes info when elasticsearch discovery disabled
• Seamless consume queue message across files
• Persist consumer offset to local store
• Add API to reset consumer offset
• Refactoring ORM framework
• Expose error of mapping put
• Refactoring pipeline framework
• Improve multi-instance check, multi-instance disabled by default
• Add CPU and memory metrics to stats api
• Seamless fetch queue files from s3 server
• Proper handle 409 version conflicts in bulk requests
• Allow memory queue to retry 1s when it is full
• Proper handle the cluster available check
• Proper handle the cluster partial failure check
• Exit bulk worker while no new messages returned
• Optimize bytes buffer usage, reduced memory usage

1.5.0 #

Breaking changes #

Features #

• Add API to scroll messages from disk queue
• Prevent out of space, disk usage reserved for disk_queue
• Add context_filter and context_limiter for general purpose
• Add bulk_request_mutate filter
• Add basic_auth filter
• Add set_context filter
• Add context_regex_replace filter
• Add to_string property to request and response context

Bug fix #

• Fix bulk response validate incorrectly caused by jsonParser
• Fix nil exception in request_path_limiter caused by refactoring
• Fix big size document out of order caused by bulk buffer

Improvements #

• Fix TCP not keepalived in some case
• Add closing progress bar to pipeline module
• Add retry_delay_in_ms config to pipeline module
• Handle partial failure in bulk requests
• Optimize scroll performance of dump_hash processor
• Improve API directory

1.4.0 #

Breaking changes #

• Rename flow config filter_v2 to filter, only support new syntax
• Rename pipeline config pipelines_v2 to pipeline, processors to processor, only support new syntax
• Rename filter request_logging to logging
• Merge dump filters to dump filter
• Response headers renamed, dashboard may broken
• Remove filter request_body_truncate and response_body_truncate

Features #

• Add option to disable file logging output
• Add option compress to queue_consumer processor

Bug fix #

• Fix invalid host header setting in elasticsearch reverse proxy
• Fix cluster available health check
• Fix gzip encoding issue for requests forwarding

Improvements #

• Support string type in in condition

1.3.0 #

Breaking changes #

• Switch to use pipelines_v2 syntax only
• Rename filter disk_enqueue to queue
• Rename processor disk_queue_consumer to queue_consumer
• Rename filter redis to redis_pubsub

Features #

• Refactoring pipeline framework, support DAG based task schedule
• Add dump_hash and index_diffs processor
• Add redis output and redis queue adapter
• Add set_request_query_args filter
• Add ldap_auth filter
• Add retry_limiter filter
• Add request_body_json_set and request_body_json_del filter
• Add stats filter
• Add health_check config to elastic module
• Add API to pipeline framework, support _start and _stop pipelines

Bug fix #

• Fix data race issue in bulk_reshuffle
• Fix fix_null_id always executed in bulk_reshuffle
• Auto handle big sized documents in bulk requests

Improvements #

• Refactoring flow runner to service pipeline
• Optimize CPU and Memory usage
• Optimize index diff service, speedup and cross version compatibility
• Set the default max file size of queue files to 1 GB
• Proper handle elasticsearch failure during startup
• Support custom depth check to queue_has_lag condition
• Support multi hosts for elasticsearch configuration
• Add parameter auto_start to prevent pipeline running on start
• Add keep_running parameter to pipeline config
• Safety shutdown pipeline and entry service
• Support more complex routing pattern rules

1.2.0 #

Features #

• Support alias in bulk_reshuffle filter.
• Support truncate in request_logging filter.
• Handle 429 retry in json_indexing service.
• Add forcemerge service.
• Add response_body_regex_replace filter.
• Add request_body_regex_replace filter.
• Add sleep filter.
• Add option to log slow requests only.
• Add cluster and bulk status to request logging.
• Add filter_v2 and support _ctx to access request context.
• Add dump_context filter.
• Add translog filter, support rotation and compression.
• Add set_response filter.
• Add set_request_header filter.
• Add set_hostname filter.
• Add set_basic_auth filter.
• Add set_response_header filter.
• Add elasticsearch_health_check filter.
• Add drop filter.

Bug fix #

• Fix truncate body filter, correctly resize the body bytes.
• Fix cache filter.
• Fix floating_ip module.
• Fix dirty write in diskqueue.
• Fix compression enabled requests.
• Fix date_range_precision_tuning filter.
• Fix invalid indices status on closed indices #23.
• Fix document hash for elasticsearch 6.x.
• Fix floating_ip feature run with daemon mode.
• Fix async bulk to work with beats.

Improvements #

• Optimize memory usage, fix memory leak.

Acknowledgement #

Thanks to the following enterprises and teams #

• China Everbright Bank, China Citic Bank, BSG, Yogoo

Thanks to the following individual contributors #

• MaQianghua, YangFan, Tanzi, FangLi

1.1.0 #

• Request Logging and Dashboard.
• Support ARM Platform [armv5\v6\v7\v8(arm64)].
• Fix Elasticsearch Nodes Auto Discovery.
• Add Request Header Filter.
• Add Request Method Filter.
• Add Sample Filter.
• Request Logging Performance Optimized (100x speedup).
• Add Request Path Filter.
• Add Debug Filter.
• Add User Info to Logging Message.
• Support Routing Partial Traffic to Specify Processing Flow (by Ratio).
• Support Traffic Clone, Support Dual-Write or 1:N Write.
• Elasticsearch topology auto discovery, support filter by nodes,tags,roles.
• Backend failure auto detection, auto retry and select another available endpoint.
• Floating IP feature ready to use.
• Add bulk_reshuffle filter.

1.0.0 #

• Rewritten for performance
• Index level request throttle
• Request caching
• Kibana MAGIC speedup
• Upstream auto discovery
• Weighted upstream selections
• Max connection limit per upstream