Configuration
Overview
Each Redis service is configured with the below basic structure:
{
// Common service configuration
"maximum-outstanding": <maximum outstanding messages>,
"connection": {
// Redis database connection configuration
},
"endpoints": [
// One endpoint per queue (both read and write)
]
}
Common service configuration items are available for all JSLEE services. The configuration items specific to the Redis service are listed below.
Field | Type | Required? | Default | Description |
---|---|---|---|---|
maximum-outstanding |
Integer | No | 50 |
The total amount of outstanding messages allowed to processing services. New messages will not be requested from the Redis server for any queue if this limit is reached. |
connection |
Object | Yes | - | The Redis database connection configuration. |
endpoints |
Array | No | (empty) | The list of message queues to manage on the Redis server. |
Connection Configuration
The following connection configuration options are available for Redis connectivity.
Field | Type | Required? | Default | Description |
---|---|---|---|---|
maxWaitingHandlers |
Integer | No | 2048 |
The maximum allowable number of outstanding requests to Redis. This determines the outstanding queue of responses the client may be waiting for, per service instance. |
maxNestedArrays |
Integer | No | 32 |
The maximum number of nested arrays that are returned from Redis. Note that the number of nested arrays in results affects parsing performance; if nested arrays are not used, this value can be low. |
select |
Integer | No | null |
The ID of the database to connect to when connecting to the Redis server. |
type |
String | No | "STANDALONE" |
The type of connection to use, either CLUSTER , SENTINEL , or STANDALONE . This should be set to align with your Redis configuration. Note that a Redis Enterprise cluster connection should use STANDALONE . |
useSlave |
String | No | "NEVER" |
Whether to use Redis slaves when running on master nodes. This may be ALWAYS to exclusively use slaves, NEVER to exclusively use masters, or SHARE to use both. |
tcp |
Object | No | (various) | Common TCP/IP connectivity options to use for all TCP/IP connections made to the Redis server. |
password |
String | No | null |
The password to use to log in to the Redis server, if required. If not specified, then no password is used to authenticate. |
endpoint |
String | No | redis://127.0.0.1:6379 |
The Redis server to connect to, as a colon-separated TCP/IP host and port. The default value is only used if endpoints is not provided. |
endpoints |
Array | No | null |
A list of endpoints to connect to first, to reach a clustered or sentinel Redis instance. Each endpoint must be in the same format as given in endpoint . |
password |
String | No | - | The password to use for the Redis server, if required. |
Cluster vs Standalone Connections
The type
configuration for the Redis connection determines how the JSLEE manages the Redis connection.
The connection type should be set to:
-
STANDALONE
should be used when the JSLEE should connect to a single Redis server only. This covers normal unclustered Redis connections, and also Redis Enterprise clusters which provide a single endpoint to manage the Redis connection (effectively hiding the cluster configuration behind a proxy). -
CLUSTER
should be used only when the JSLEE is to connect to multiple Redis servers which are themselves clustered. This option should be used only when an Open Source Redis cluster is being used.
Endpoints List
When connecting to a Redis cluster, the cluster’s hostname/IP and port can be given using either
endpoint
or endpoints
. The JSLEE Redis cluster connection manager will automatically connect
to the endpoints configured, and then use the CLUSTER NODES
Redis command to identify any additional
nodes.
The Redis service uses the CLUSTER SLOTS
Redis command to determine which redis nodes hold which
hash slots.
This allows the configuration to maintain a subset of node details for the cluster, rather than require the configuration to define all nodes explicitly.
The Redis cluster connections will be automatically maintained where possible, however the JSLEE will not automatically pick up new nodes. To enable new nodes, send a configuration reload to the Redis service via the JSLEE console.
Endpoints Configuration
The endpoints
list is a list of logical message queues for the Redis service to interact with.
Each endpoint queue is configured with a queue name and processing configuration:
{
"name": "myqueue",
"enabled": true,
"source-list": "<source-list-x>",
"sink-list": "<processed-list-x>",
"error-list": "<error-list-x>",
"retry-list": "<retry-list-x>",
"processing-address": "<jslee-service>",
"processing-list": "<processing-list-x>",
"expect-response-class-codes": true
}
If the source-list
parameter is defined, the processing-address
parameter creates a
dependency for the destination value - the endpoint will not
poll for list data unless at least one instance of this address is available.
The following configuration options are available for each configured queue:
Field | Type | Required? | Default | Description |
---|---|---|---|---|
name |
String | No | (source/sink name) | The endpoint name that can be referred to by other JSLEE services. |
enabled |
Boolean | No | true |
Whether this queue connection to Redis should be enabled or not. Takes effect at service startup and after config reload, but can be overridden by the Endpoint command through the JSLEE console. |
source-list |
String | No | - | The name of the Redis queue for the endpoint to read from and write to. The queue need not exist prior to use. If using a Redis cluster, this must be aligned with the processing-list queue name. |
processing-list |
String | Conditional | - | The name of the Redis queue for the endpoint to use during processing. Redis executes BRPOPLPUSH <source-list> <processing-list> . Required ifsource-list is provided. If using a Redis cluster, this must be aligned with the source-list queue name. |
processing-address |
String | Conditional | - | The address to use when sending all messages read from this queue onward. Required ifsource-list is provided. |
processing-key |
String | No | - | An RFC 6901 document path to be appended to EDR slee-session-id values. If not specified or if the value is unable to be read from this path for any reason, no appending will be done. Note that this requires that the information read from Redis is JSON. |
sink-list |
String | Conditional | - | The name of the Redis list to transfer successfully processed messages to. Must be configured if delete-after-success is false (the default). |
error-list |
Strung | Conditional | - | The name of the Redis list to transfer messages to when the processing endpoint responds with an error. Must be configured if delete-on-error is false (the default). |
retry-list |
String | No | - | The name of the Redis list to transfer messages to when a transient error of some form is received. This includes timeouts attempting to process the message, failures sending messages to the processing address (e.g. because it is disabled), and (if enabled using expect-response-class-codes ) error codes received from the processing address that indicate a transient failure. If a transient failure code is received but the retry-list is not listed, the error-list is used instead. |
delete-after-success |
Boolean | No | false |
If true, delete a successfully transferred message, rather than transferring to sink-list . |
delete-after-error |
Boolean | No | false |
If true, delete the message from processing-list when an error is received from the processing-endpoint (either due to delivery or processing error), rather than transferring to error-list . |
expect-response-class-codes |
Boolean | No | true |
The Redis queue can expect ResponseClass error codes from the processing address, allowing the service to distinguish between permanent and temporary errors. The processing service must consistently reply with ResponseClass codes for this to be enabled. |
Redis Cluster Key Alignment
When using a source-list
pushing to a processing-list
in a Redis cluster, the specified values must include a
Redis hash tag signifier, defined within
curly braces. This value must be the same between the two keys, as the BRPOPLPUSH
operation used by Redis contains
two actions but is atomic and must therefore operate on the same cluster host.
A sample configuration might be:
"endpoints": [
{
"name": "mno1",
"source-list": "{mno1}",
"error-list": "mno1-error",
"delete-after-success": true,
"processing-address": "smsgw",
"processing-list": "{mno1}-processing"
}
This ensures that the string mno1
is used as the lookup value for key insertion for both lists, allowing the atomic
operation to succeed on a single host.
This is not required for the error-list
, sink-list
, or retry-list
keys, as these are only used by single-action
operations.
Temporary vs. Permanent Errors
The Redis queue management is sensitive to the form of each failure that occurs when it passes a request message to the
processing-address
. The following response handling logic is employed by the Redis service:
- Any successful response from the processing service is considered a success, and the message is either removed or
added to the
sink-list
if defined. - In cases where a timeout occurs, or there is a failure to reach the service defined by the
processing-address
, the message will be transferred to theretry-list
if one is defined, otherwise the message will be transferred to theerror-list
(or deleted). - If
expect-response-class-codes
is false, all other error responses are considered permanent errors, and the message will be transferred to theerror-list
(or deleted). - If
expect-response-class-codes
is true, and is not a timeout or service connection error, the error received will be processed:- If the error received can be considered a
ResponseClass
error code, the code is analysed. - If the error code is a transient (temporary) error code, the message will be transferred to the
retry-list
, if one is defined, otherwise it will be transferred to theerror-list
(or deleted). - If the error code is a permanent error code, the message will be transferred to the
error-list
(or deleted). - If the error code is missing, or cannot be processed as a
ResponseCode
error code, a log message is printed, and the message is transferred to theerror-list
(or deleted).
- If the error received can be considered a
EDRs
EDRs are generated for messages received and sent using the reliable queuing mechanism, but not for key/value storage.
The JSLEE EDR subsystem is used for this purpose and EDRs will be generated if the service configuration includes an
appropriate edr
subsection.
Refer to the Redis service EDR section for information on the fields generated in Redis EDRs.
Statistics
The Redis verticle publishes statistics using built in JSLEE statistics publishing system. Refer to the Redis Statistics for a full list of statistics generated by the Redis service.
Connection Encryption
The JSLEE Redis connection does not support TLS, SSL or other on-the-wire encryption methods natively. For further information on encrypting traffic between the JSLEE and Redis, please review the documentation provided by Redis on the Redis security page.
As noted by this documentation, spiped is recommended for on-the-wire encryption.
For connection security, the Redis server password can be set in the JSLEE password
field in the connection
section of the
service configuration:
{
"connection": {
"password": "mypassword",
// ...
},
}