PreEmptive Analytics Workbench User Guide

Error Queue Processing

As explained in the overview, the Computation Service is responsible for processing incoming messages and storing aggregated results in MongoDB. This Windows Service performs this task automatically, based on messages stored by the Endpoint Web Service into the RabbitMQ queue, workbench-endpoint.

Because the Computation Service has a high degree of customization, and because it does not receive requests from an external entity (like the Endpoint and Interaction Web Services do), it provides a mechanism to tolerate errors that occur during processing, including those caused by misconfigured customization. In addition to logging the error and reporting the failure to WMI counters, envelopes in the errored batch are queued in the error queue, workbench-error.

Behavior

Envelopes in the error queue are reprocessed by the Computation Service:

  • Automatically, every 6 hours the service is running (configurable)
  • Manually, by use of the Administration Tool's process command

When triggered, this error queue processing re-attempts every envelope stored in the error queue once, removing it if the envelope succeeds, and re-queuing it back in the error queue if it fails again.

Unlike processing from the endpoint queue, which operates in batches of envelopes, error queue processing operates on only 1 envelope at a time. Some errors occur only after an entire batch is processed, so by limiting the "batch" size to 1, it ensures an envelope's issue does not block processing of other envelopes.

To limit the error queue's size, envelopes will be discarded after they have been in the queue for at least 30 days (configurable).

Configuration

Aspects of this error retry behavior can be changed through the Computation Service's configuration file, located at <Installation Folder>\Windows\Computation Service\WorkbenchComputationService.exe.config.

Note: the Computation Service must be restarted for these changes to take effect.

<add key="ErrorComputeDelay" value="06:00:00"/>
<add key="ErrorGiveup" value="30.00:00:00"/>

Where:

  • ErrorComputeDelay is the amount of time between automatic triggering of error queue processing. Defaults to 06:00:00 (6 hours).
  • ErrorGiveup is the amount of time that an envelope must be in the error queue before being considered for discarding. Defaults to 30.00:00:00 (30 days).


Workbench Version 1.2.0. Copyright © 2016 PreEmptive Solutions, LLC