PreEmptive Analytics Workbench User Guide

System Requirements

Minimum Requirements

The host with can be run on either physical hardware or a virtual machine, with the following minimum specifications:

  • CPU: 64-bit, 2 cores/CPUs, at 2.0 GHz
  • RAM: 3.5 GB
  • Disk: A drive with 85+ GB free
  • OS: 64-bit version of one of the following:
    • Windows Server 2012 R2 or 2016 (recommended)
    • Windows Server 2008 R2 or 2012
    • Windows 10, 8, 8.1, or 7

The following Windows features must also be installed:

  1. Internet Information Services (IIS): 7.5, 8.0, or 10.0
    • Static Content
    • IIS Management Console
    • If you plan to configure SSL, you may also want to install the URL Rewrite module.
  2. .NET Framework 4.5
    • ASP .NET 4.5

The following applications must be installed prior to the Workbench:

After installing, to access the Portal, use one of:

  • Chrome
  • Firefox
  • Internet Explorer 9, or later
    • with Compatibility Mode turned off
    • with Enhanced Security Configuration turned off (Server OSes)
  • Safari 5, or later
  • Mobile Safari (iOS 5+)
  • Mobile Chrome (Android 4.0+)
  • Mobile IE (Windows Phone 8+)

JavaScript must be enabled.

Note that additional configuration is required to allow Portal connections from other hosts.

Azure Requirements

If you are installing the Workbench on an Azure virtual machine, you should:

  • Use an A2 (or better) VM size, in the Standard tier. The Basic tier will work, but Basic instances have reduced disk performance compared to Standard.
  • Attach a second disk, with read/write caching, to use for MongoDB storage (only). The read/write cache makes a significant difference in disk performance for the Workbench.
    • Be sure not to use the default D: drive for storage; it is a temporary drive that will be erased each time the instance reboots. When you attach a new drive via Azure, you must also mount it via Disk Management in the OS and give it a new drive letter (e.g. F:).
  • Review the latest MongoDB recommendations about Azure deployments.

Scalability Planning

The minimum configuration above is sufficient for most test and moderate-volume instances but may not be sufficient for high data volumes, many distinct applications (or application versions), many additional custom indexers, multiple additional custom patterns, or large amounts of custom data. In such cases, please review the guidelines below, or contact support so we can review your needs and offer tailored guidance.

Data Hub

To begin estimating your resource needs, first review the System Requirements section of the Data Hub User Guide and consider how they affect the Workbench host.

For the normal "Hub + Workbench" host, the Hub just provides a temporary queue during Workbench upgrades or configuration changes - so data storage isn't usually much of an issue. It also provides an error queue for any messages that the Workbench can't parse, but that error queue is time-limited so it generally will not get large. Message rates into a single Workbench are also generally much lower than the Hub upper limits, so CPU and memory requirements are generally very modest.

In most normal scenarios, the Hub just needs ~10GB of free disk space, and ~1GB of free memory.

Workbench

CPU

There are two primary consumers of CPU in the Workbench: the Computation Service and the Query Web Service (plus MongoDB). Please refer to the architecture diagram if needed. We assume that the Portal will be configured for remote access; if not, the browser will also consume CPU when viewing the Portal locally.

With the minimum requirements described above, the Computation Service can consume approximately 10 envelopes per second. This rate is not typically CPU-bound, so additional CPU probably won't improve the rate significantly.

Query load depends on the amount of data in the database, and on the number of concurrent Workbench users. The minimum requirements above can provide acceptable performance (below-5-second report load-time) for up to about 1 million app runs, for a single Portal user. This rate is usually CPU-bound, so additional CPU will provide additional scalability. Individual queries are mostly single-threaded, but some reports request multiple queries concurrently, so higher clock speeds and more cores will help. More cores will also help improve responsiveness with concurrent users.

Memory

Memory use is generally near-static, even under load. There are 4 primary consumers of memory:

  • The Computation Service
  • The Query Web Service
  • RabbitMQ
  • MongoDB

The Computation Service will typically only use about 200MB of memory, with the default indexers. Very large messages can increase this amount.

The Query Web Service will typically use about 200MB per concurrent user, although it can use more (up to about 600MB, observed) with e.g. direct API access and a very large query.

RabbitMQ will limit itself to 25% of available system memory, but will generally use much less than that (a few hundred MB).

MongoDB will typically only consume a few hundred MB as "private bytes" but will often consume all available memory (over time) as "shared memory". For scalability purposes, the "private bytes" value is the one that must be supported. More available memory is always better for MongoDB, but 512MB is sufficient.

In total, ~1GB of available memory is sufficient for most installations, plus ~200MB per concurrent user of the Portal.

Storage

First, we recommend putting the MongoDB data and log folders on a secondary, dedicated disk. This limits disk contention, eliminates any risk of the OS disk filling up, and makes it easier to expand the available disk space if needed. Additionally, as the MongoDB Production Notes recommend setting the Windows page file to the largest feasible multiple of 32 GB, keeping the database on a separate disk permits a larger page file on the OS disk.

Second, remember that the Workbench has two types of storage: temporary and permanent (both in the same MongoDB instance). These two storage areas have growth rates that are independent of each other. Note that for the first few years of Workbench use, temporary-storage requirements typically exceed permanent storage requirements.

Third, for most use-cases for a single instance of the Workbench, 50GB of free disk is sufficient for multiple years of history.

Finally, calculating precise storage requirements is very complex and depends on exactly how your Workbench is configured. In general, these are the primary drivers of storage use (in order of significance):

  • The incoming message rate has a roughly linear effect on both temporary storage and permanent storage.
  • The Workbench will consume up to a maximum of about 6 GiB in "overhead" over the course of the first few months of use. This amount will scale roughly with message rate.
  • The temp-bin cleanup setting (default 30) has a roughly-linear effect on temporary storage.
  • The number of installed patterns (default 1) affects permanent storage exponentially. With one or two patterns it might be a small portion of the permanent storage requirements, but with e.g. five patterns, it can dominate the permanent storage requirements.
  • The overall number of indexers (default 7) has a roughly-linear effect on temporary and permanent storage use.
  • The number of distinct applications/versions that are concurrently (within each hour of each day) reporting data has a roughly-linear effect on permanent storage use. A distinct app-version is defined as a unique tuple of (Company ID + Company Name + App ID + App Name + Version #).

Here are some examples to help you gauge real-world scenarios:

Scenario Single app Mid-volume, customized High-volume, heavily customized
Average incoming messages per second 0.1 1 10
# of pattern indexers 1 2 3
Total # of indexers (incl. pattern) 9 10 15
Distinct application-versions 5 40 100
Storage after 1 year 7.5 GiB 23.8 GiB 234 GiB
Storage after 2 years 7.8 GiB 29.0 GiB 273 GiB
Storage after 3 years 8.1 GiB 34.2 GiB 312 GiB

Note: Please see the large / excessive data section for additional considerations.

Incoming message rate

The incoming message rate is a function of:

  • How much instrumentation you include in your app(s) – sessions, features (this can vary widely), exceptions, etc. – and how often those things happen.
  • How many instances of your instrumented apps are running at any given time (i.e. sending messages).

The net result of these two could be anything from "a few messages a day" to "30 messages a second" without stretching the limits of "normal".

To give you a sense of our experience: we rarely see a single production app exceeding 1 message/second. (Most apps are much slower.) But we have some that continually deliver 4-8 per second, and one that sends floods of thousands of messages in a few seconds, every few hours.

Note that if you observe the data transmitted over the network, individual messages are usually transmitted as part of larger message batches.

Bandwidth

Bandwidth is used in two primary ways: incoming data, and outgoing queries (usually to the Portal).

For incoming data bandwidth, you'll need to estimate your incoming data rate and then please see the discussion below about incoming message size. In most cases this is extremely low - less than a few messages per second, at a few kilobytes each.

For outgoing (Portal) bandwidth, the rate can vary widely. The key idea is that some data processing is done in the browser - which means that the browser has to download raw data in order to do that processing. The amount of data depends on the amount of data in the Workbench, the time period being queried, the particular report in use, and the number of concurrent users.

To put it in concrete terms, the Overview page, with the default 30-day window, typically uses a few hundred kilobytes of bandwidth. The Version Adoption report, for the same time window, uses an amount of data that varies with the number of distinct application records in the Workbench, but typically 1-2MB. At the far extreme, the Custom Data report on a Workbench with a large amount of custom data, may transfer tens or hundreds of megabytes of data. (Please see large / excessive data for mitigation strategies.)

Incoming message size

Most standard messages (e.g. session-start, feature-tick, exception, etc.) are rather small - 1 or 2 kilobytes. But it’s possible to make them much larger, as described below in large / excessive data, so you should consider your instrumentation approach before estimating bandwidth use.

It's also important to note that all the PreEmptive Analytics instrumentation clients/APIs automatically bundle individual messages into "batches" (also called "envelopes") which is what actually goes over the network. On average, under normal instrumentation scenarios, most envelopes have 3-4 messages in them.

To give you a sense of our experience: the average envelope size across all data coming to us from customers is just under 4KB. About 50% of envelopes are ~3KB.

Large / excessive data

The advice given above is based on some assumptions / averages about message size. Specifically, they assume "typical" patterns of data generation. Atypical patterns, especially around custom data or extremely-high volumes of a single message type, can have a negative effect on the performance and scalability of the Workbench.

To explain: an instrumented application normally sends a variety of pre-defined message types, like app-starts, feature ticks, exceptions, etc. These message-types are typically delivered in relatively-standard distributions - more features than sessions, fewer exceptions than sessions, etc. They also typically have a moderate amount of custom data (e.g. a few fields per feature). Instrumentation data that fits this profile will fit easily within the resource requirements outlined above. But it is possible for applications to send data that greatly exceeds these norms - in which case the information above may not be sufficient.

There are a few specific ways that this might happen:

Excessive custom data

An application can send an unusually-large amount of custom data. Most applications send some custom data - perhaps a few fields per feature, with values that are a few-dozen bytes. But it is possible for an application to send an arbitrary number of fields, each with a value that is kilobytes long - in which case the storage calculations above are insufficient, and the default Custom Data Server Query will take an especially long time. Most other aspects of the system will be fine.

If you plan to make extensive use of custom data, we recommend removing the default custom data indexers and creating purpose-specific custom data filters instead. This will use much less storage, will greatly improve query performance, and will probably provide much more value from using the Portal.

Extremely large exception messages

An application can send unusually-large exception messages. Some applications generate stack traces with multi-kilobyte messages, or thousand-line call stacks. This will increase the storage requirements and will slow the Exceptions report.

An extreme amount of any specific message type

If an application sends an extremely-unusual amount (e.g. 1000/session) of any particular message type - e.g. features or exceptions - it can affect performance and storage requirements. Only rarely have we seen this happen, and typically it turns out to be an unintended consequence rather than something intentional.

If you believe that you do need to generate data with such extreme ratios, please contact us - we can often suggest better ways to instrument that will reduce load on the app and on the Workbench, and will make the data easier to analyze.



Workbench Version 1.2.0. Copyright © 2016 PreEmptive Solutions, LLC