PreEmptive Analytics Workbench User Guide

Upgrading to Workbench 1.2

This section provides instructions for upgrading Workbench from 1.1.x to 1.2.x. Please read it thoroughly before starting your upgrade process.

Upgrade Compatability

Workbench 1.2 only supports upgrading from Workbench 1.1. If you wish to upgrade from Workbench 1.0, you must first upgrade to Workbench 1.1.

Upgrading the Workbench requires upgrading MongoDB to version 3.2. The instructions below will guide you through this process, including if a Standalone Repository is sharing the MongoDB instance.

Upgrade Duration

The upgrade will typically take up to an hour to complete. This is because there were no major schema changes in the database in this version, and MongoDB 2.6's data format is compatible with MongoDB 3.2.

However, there will still be an outage for the Workbench and any Standalone Repository on the host during the upgrade. Additionally, no Replays from the Repository can occur during the upgrade. Please schedule the upgrade accordingly.

When to Replay instead of Upgrading

There are two main options for migrating from Workbench 1.1 to 1.2:

  1. Upgrading Workbench 1.1 by following the Upgrade Steps in this section.
  2. Replaying from a Standalone Repository to a fresh install of Workbench 1.2.

In most cases, Upgrading (option 1) is preferred over replaying (option 2), because it is much faster. However, there is a known issue that may cause some Workbench administrators to consider the latter option. For details, see "Upgraded Exception Data has Inconsistent Stack Traces with Newly-Received Exception Data".

Upgrade Steps

To upgrade from Workbench 1.1, follow the steps listed below.

Note: We recommend disabling anything that may cause an unexpected reboot during the upgrade, such as Windows Update.

Queue data upstream

Assuming that there is a Data Hub "upstream" from the Workbench (typically the same one required to be installed on the same host), the first step is to have that Data Hub queue any data that arrives. This ensures that the Workbench can be safely offline, and database operations can be safely performed, without data arriving at an unexpected time.

To queue incoming data:

  1. Open a command prompt as an administrator.

  2. Run netstat -a -n.

  3. In the output, look for rows whose State is LISTENING and check the port number listed in the Local Address column. These are the ports that are currently in use.

  4. As an administrator, launch a text editor and open the Hub Dispatch Configuration (e.g., C:\Program Files (x86)\PreEmptive Solutions\PreEmptive Analytics Data Hub\DispatchService\Dispatch.config).

  5. For the Workbench destination entry, change the port number in the url property to a port that is NOT in the output from netstat (step 3).

    • For instance, a typical url for a Workbench destination would be http://127.0.0.1:81/endpoint. You might change this to http://127.0.0.1:881/endpoint, assuming port 881 is not in use.
  6. Repeat the previous step for the Standalone Repository destination entry, if installed.

  7. Save the file.

  8. Open the Services control panel (Run > services.msc).

  9. Restart the PreEmptive Analytics Data Hub Dispatch Service.

You may wish to watch the RabbitMQ queues (http://localhost:15672/#/queues, login: guest/guest) to see that data is being queued in the appropriate offline queue(s), before proceeding.

Take a backup

Please ensure you have a recent backup before continuing the rest of the upgrade process.

If you wish to have a "just before upgrade" backup, this is the time to do it.

Note IIS Customizations

The following IIS web sites will be removed and re-installed during the upgrade:

  • PreEmptive Analytics Standalone Repository (if a Standalone Repository is also installed on this host)
  • PreEmptive Analytics Workbench Endpoint
  • PreEmptive Analytics Workbench Portal

Any changes made to these web sites since their installation will be lost during the upgrade. Changes that will be removed include:

  • Changed bindings (i.e., open hostnames and ports),
  • Added rewrite rules, and
  • Changes made to the IIS sites by following the instructions in the Remote Access Security section.

Please take note of any of these customizations now. You will need to re-apply them in a later step.

Remove Custom Workbench Plugins

Plugins built for prior versions of the Workbench will need to be recompiled due to the assemblies PreEmptive.Components.Common.dll and PreEmptive.Messaging.Schema.dll no longer being strong-named. Installing the Workbench will provide the new versions of those assemblies, so for now, move the Plugins out of the Workbench's configuration.

If you have made changes to the Plugins, follow these steps:

  1. Navigate to the Plugins folder (e.g., C:\Program Files (x86)\PreEmptive Solutions\PreEmptive Analytics Workbench\config\Plugins).
  2. Within the preemptive subfolder and its subfolders:
    • Note which Plugins are present. If you have previously removed one of these Plugins, you will need to re-remove it after the upgrade.
    • Copy any added files, such additional .dat files in the Geoloc subfolder, to a safe location outside of the Workbench's installation folder.
  3. Note which non-preemptive Plugins are present in the Plugins folder and its subfolders.
  4. Ensure that for every such Plugin, you have the appropriate source code available in a development environment. You will need this to recompile the same Plugins after the upgrade.
  5. Remove all non-preemptive Plugins.

Uninstall Workbench 1.1 (preserving data)

Before the database migration occurs, Workbench 1.1 must be uninstalled with the "preserve data" option selected.

  1. Open the Programs and Features control panel.
  2. Select "PreEmptive Analytics Workbench".
  3. Click "Uninstall/Change".
  4. In the dialog that appears, ensure the "Preserve databases and configuration for future upgrades" option is checked.
  5. Click "Remove".
  6. Confirm the uninstall by clicking "Remove" again.
    • The dialog may state that a reboot is needed after the uninstall. A later step will reboot the host; please wait until this step to reboot.
  7. After the software uninstalls, click "Close".

Uninstall Standalone Repository 1.1 (preserving data)

If a Standalone Repository uses the same MongoDB instance as the Workbench that is being upgraded, that Standalone Repository instance must also be uninstalled before the MongoDB upgrade.

  1. Open the Programs and Features control panel.
  2. Select "PreEmptive Analytics Standalone Repository".
  3. Click "Uninstall/Change".
  4. In the dialog that appears, click "Next".
  5. Ensure the "Preserve databases and configuration for future upgrades" option is checked.
  6. Click "Remove".
  7. Confirm the uninstall by clicking "Remove" again.
    • The dialog may state that a reboot is needed after the uninstall. A later step will reboot the host; please wait until this step to reboot.
  8. After the software uninstalls, click "Close".

Uninstall Replayer 1.1

If the Replayer is also installed on this host, we recommend uninstalling it along with the other components:

  1. If you have made any changes to .config files (e.g., bin\Web.config and bin\NLog.config) in the Replayer install folder (e.g., C:\Program Files (x86)\PreEmptive Solutions\PreEmptive Analytics Replayer), then you will need to save those files elsewhere during the uninstall/reinstall process and apply them later.
  2. Open the Programs and Features control panel.
  3. Select "PreEmptive Analytics Replayer".
  4. Click "Uninstall/Change".
  5. Follow the prompts to uninstall the Replayer.
    • The dialog may state that a reboot is needed after the uninstall. A later step will reboot the host; please wait until this step to reboot.

Set Windows page file size

Per the MongoDB Production Notes, we recommend setting the Windows page file to the largest feasible multiple of 32 GB, if not already done so.

Reboot the Host

A reboot may now be necessary due to an uninstall and/or page file change. This is the best time for that reboot to take place.

While the reboot occurs, the Workbench host's Data Hub will be unavailable. In a typical installation, there will be an independent host with a Data Hub upstream from the Workbench host's Data Hub. This upstream Data Hub will queue Analytics envelopes during the reboot, and start transmitting them once the Workbench host is back online. Thus, no incoming data will be lost during this outage.

Stop MongoDB

Now that no applications are using the MongoDB instance, stop the MongoDB service.

  1. Open the Services control panel (Run > services.msc).
  2. Stop the MongoDB service.
    • Note: You may see an error "The pipe has been ended". This is a known issue in MongoDB 2.6; the service has successfully stopped in this case.

Update the MongoDB Configuration

In MongoDB 2.6, there was only one storage engine available: MMAPv1. MongoDB 3.0 introduced a new storage engine, WiredTiger, and MongoDB 3.2 made WiredTiger the default storage engine. However, PreEmptive Analytics components only support the MMAPv1 storage engine. Thus, the mongod.cfg configuration must be updated to ensure MMAPv1 is still used once the MongoDB instance has been upgraded.

  1. Open the C:\mongodb\mongod.cfg file in a text editor.
  2. If this file has options specified in the format dbpath=c:\data\db, it is using a deprecated format. You will need to change this to the YAML format introduced in newer versions of MongoDB. Consult those links and the current MongoDB installation instructions for reference.
  3. Under the storage key, add a subkey engine with the value mmapv1.
  4. Add a net key, and add a subkey bindIp with the value 127.0.0.1.
  5. Save the configuration file.

Your configuration should look like this:

storage:
   dbPath: P:\data\db
   engine: mmapv1
systemLog:
   destination: file
   path: C:\mongodb\log\mongo.log
operationProfiling:
   slowOpThresholdMs: 1000
net:
   bindIp: 127.0.0.1

Important Note: The YAML format of the configuration above does not allow Tab characters. Use spaces instead.

Replace MongoDB Binaries

Once the configuration is updated, MongoDB's binaries can be upgraded from 2.6 to 3.2.

  1. Download MongoDB 3.2.x (x64) from https://www.mongodb.org/downloads.
    • Select "Community Server".
    • From the Version drop-down, select "Windows Server 2008 RS 64-bit and later, without SSL support x64".
    • Click the "All Version Binaries" link.
    • Search for and download the ZIP archive win32/mongodb-win32-x86_64-2008plus-3.2.10.zip or that of a later version.
  2. Extract the contents. You should have a folder named something similar to mongodb-win32-x86_64-2008plus-3.2.10 in your Downloads folder.
  3. Open the extracted folder.
  4. Copy the bin folder.
  5. Navigate to the location of the MongoDB 2.6 install (e.g., C:\mongodb) and paste the copied bin folder.
  6. Choose "Replace the files in the destination" in the dialog that appears.

Start MongoDB

Now MongoDB 3.2 can be started.

  1. Open the Services control panel (Run > services.msc).
  2. Start the MongoDB service.

Provided the instructions above have been followed, the MongoDB service will use the existing PreEmptive Analytics databases.

Install Standalone Repository 1.2

If you previously uninstalled Standalone Repository 1.1, now is the time to install Standalone Repository 1.2. The new version of the Repository will contain the same data as the prior version.

  1. Copy the Standalone Repository 1.2 distribution to the host.
  2. Execute PreEmptive.Analytics.Standalone.Repository.exe and follow the prompts in the installer.
  3. As part of the installer, the database version will be upgraded.
  4. Once the upgrade is complete, click "Close" in the Upgrade Tool to finish the installation.
  5. Click "Close" on the final page of the installation.

Install Replayer 1.2

If you previously uninstalled Replayer 1.1, you can install Replayer 1.2 at this time.

  1. Copy the Replayer 1.2 distribution to the host.
  2. Execute PreEmptive.Analytics.Standalone.Replayer.exe and follow the prompts in the installer.
  3. Click "Close" on the final page of the installation.
  4. If you saved any configuration when uninstalling Replayer 1.1, copy or re-apply those settings to the newly-installed configuration files as needed.

Install Workbench 1.2 (upgrading data)

After MongoDB has been upgraded, Workbench 1.2 can be installed.

  1. Copy the Workbench 1.2 distribution to the host.
  2. Execute Workbench.exe and follow the prompts in the installer.
  3. As part of the installer, the database version will be upgraded.
  4. Once the upgrade is complete, click "Close" in the Upgrade Tool to finish the installation.
  5. Click "Close" on the final page of the installation.

Re-apply IIS Customizations

At this point, all IIS websites have been re-installed. Now is the time to re-apply any IIS customizations previously noted.

In particular, if you had previously followed the instructions in the Remote Access Security section, you will need to perform those steps again.

Recompile and Install Custom Workbench Plugins

If you previously removed Custom Workbench Plugins, now is the time to recompile and add them back.

For each non-preemptive Plugin:

  1. Copy any applicable DLLs from the Computation Service folder (e.g., C:\Program Files (x86)\PreEmptive Solutions\PreEmptive Analytics Workbench\Windows\Computation Service) to the necessary location in the development environment for the Plugin.
  2. Build and deploy the Plugin per the normal procedure.

You will also need to apply the relevant changes to the preemptive Plugin folder:

  1. Remove each PreEmptive Plugin that was removed in the previous version of the Workbench.
  2. Copy back any files that were added to the PreEmptive Plugin folder in the previous version of the Workbench, such as .dat files in the Geoloc folder.

Verify the Workbench Installation

The Workbench Portal should now be available. By default, it can be accessed on the Workbench host at http://127.0.0.1:88. The Portal should display the data as it was displayed in the previous installation, though the version number in the page's footer will be updated.

With the Workbench fully operational, you should verify incoming data is being processed correctly. This can be done by following the steps on the Testing the Installation page. Note that you will probably want to put the test client or webpage on the Workbench host, and transmit to the Workbench endpoint directly. This will bypass the Data Hub, which is currently configured to queue incoming data rather than dispatch it.

Reconfigure and release the upstream queue(s)

At this point the Workbench is installed and ready to resume receiving data. To allow data to come through, you must undo what you did above.

To allow queued data to be transmitted again:

  1. As an administrator, launch a text editor and open the Hub Dispatch Configuration (e.g., C:\Program Files (x86)\PreEmptive Solutions\PreEmptive Analytics Data Hub\DispatchService\Dispatch.config).

  2. For the Workbench destination entry, change the value of the url property to the URL of the Workbench's endpoint.

    • The default Workbench endpoint URL is http://127.0.0.1:81/endpoint.
  3. Repeat the previous step for the Standalone Repository destination entry, if installed.

    • The default Standalone Repository endpoint URL is http://127.0.0.1:99/endpoint.
  4. Save the file.

  5. Open the Services control panel (Run > services.msc).

  6. Restart the PreEmptive Analytics Data Hub Dispatch Service.

The offline RabbitMQ queue(s) will begin emptying about 60 seconds after the Dispatch Service is restarted. You may wish to watch the queue(s) empty via the RabbitMQ Management Portal (http://localhost:15672/#/queues, login: guest/guest).

As data is ingested by the Workbench, you should see updates to the data in the Workbench Portal, located by default at http://127.0.0.1:88.

As data is ingested by the Standalone Repository, run the Replayer command report multiple times. The reported number of envelopes should increase between runs, as the Repository continues to accept envelopes.

Upgrade Complete

Now the upgrade is complete. Be sure to re-enable anything you disabled before the upgrade, such as Windows Update.



Workbench Version 1.2.0. Copyright © 2016 PreEmptive Solutions, LLC