Introduction
This is the second article in my ongoing series on how to use the new, free features available in Dotfuscator Software Services Community Edition in Visual Studio 2010. In addition to obfuscation, it allows you to track how users are using your applications; and, it can inject additional functionality such as tamper detection and application expiration. For an introduction to Runtime Intelligence and an overview of the new features included in Dotfuscator CE, see the introductory article in this series What Is Runtime Intelligence . I also recommend that you read the What’s New with Dotfuscator in Visual Studio 2010 Beta 1 post, a walkthrough that demonstrates some of the same new Dotfuscator CE features I will be using here.
For this example we will be using the Witty Twitter application as our sample application. This is an open source WPF Twitter client and is available here .
Our current problem as the developer of a Windows client application is once the end user has downloaded our software we have no way of knowing if they actually installed it, much less used it. There are a number of ways that I have seen people try to track the ratio of installations to downloads, including opening a special web page as part of the installation routine and using web analytics data to infer installations. There is a similar issue with usage. You don’t know if anyone started the application, unless it crashes spectacularly, and then you only hear from the three irate users and not the millions of satisfied users.
By using the new Runtime Intelligence capabilities in Dotfuscator CE you can quickly and easily determine how many people actually install and run your application. You can gather additional details such as which operating systems and CLR versions they are using. Since Dotfuscator CE can do this through post compilation code injection you don’t need to modify your source code to add these new capabilities to your application.
To accomplish all of this I will be using the publicly available Visual Studio 2010 Beta 1. As long as you choose to install Dotfuscator and the Windows SDK you will have all of the tools you will need.
Implementation Without Source Code Modification
This article shows how you can add application analytics to an existing application without modifying the source code. The first step is to download the latest Witty Twitter installer and install it.
Next, copy all of the files from the Witty installation directory to a temporary location to use as a working directory. This is necessary because the Witty executable has dependencies on other assemblies that are installed with it and Dotfuscator will look in the input directory by default to resolve dependencies. This also allows us to compare our final performance and functionality to the original.
Now that we have the application files in a new working folder, start Visual Studio 2010. From the Visual Studio Tools menu select Dotfuscator Software Services. When Dotfuscator starts up, accept the license agreement and then you will be in Dotfuscator CE’s new WPF user interface.
Now, add the input assembly you want to instrument. Right click on the Project node of the left hand menu (titled “Dotfuscator1”) and select “Add Assemblies”. Navigate to your working directory and select the Witty.exe file.
By default, Dotfuscator will harden your application by obfuscating it and renaming all possible items within your assembly. For this demonstration you don’t need renaming, so turn it off by selecting the Renaming menu node, right clicking on it, and unchecking the Enable option. Next, enable instrumentation: navigate to the Instrumentation node; select the Options tab; check the Enable Instrumentation checkbox; then check the “Send application analytics messages” checkbox. This tells Dotfuscator to inject the analytics instrumentation into your application.
Instrumentation Step 1 – Required Assembly Attributes
All instrumented assemblies must have identifying data embedded in them so that it is accessible at runtime. This data is included in every message and is used to identify the application, its owner, and its components for reporting purposes. Additionally, you must identify the points in your application where usage tracking is going to be injected. Both can be done either by using custom attributes to decorate your assembly and methods, or by using Dotfuscator’s Extended Attributes (where the configuration information is stored in the Dotfuscator project file for use during the Dotfuscation process). For this exercise we are not modifying the application’s source code; therefore we will use Extended Attributes exclusively.
You add the required identifying data using assembly attributes as described in the Configuring Feature Analytics section of the “What’s New With Dotfuscator in Visual Studio 2010 Beta 1” post. You will need to add both a BusinessAttribute and an ApplicationAttribute to the project with the appropriate CompanyKey and ApplicationKey values.
Instrumentation Step 2 – Tracking Application Usage
The final step is to tell the code injection engine in Dotfuscator where to inject the tracking logic. Runtime Intelligence is designed to track usage within an application session, so it records both the start up of an application and its successful exit. This allows you to track the actual usage of your application and to gather information on how often your application terminates unexpectedly. There are two injection points that need to be defined: the application start up and application shutdown methods. This is accomplished by adding the Setup and Teardown attributes as described in the Configuring Feature Analytics section of the “What’s New With Dotfuscator in Visual Studio 2010 Beta 1” post.
The Main method of the App class in the Witty Twitter application is the logical place for the startup. In Dotfuscator CE’s Instrumentation treeview, navigate to the Main method on the Witty.App class. Right click on the Main method and select Add Attribute. Select the SetupAttribute. Leave all of the SetupAttribute’s properties at their default values.
Finally, the normal exit point of the application needs to be specified so that code to implement a clean shutdown can be injected into the correct location. Having a shutdown process provides a dual benefit: first, it ensures that all cached usage data is flushed from the in-memory buffer; second, it sends a closing message to the data collection portal so that this run of the application can be correctly tagged that it completed normally and did not shut down unexpectedly. The best place to perform shutdown is in the last possible method in the execution flow, where you are certain that the user intends to exit the application. For Witty, this is the OnClosed method in the MainWindow class. To add the Teardown attribute, right click on the OnClosed method of the MainWindow class, select Add Attribute, and choose the TeardownAttribute.
These two steps are all that is required to alter an existing application to report on its usage live from the field. To finish, save the Dotfuscator project, then click on the Build icon to start a build. By reviewing the build output pane, you can see that your application had both an analytics setup and analytics teardown injected into it and that renaming was disabled.
By default, Dotfuscator saves the assemblies it outputs into a subdirectory named “Dotfuscated” below the project file. Navigate to that directory and you will see a new Witty.exe file that has the same functionality as the original, but also reports its usage back to the Free Runtime Intelligence portal. Now, back up the original Witty.exe in the Witty Program Files folder and copy the version from the Dotfuscated folder to the Witty Program Files folder and run the program.
Whenever the modified Witty starts up, it will send an application start message to the Free Runtime Intelligence Portal. When you exit the application, it will send an application shutdown message, indicating that the application terminated normally.
Reviewing Accumulated Usage Data
To review your application’s usage, navigate to the free Runtime Intelligence Reporting Portal at http://free.runtimintelligence.com . Navigate to the Login page and enter the CompanyKey value that you created as part of the BusinessAttribute above. As the free portal service does not provide any authentication or privacy guarantees, any data that is sent to it should be considered publicly accessible although not publically visible by default.
As Runtime Intelligence is an analytics and trending solution, usage data is not immediately available in the reports. There will be a delay before data from an application run is aggregated into the reports.
High Level Graphical Analysis
The first usage report we will look at is the “Application Dashboard” report. This is a high level graphical overview of how often your applications ran. In this example we can see that for the default reporting period (the past 30 days) we had a total of 10 application runs. The Witty Twitter application accounted for 60% of that activity with the remaining 40% being the MedicalImage application. The number in parentheses shows the number of incomplete application runs (those whose startup message was received but the method instrumented with the teardown code was never executed). From this, we can infer that up to 10% of the time our users may have experienced an application failure with our MedicalImage application. This is an indicator of possible application instability not a definitive measurement. There may be an additional exit point in our application that we did not notice, the user may have completely disconnected their computer from the network while running the application, or any number of other issues could have prevented the portal from receiving the teardown message.
The “Incomplete App Runs” chart gives a representation of how stable applications are overall and in relation to each other. In this case we can visibly compare the stability of our applications.
The next chart is “Application Usage Over Time”. This chart tracks the frequency of our applications usage, incomplete sessions and tampers over time. It provides immediate feedback, allowing us to see if incomplete application sessions are trending higher or lower as we release new versions and how often our application binaries are tampered with in the wild.
In addition to the usage data, we can see an overview of which operating systems and framework versions our applications are running under. We can also see the percentage of application sessions that are unstable under particular operating systems or frameworks.
High Level Tabular Analysis
Another usage report, the “Application Scorecard” provides a more detailed tabular view of application usage. Usage reporting is broken down by application unique identifier and application version. Application usage (including incomplete and tampered instances) is broken down by the application’s unique identifier and version. From this you can see the most used version of the operating system and framework per application version. In the commercial version of Runtime Intelligence, you have the ability to uniquely identity each running instance of your application with a serial number. This report will then summarize how many unique serial numbers were run and the total number of unique users.
The commercial version of Runtime Intelligence includes an additional report “Usage By Serial Number” - the free portal provides a sample.
By default, the data shown in the application reports consists of the total usage for all applications with the same CompanyKey with activity in the last month. To change the report parameters, expand the parameter toolbar at the top of the report. For all reports you can change the date range. In the application reports you can also filter by an individual application or a specific version of an application, as well as specific operating system and framework versions. Once you select the criteria for a new filter condition, click the “Refresh” button to display the filtered report.
Measuring Application Usage and Adoption
The number of unique users in the “Application Scorecard” report is a good indicator of how many unique users ran a particular version of your application. You can compare this value with measurements of how many downloads of a given version occurred and infer an adoption ratio for your application. This data is an excellent indicator of the adoption rates of your applications. If your usage numbers are high but the number of users is low then you can infer that your application is both useful and being used but may not be visible enough in the marketplace. If you have a high number of users but low average usage you may want to consider discoverability or documentation improvements as many of your users may be frustrated and giving up on using the application.
In addition, by examining the “Application Usage Over Time” section of the “Application Dashboard” and setting the filter to different versions of your application, you can visualize the upgrade pattern of your applications as you watch the usage numbers of older versions decline over time, to be replaced with higher usage of newer versions.
By examining the operating system and framework measurements, you can determine how rapidly your users upgrade their platforms. This visibility can help you plan support for new technologies. You can also refine your testing to ensure that your test environments match those of your users.
Usage information can be used to determine support lifecycles. You can back up your supported versions policies with hard data of how often older versions of your software are used. This information can also be used to measure the success of marketing efforts by providing measurements of how (or if) various campaigns resulted in higher usage of your applications.
Conclusion And Next Steps
In this post we have taken an application and added significant value by giving the application creator the ability to track actual real world usage in near real time. We have enabled trend analysis over multiple releases of the application to measure upgrade adoption. In addition, having hard data on usage patterns and runtime environments is invaluable when analyzing where to focus your development and testing time.
In future posts I will show how to perform more in-depth application tracking, how to protect your application binaries from tampering, and how to inject custom expiration behavior into your applications.