Archive for the ‘Application Lifecycle Management’ Category

Increasing App sales with Analytics: Free apps versus trials

Tuesday, June 14th, 2011 by Sebastian Holst

In my previous entry I introduced my app, A Pose for That, explained how I had instrumented my app with Runtime Intelligence to better track user experience and behavior. As a case in point, I illustrated how strategically placing upgrade opportunities in various locations inside my trial version, I was able to increase my conversion rates – perhaps by as much as 50%!

HOWEVER, when I was at MIX11, a very experienced developer (let’s call him David because that’s actually his name) told me that he had already established the optimal app blend to maximize revenue – it was to have a free app (not a trial version) that also offered ways to upgrade to the premium app. He pointed out that trial apps do not show up as free in the marketplace and are therefore almost always overlooked by most casual marketplace browsers. A free app gets the eyeballs that a trial misses.

For those who know me, they know one of my core principles is that my ideas never have to be original, they only have to be good – so is David’s idea really a good one?

To test it out, I created Yoga-pedia, a free app that included the browsing capabilities of A Pose for That with good imagery and instruction, but did not include the pairing of poses to real-world situations (a feature I believe is valuable) or flows (the stringing together of multiple poses). On the welcome page (and one or two other places) I give users a chance to learn more about our software and upgrade; Here is the welcome page and the “tell me more about why I should upgrade page.”

I instrumented the various points where users can upgrade in both the trial and the free app so that I can compare BOTH the usage levels of the two apps AND the upgrade requests that stem from that usage. So… let’s go to the video tape – or better yet, Runtime Intelligence. (Note – these specific graphs are built by extracting the data from the runtime intelligence repository into a spreadsheet and then generating a simply pivot table).

By looking at application starts (not downloads in the marketplace sense of the word), the graph showing App Runs seems to support David’s logic; my free version, Yoga-Pedia, takes off like a rocket and within 24 hours eclipses trial activity in dramatic fashion. …but, it also seems to be cannibalizing trial activity too – Should I care? (NOTE – I am combining usage of multiple applications – not always easy to do with canned dashboards)

I probably should care IF users are more likely to upgrade from A Pose for That trials versus from Yoga-pedia. In other words, are my sales going up because of the free app even though it is depressing my trial volume? Let’s go to Runtime Intelligence one more time…

What the graph above shows is that upgrades from my trials also decreased dramatically with the launch of Yoga-pedia, BUT the volume of upgrade requests from within Yoga-pedia more than made up for that shortfall. (NOTE – I am combining feature usage across multiple applications – not always easy to do with canned dashboards)

In the one week where both the free and the trial versions lived side-by-side, the free version generated 86% of the upgrades.

More important is the bottom line: I saw an 85% increase in the total number of upgrades when I had the combination of both a free and trial version of my app available.

Coming up next (I promise this time) will be a discussion of the last leg of David’s magic formula for success – making your free version ad-driven. What will Runtime Intelligence be able to tell us about that?

Build a Windows Phone 7 app and secure a $100 donation for your local user group

Friday, February 18th, 2011 by Sebastian Holst

Grow Your WP7 Dev Skills

The only thing better than doing something good is when it also helps you do well – and that is the spirit behind a joint initiative between PreEmptive Solutions, Telerik, and Microsoft.

What’s the deal? Simply put, any developer who develops their own Windows Phone app using both Telerik controls and Runtime Intelligence will:

  • Have $100 donated to their local user group (that’s doing good), AND
  • Have an app on the marketplace that combines a great user experience with the usage and adoption analytics to prove (and improve) it (that’s doing well).

Of course, nothing can be this simple right? WRONG! All you need to do is:

  • Develop and deploy any WP7 application using Telerik RadControls for Windows Phone AND PreEmptive Runtime Intelligence for Windows Phone, AND
  • Get this done and certified before May 31st 2011

By the way, there is no limit to how many apps you and your user group can submit, but we will only recognize the first 500 qualifying applications – so build early and build often!

One more important point - developers will also have free access to all of the required software. So go on – click through and do some good at http://www.telerik.com/windowsphoneFTW.

Riddle me this! Where can French, Italians, and Germans all agree?

Tuesday, February 1st, 2011 by Sebastian Holst

Well, apparently, its in their tendency to pair wine with spicy Chinese food.

How did I come up with such a farfetched notion? The answer to “how” is “easily” when I started with a WP7 app like VinoMatch’s Mobile Sommelier that was instrumented with Runtime Intelligence for Windows Phone. But let me back-up a little.

On Nov 4, 2010 MSFT and LG announced that LG Owners Get Free Access to Popular Windows Phone 7 Applications. You can read more about the 10 premium apps and another 10 LG-specific apps on LG’s Facebook page here. They call the program “hAPPiness!”

Anyhow, it turns out that all 10 of the premium apps were instrumented with Runtime Intelligence and have been sending analytics since the January launch of the program.

Last week, Microsoft invited lead developers from all ten development teams on campus for three days to gather feedback as well as to hold some specialized training on a variety of technical topics – and I was fortunate enough to be invited to lead a discussion on best practices for incorporating mobile analytics into the development process. In support of this presentation, I was given permission to look into the runtime data that these 10 apps had been collecting to see what I could see.

The 10 premium apps that are free from LG through March 10th are:

1. Cocktail Flow by Gergely Orosz
2. Color Sprouts by Jarek Kowalski
3. Colorize by Kitron Software
4. Doodle God by JoyBits Ltd. (a Top 100 Paid App)
5. Envision by Dotnetfactory
6. Krashlander by Farseer Games (a Top 100 Paid App)
7. Mobile Sommelier by VinoMatch
8. Mr. Hat and the Magic Cube by BRAVO game studios
9. Talking Ragdoll by Spritehand LLC (a Top 100 Paid App)
10. Weave by Seles Games (a Top 100 Paid App)

While I can’t share the entire presentation here, VinoMatch and Gergely Orosz were kind enough to give me permission to include some of their mobile analytics in this discussion. I want to highlight two powerful analytical techniques using their real-world data.

Conversion feature analysis

Embedded within most apps are a few keystone features that drive conversion (from eval to paid, prospect to customer, fan to fanatic) and Cocktail Flow has one under development, Shopping Assistant. It does not yet connect directly to a store and, as a nascent feature, is not yet a centerpiece in their UX design. Here is what I was able to deduce from the Runtime Intelligence data:

  • Only 5.5% of all user sessions used the shopping assistant feature.
  • Of those, 49% used shopping assistant only once inside their session, and 51% used shopping assistant multiple times. Of those 51%, they used it for an average of 2.9 times per session. What were they doing?

With Runtime Intelligence, I am able to map out the full usage trail, feature-by-feature, session-by-session, and so I am also able to visualize:

  • Where in the sequence of features of each user session shopping assistance was being called.
  • And when it was used, was it a part of a “one use” session or one of those sessions where the user called the shopping assistant multiple times in a single session.

I can also see that the overwhelming number of Cocktail Flow users using this feature rarely used more than 10 features in their session and that virtually all lengthy sessions (taking more than 14 actions) were always churning (going back to the shopping assistant feature multiple times).

When this feature is functionally complete to the developer’s satisfaction, they will now have benchmarks to measure feature adoption (increase it over 5.5%) and session behaviors (when, how often, and under what circumstances the feature is used) all relating to what may ultimately be their most lucrative app feature.

Mutli-variable AB testing and cross-domain user profiling


In the Cocktail Flow example, we focused on a single feature. With Mobile Sommelier, we are going to add a few more dimensions.

The following graphic shows, as a relative percentage, how often ten selected features inside the Mobile Sommelier application were used over a 6 day period.

What we can see immediately is that there is very little variability over time. The population of users generally uses the application in the same way over time.

What can we do with this perspective?

If we were to release a new version of this App, we could immediately see any meta shift in behavior across the two versions (Runtime Intelligence will automatically roll-up multiple versions and then break them out when asked).

What CAN’T we do?

While we can see how often the “ShowTasteCard” feature has been used, we CAN’T readily determine which wine/food pairing users are most (or least) interested in. We can’t analyze custom, application specific data.

Luckily, Runtime Intelligence CAN grab this data – with no programming – all post-compile. For example, this (very busy) chart shows the how often (by percentage) users want to pair wine with specific foods (like BBQ or spicy Chinese) AND it is further broken out by country.

Of course, this is too busy to read in its current form, but the trend lines do immediately show that while all users may use the “ShowTasteCard” with generally the same frequency (as indicated in the last graphic), it is clear that they use the feature DIFFERENTLY, e.g. they care about different topics. Culture counts!

Let’s drill down.

Focusing in on just three countries (France, Germany, and Italy) and focusing on just a few of the food options (from AntiPasto through Chocolate Cake), we can see genuine differences in user behavior and preferences.

Perhaps it is not surprising that French are more interested in Caviar, Germans in apple pie, and the Italians in antipasto.

On the other hand, maybe we were not expecting Italians to show such a strong interest/preference for BBQ. Could this data point to a way to sell Zinfandel (a US export based on a grape that originated in Italy) into the Italian market? – as the US wine for US food (BBQ).

Some general best practices

At the end of the session, the developers generally agreed on the following developer patterns:

  • Instrument key usage and application milestones (Not just clicks and page views)
  • Capture user selections and preferences (Not just selection and preference setting events)

Other conventions and best practices

  • Implement exception reporting
  • Integrate runtime intelligence into your CRM, ALM, and Marketplace repositories

Application Analytics are not your older brother’s web clone for the phone!

These have been just a few small examples of the rich set of analytics that are now available (both on and off of the phone). So, if you think Runtime Intelligence is just another web analytic clone for the phone, think again – Runtime Intelligence provides application analytics – not web analytics (see my earlier blog Application analytics: a new game brings new rules).

In a nutshell, this means that these analytics include:

  • Custom data (including complex objects)
  • Session, feature, event, and method-level precision (not just event)
  • Exception reporting (unhandled, handled, and thrown)
  • Support for both Silverlight and XNA
  • Opt-in policy enforcement, SSL transmission, and caching
  • And were implemented using post-compile injection eliminating the requirement to change (or even recompile) source code.

Anyhow, I am ALWAYS eager to hear from developers who have employed analytics in cool and innovative ways – PLEASE reach out and share the wealth! Thanks

Automating Runtime Intelligence Code Injection

Monday, August 30th, 2010 by Joe Kuemerle

With the release of Dotfuscator Community Edition in Visual Studio 2010, developers were given a way to quickly and easily add application analytics functionality to their projects at no additional cost. Using Runtime Intelligence, developers or build managers can implement a customer experience improvement program to track how people are actually using the deployed applications in the wild. In May 2010, PreEmptive Solutions partnered with CodePlex to provide commercial level Runtime Intelligence data reporting to any project hosted on CodePlex.

The version of Dotfuscator that ships with Visual Studio 2010 can’t be added to an automatic build script because it requires user interaction to perform a build. The commercial version of Dotfuscator has both an MSBuild task and a command line option for build automation, but purchasing a commercial license for a pure open source project is normally not an option. At the request of the community, we are making two options available to help alleviate this issue.

The first option, available to anyone who has Visual Studio 2010 Professional or higher, is to register for and download a patch to upgrade the installed version of Dotfuscator CE to version 5.0.2601. This version of Dotfuscator includes a command line interface and removes the restrictions that block automation. It does not require any user action to start a build, nor does it require Visual Studio to be running. The instrumentation abilities are identical to the previous versions: up to 10 named features with none of the advanced functionality from the commercial version (custom data gathered at runtime, detailed system and performance data, unique serial numbers, custom tamper and expiration behavior). To download this patch, you will need to create an account here . Go to My Account and the patch will be available in the Downloads section. After you install the patch, you will have a new DotfuscatorCLI.exe application in your Dotfuscator CE installation directory which you can use to automate builds. The command line is similar to the commercial version of Dotfuscator and is documented in the Command Line Interface reference in the online documentation .

The second option is that open source projects hosted on CodePlex can use the commercial version of Dotfuscator to instrument the application. Any project administrator can request a PreEmptive Open Source Project License here and receive the full version of Dotfuscator to use as the Runtime Intelligence code injection tool for their project. The project will then have access to the full range of instrumentation features available to commercial users including custom data gathered at runtime, support for occasionally connected clients, easy integration for Silverlight XAP and ClickOnce packages and ongoing enhancements as we release new commercial versions of Dotfuscator. Additionally, the project will be able to use the MSBuild task or the command line to easily integrate into their automated build environment.

With these options, developers are now better able to get accurate and actionable feedback on how their applications are really being used and can tighten the feedback look between themselves and their users. Better information on usability and functionality will lead to improved applications, and now it’s easier than ever to get that data.

How do I love thee? Let me count the ways.

Thursday, August 20th, 2009 by Sebastian Holst

“How do I love thee? Let me count the ways. I love thee to the depth and breadth and height” - Sonnet 43, Elizabeth Barrett Browning

So “what’s love got to do with it?” (Private Dancer, Tina Turner) Hint: if people live for love, then businesses live for money

On July 14th, Microsoft announced Azure pricing and a “grace period” through PDC 2009. A primary rationale here is to enable development organizations to optimize deployment and monetization models to maximize Azure commercial opportunities.

So, whether you are a romantic (like Ms Browning above) or perhaps more hardened like Tina Turner’s Private Dancer (or Stanley Kubrick a la Full Metal Jacket), one thing is for sure - Microsoft wants Azure to “love you long time.” How deep, wide, high or long is the question.

Check out a this article in SD Times - PreEmptive’s Dotfuscator instruments Azure applications By David Worthington – where Dave Worthington makes many of the very same points.

Of course, we announced Runtime Intelligence Service (RIS) Azure support to help developers answer these very questions. While perhaps not as soaring as a sonnet – Runtime Intelligence allows for any .NET component deployed into Azure to be injected (post-build) with session, feature and method level monitoring. The runtime intelligence is streamed out of Azure for analysis. Other than writing a custom solution, this is perhaps the only means to measure adoption, usage patterns and performance inside Azure in near real-time.

Now, my posts are all intended to help you (blog followers) find more ways to make more money (we want to spread the love). So, you will note that I very specifically said the RIS helps to answer these questions. What the Azure development community really needs is an ROI calculator that will combine real usage data (from both legacy and piloted Azure applications) with Microsoft pricing and the offset IT expenses to come up with an Azure ROI calculator. I know there are lots of calculators being written – but how many of them can incorporate actual usage data before and after deployment to the cloud? That’s not our business – but could it be yours?

If yes, let me know and I will make sure you have what you need to call our RI Service via our RESTful API – making your calculator uniquely able to reliably predict cloud ROI.

As always, i have a more philosophical take on this issue on my personal blog at http://apps-are-people-too.blogspot.com/2009/08/how-do-i-love-thee-let-me-count-ways.html