Archive for the ‘Dotfuscator’ Category

« Older Entries

The Microsoft sponsored service for WP7 ends – the PreEmptive sponsored service debuts

Tuesday, November 29th, 2011 by Sebastian Holst

At 24:00 EST on December 9, 2011 the Microsoft sponsored protection and analytics service for Windows Phone 7 will be shut-off.

A different service fully and solely subsidized by PreEmptive Solutions will take its place. This service is materially different – please read the following notice carefully for information on how to continue to work with PreEmptive Solutions technology for Windows Phone.

Background

While Microsoft’s sponsorship expired on September 30, 2011, PreEmptive continued the service for an additional 60 days at our own expense while we explored a variety of options to continue our support for the Windows Phone development community. Microsoft’s sponsored service has ended, but our commitment and support for this community continues unabated.

PreEmptive’s challenge was to find an affordable means to support for the burgeoning WP7 development community without compromising the quality and capabilities unique to our protection and application analytics technologies; we believe the following provides both a valuable set of services at little and no cost for small WP7 development efforts with a smooth “on-ramp” for larger development projects and for organizations with more demanding service levels, governance or scalability requirements.

Summary

Obfuscation and Instrumentation continues at no cost through 12/31/2012.

Dotfuscator for Windows Phone, the post-compile tool that obfuscates and injects application instrumentation will continue to be offered to Windows Phone 7 developers at no cost through December 31, 2012.

Mobile analytics endpoint (wp7.runtimeintelligence.com) will be shut-off on 12/9/2011

The current analytics endpoint will be discontinued. However, developers have a number of options that they can consider;

· Subscribe to the PreEmptive Solutions commercial endpoint. This is a fee-based option that includes all of the features currently offered PLUS an advanced mobile portal, a RESTful API, and a higher service level (plus support beyond WP7). For more information, email sales@preemptive.com.

· License PreEmptive Analytics for TFS. This is also a fee-based option. This solution is an on-premises solution focused on exceptions rather than feature tracking. For more information, see Using Analytics for Windows Phone and Azure Exception Tracking - User Community Virtual Series and email sales@preemptive.com.

· Develop and host a homegrown endpoint. This is a no-fee option but development will be required. The CodePlex Runtime Intelligence Endpoint Starter Kit repository starter kit project may be of some help.

· Plan to migrate to the PreEmptive Analytics for TFS community edition to be included with Dev-11. This is a no-fee option but is NOT yet generally available from Microsoft. For more information, see the video A Lap Around PreEmptive Analytics for TFS with Justin Marks .

· Publish their app as a CodePlex project and utilize the CodePlex analytics endpoint (this is different than the option above). This is a no-fee option. For more information, see this tutorial (note that this assumes the developer is limited to the Community Edition of Dotfuscator – but the WP7 edition has full functionality).

The following feature summary table highlights the three principle options available to the Windows Phone 7 development community with a comparison to the discontinued Microsoft sponsored service. (click thumbnail to enlarge)

Click to enlarge

Click to enlarge

FAQ

Q: Will developers have to republish my WP7 app on or before December 9, 2011?

No

Q: Will users notice any difference in app behavior after December 9, 2011?

No

Q: Will I have to re-register my installation of Dotfuscator for Windows Phone 7?

No

Q: If I have only been using Dotfuscator for obfuscation, will I lose any functionality or will I have to do anything differently?

No

Q: Will developers have access to earlier runtime data generated by Runtime Intelligence for Windows Phone after December 9, 2011?

They will not.

Q: Where can developers ask additional questions regarding migration, upgrades or discontinuing use of PreEmptive Solutions technologies?

Post to the PreEmptive forum at http://www.preemptive.com/forum/index.php?f=26&sid=dfe90c2ba80de07692372dae962c58b2&rb_v=viewforum

PLEASE NOTE – this is NOT a moderated forum.

Q: Why would a development organization upgrade to a professional SKU of PreEmptive Analytics?

· Multi-platform (WP7, Android, JavaScript, all .NET and Java, native API…)

· Private endpoint (for large-scale enterprises with demanding scalability, governance, or other unique requirements).

· Analytics for TFS option (out-of-the-box integration with Microsoft Team Foundation Server).

· True application analytics such as custom data fields, development ownership of data, true SLA and support for developers, etc.

Conclusion

Of course we would have preferred that Microsoft had opted to extend their sponsorship for Runtime Intelligence for Windows Phone, but that was not the decision that they ultimately made. However, over the past 12 months, we had a front row seat watching a flood of innovative apps launch. We know that a good percentage of the WP7 development community relied upon PreEmptive Solutions for both analytics and protection (in a recent analysis of the marketplace, it was shown that 17% of all apps used either protection, analytics or both).

This experience combined with our confidence in the future of the Windows Phone platform has prompted us to extend free access to Dotfuscator for Windows Phone. We look forward to our continued support and participation in the growth and success of this exciting technology and marketplace.

Posted in Dotfuscator, Runtime Intelligence | No Comments »

Runtime Intelligence and Dotfuscator for WP7 developers speak (Mikey likes it!)

Monday, September 26th, 2011 by Sebastian Holst

In my last post, I drew a correlation between apps that used Runtime Intelligence and their relative (positive) success as measured by user ratings and engagement. While it was fairly clear that developers who chose to use Runtime Intelligence built more successful apps than their counterparts, it really said nothing about a) Runtime Intelligence analytics’ contribution to their success or b) developer satisfaction with Runtime Intelligence overall.

Well, there’s really only one way to answer these questions …and that’s to ASK THEM.

I sent out an electronic survey starting on Monday of last week (September 19) and have received over 200 developer responses. Here is what they said…

Who is doing what?

  • 32% indicated that they are only using Dotfuscator to protect their application
  • 24.5% said they were only using Runtime Intelligence
  • And 43.4% indicated that they were using both.

Do these “smarter than the average bear” developers see the value?

In a word, YES.

Looking only at those developers who indicated that they already had their applications in the marketplace (representing over 100 development organizations):

  • 60% indicated that analytics set the wp7 platform apart from all other platforms or added significant value to the platform.
  • 68% indicated that protection set the wp7 platform apart from all other platforms or added significant value to the platform.

Analytics’ perceived value increases by 450% with developer experience

When looking at those developers that indicated that analytics and/or protection “set the WP7 development platform apart from all others,” analytics’ value actually increased by 450% (from 2% to 9%) as developers moved from no app, to less than four weeks to a ship date, to actually having an app in the marketplace (and getting analytics back). Interestingly, obfuscation (protection) peaked in value just prior to shipping.

So what’s the takeaway?

In my last post, we established the user of Runtime Intelligence were more successful than other WP7 developers. In this post, we see that these developers credit their success, to some material degree, to either Runtime Intelligence or Dotfuscator protection (or both).

In their own words
(selected - but unedited - responses to the open-ended question “what are you most excited about?”)

I like being able to get crash reports without much additional work.

It gives the developer ability to know about usage patterns in an application. Obviously code obfuscation is a necessity, especially for paid apps.

It offers a unique way to see how users interact with the application, and with the latest release it also has error reporting. Awesome!

I’m an excitable person.

Fabulous data provided by RIS to analyze the performance, usage and app demographics.

So I can know what is happening in my app and protect my code.

Used correctly, the analytics really let me see how and by whom my application is being used. I get more insight into this information than I could if I set up a usability lab or just did extensive user testing. There is no better way to observe than to do so in production.

The concept of attaching runtime analytics after the compilation process is very useful for us (standard software development, single application in various customer-specific configurations), since we are able to attach this on a per-customer basis and don’t have to manage it in code.

UI for parameterization

It gives detailed statistics about the usage of all parts of the software and helps to recognize the hot features of the software are and which parts are less used. This adds great value into the effort of making software better.

Really gives me insight into what my customers are doing with my applications. They help me to understand where I can enhance functionality and add value.

Quality of product

Analytics give me an idea on what I should work on next to improve my application

Runtime analytics is cool because there is no code to write.

I can collect the exact information i need.

It allows me to phase out or strengthen certain parts of my apps. I currently have seven apps and the instrumentation is crucial.

Because i can have a deep analysis of when and especially how my application is being used. If you add the fact that all these data are aggregated and presented in such a nice way by the portal, you end up with a great product

I produce libraries (DLL’s) that are handed to third parties, hence the need for obfuscation.

Kickass obfuscator.

It helps me keep track of any bugs. And it allows feature tracking. And it gives me the cool world map that shows where some of the users are.

Posted in Dotfuscator, Runtime Intelligence | No Comments »

The new WP7 App Hub reporting is great – and it’s even better with analytics!

Tuesday, July 19th, 2011 by Sebastian Holst

Warning – this is a cliff hanger post. If you don’t like mysteries, come back in two weeks…

Like anyone else who has an app inside the WP7 App Marketplace, I noticed that the App Hub was down most of yesterday with the promise of a functional upgrade in the works – and today I was very pleasantly surprised to see the result; a streamlined experience with expanded capabilities.

One of the first things that caught my attention was the exception reporting by app and by date; very useful indeed. Of course, MSFT is quick to point out that (and I quote) “Crash count alone isn’t a direct measure of app quality. Popular apps may have higher crash counts due to higher usage.”

Well that seems self-evident, but without usage metrics how can I evaluate the severity of my exception report counts? …. (and now, unless this is the first post of mine that you have ever read, you must know what’s coming).

To the cloud! (Sorry, I couldn’t resist). Using Runtime Intelligence for Windows Phone, I’m able to measure total sessions – by extracting these counts by day and mashing it up with exception counts from the marketplace – I can now supply the missing ingredient to make the exception count on the App Hub meaningful. (NOTE – I had to manually transcribe exception counts from the App Hub as there is no tabular option and the detailed download drops the daily count as it de-dupes the exceptions).

The App Hub is careful to point out that only apps running NODO (or Mango) can report exceptions, so I first had to remove the Runtime Intelligence session data coming from earlier versions of WP7 (an interesting statistic on its own).

Here is what I see… (and a warning here – the numbers aren’t pretty)

I took two apps of mine; Yoga-pedia and A Pose for That and looked at their respective usage on NODO+ phones via Runtime Intelligence and exception reports from the App Hub and then calculated the ratio of sessions to exceptions.

The time period I used for this test was the two weeks from June 12 to June 25. During that time, this is what I observed:

  • 66% of A Pose for That sessions were run on NODO.
  • 58% of Yoga-pedia sessions were run on NODO.

Here is the ratio of exceptions reported by MSFT and sessions from Runtime Intelligence… (click to enlarge)

Ratio of session counts and exception counts by day

Now there are three likely scenarios here.

  1. Over this two week period, both apps were crashing every 1 in 10 times they were run (HORRIBLE). I don’t think this is the case because I have run these apps myself on multiple phones hundreds of times and they have NEVER crashed.
  2. The App Hub is over-reporting exceptions (or somehow incorrectly associating exceptions with these apps). This is a beta feature on the App Hub – it’s certainly possible.
  3. Runtime Intelligence is way under-reporting the total number of sessions in a given day. Certainly possible, but given the unit testing I have done, I don’t see this as being a major contributing factor to these ratios – but certainly a possibility.


Now, I had already put a “feature tick” on the default unhandled exception handler to count how many times it was invoked during this same period. The counts I have are well below the App Hub numbers (which might suggest number 2 above is the culprit – BUT NOT SO FAST). It is more than likely that certain exceptions (perhaps a majority) would interrupt the normal feature tracking transmission mechanism so I would expect that count from Runtime Intelligence to be artificially LOW.

As is often the case when managing an application “in the wild”, an unanticipated question has arisen and I find that I don’t have enough data. That’s why its ALWAYS so important to

  • plan in advance what data is worth collecting to minimize the likelihood that you will end up in this situation and
  • be sure that your analytic solution supports rapid and easy iterations and refinements to compensate for when your planning falls short.


So how am I going to determine if

  1. my apps offer a LOUSY customer experience everywhere except for my personal phones or
  2. one or both exception reporting counts and session tracking counts are flawed?

Easy - I’m going to post an update of my apps to the marketplace this weekend with Runtime Intelligence Exception reporting turned on. What?

Runtime Intelligence for Windows Phone includes its own exception tracking capabilities – it does require that the developer activate it (that’s why I don’t have that data now), but it offers a lot more data and it can be invoked for unhandled, handled, and thrown exceptions. Further, it can be configured to collect additional information (custom for the app), AND it can be extended to offer the user a dialogue to provide additional feedback if they like.

I will post my results over the next few weeks – meanwhile, if anyone has any suggestions or ideas – please let me know… I honestly have no idea how this little mystery will play itself out.

Before I sign off – here is one more tantalizing clue (although it may also be a red herring). When I look at the limited unhandled exception data currently being returned by Runtime Intelligence (I can see tower location, device manufacturer, OS, etc.), I see that well over 50% of the phones that had an exception were localized to a language OTHER THAN en-US – and that is way out of proportion to the actual usage trends that I have been tracking (and posted in earlier entries). Further, the localizations that had the greatest “disproportionate” number of unhandled exceptions were de-DE and de-AT. Coincidence? Conspiracy? We don’t need to guess – we will soon have the facts!

PS here are two links that may be of interest:


Enjoy!

Posted in Dotfuscator, Runtime Intelligence | 1 Comment »

A Webinar on Monetizing Mobile Apps with Analytics

Saturday, June 25th, 2011 by Sebastian Holst
For those who want a little more detail on the specific coding steps as well as an update on the latest application of analytics to mobile app development, we’ve scheduled a webinar. Here’s the info….. (the first time slot of 100 filled up in a few hours - so these are additional dates. We’ll keep scheduling these as long as interest is there) Cheers.

Title: Monetize Mobile Apps with Analytics
Registration: Thurs, Aug 4th at 11:30 EDT https://www3.gotomeeting.com/register/676857398
Registration: Wed, Aug 17th at 12:30 EDT https://www3.gotomeeting.com/register/433412582

Description: In this 60 minute webinar, we will take a live WP7 app and use real-world analytics to illustrate:

  1. The impact of try/buy scenarios on paid apps
  2. The relationship between free and paid versions of an app
  3. Strategies for ad-driven app design that consider page location, first time, occasional, and power user patterns, cultural trends, and other demographics including carrier and model profiling.

Preparation: NONE required. However, attendees are likely to get more from the presentation if they have already:
• Installed and are familiar with the Microsoft Windows Phone 7 development tools
• Installed and have some familiarity with PreEmptive Solutions Runtime Intelligence
• Installed and navigated around the free SKU of the sample application that will be referenced in the presentation. The free app is Yoga-pedia.

Posted in Dotfuscator, Runtime Intelligence | No Comments »

Improving Ad performance: Correlating ad activity with feature usage and user behavior

Friday, June 24th, 2011 by Sebastian Holst

In this third installment on application analytics patterns and practices I’m going to focus on how Runtime Intelligence can be used to shed light on Ad activity within the context of one or more applications. While the use cases covered here are nowhere near exhaustive, I’m going to show how to answer the following questions (and hopefully give some indication as to why you may care about the answers):

  • What are ad impression volumes across multiple apps?
  • What are the click-through rates (the ratio of users clicking on ads to the volume of impressions) across various pivots?
  • What influence does culture (country of origin) have on click-through rates, e.g. are Germans more or less likely to click on ads versus Italians?
  • What carriers/ISP providers are giving me the most business, e.g. where are my users most likely to be found?
  • Where are users spending most of their time inside an app? Does that usage pattern correlate with a user’s likelihood of clicking on an ad?
  • Do returning users interact with ads differently than first time users or power users?
Many of these metrics are valuable in scenarios other than ad effectiveness of course (knowing where users spend their time and understanding how power users behave are two obvious examples), but for this installment, I am going to focus exclusively on how Ad interaction can be viewed across these metrics.

Implementation
I’m using the same trusty one line method WhatPoseWhen that I described in the first installment – this time, I call the method on the New Ad event (to count impressions) and the Ad Engaged event (to count clicks on ads). I could just as easily collect data on any other ad-related event and grab any data that is available to the program at that point in its execution as well. Here is the code for that method in its entirety:

private void WhatPoseWhen (string page, string selection)
{ return; }

The first parameter tells me from which page the method is being called and the second parameter tells me why I might care, e.g. was a new ad displayed, etc.)

I pass the page name and the ad event into WhatPoseWhen and Runtime Intelligence grabs these parameters and sends them up to the repository (no programming for this). I can then correlate the ad activity within the context of sessions, feature usage, and runtime stack data that I am getting as a part of runtime intelligence.

For these metrics, I export my CSV data into a regular excel spreadsheet and then generate the pivot tables shown below.

App background
I always like to use data from true production apps rather than fabricate data sets; I am using two apps that I wrote and launched on the marketplace that are both ad driven, Yoga-pedia and A Free WPC Yogi – the former is a free version of a yoga app that (hopefully) helps to drive sales of a for an upgrade to A Pose for That. A Free WPC Yogi plays a similar role for The WPC Yogi, a tailored version of A Pose for That targeting WPC 2011 attendees.

The following post uses their Ad activity over the same one week period.

Impression counts
The following pie chart shows the “new ad” event count by application. As you can see, Yoga-pedia has roughly 4X the number of ad impressions and given the fact that these apps are very similar (but not identical) in their behavior, this also roughly correlates to the volume of usage as well.

Click-through rates
However, when I divide the total number of “ad engaged” events by the total number of “new ads,” I see that A Free WPC Yogi has a 28% higher click-through rate (1.78% versus 1.37). In point of fact the demographics of the app users are quite different (randomized consumers versus MSFT partners who are attending WPC 2011).

Advantage: This intelligence helps to segment users by differences in their behavior and to do a better job of targeting those differences across apps.

Impressions by country (or culture)
Runtime Intelligence can grab the IP address of the sending tower – this is not personally identifiable and cannot be used to locate an individual with any precision – but it is more than adequate to identify country, state, and city. In the following graph, I simply count new ad events by country and show the top 10 countries by impression volume.

Advantage: If your app has a cultural bias that would benefit from localization, understanding where your users are can help prioritize those localization efforts.

Click-through rates by country (or culture)
The following bar chart calculates the click-through rates for the top 10 countries listed above. What is interesting here is that there appears to be a significant difference in click through rates by country (culture).

Advantage: Understanding when/if users from specific cultures are significantly more likely to respond to (click on) ads can further help to prioritize localization or marketing investments.

Impressions by ISP provider (top 25)
To produce the next graph, I used an application to tell me who owned the IP addresses that my mobile clients are using (I used IP2Location – but there are many of them out there).

This is a nice way to see who my users favor in terms of their carrier. Here I only show the top 25.

Advantage: Understanding carrier popularity will help focus business development/marketing efforts and better manage potential risks associated with how your users may be negatively impacted by upgrade schedules (delays). Will your next app be dependent upon Mango?

Sessions per app page
In the raw CSV files that can be exported from the Runtime Intelligence portal, there is a column, ApplicationGroupId. The value in this column is unique for all signals (messages) that are sent from within a single app session. In other words, I can use this field to organize all user activity into the relative user sessions using this field. This is helpful for plotting specific user patterns.

The following graph simply counts the unique occurrences of ApplicationGroupId values by page name value (recall that this is the first parameter of the WhatPoseWhen method). This avoids counting multiple views of a single page within a single session and tells me how popular specific pages are across my user base. For this posting and for illustration, I’m only showing data for five specific pages.

FindAPoseDetail and BrowseSelectPose are central to the user experience (browsing for yoga poses and then drilling into a specific pose for detailed imagery and instruction). TellMeMore is the page where I describe what comes with the paid version of the app (nice to see that 10% of my users deliberately choose to investigate the upgrade possibility) and AppGuide and TopicList are essentially app documentation and I can see that these pages are not hit very often – and that’s not a bad thing – users should not need to use the documentation after their first use.

So – this graph is telling me that

a) My users are spending their time using the app rather than trying to use the app
b) I am at least getting my user’s attention regarding a possible upgrade – perhaps my content is not compelling enough if my conversion rate does not correlate.

Advantage: broad user proofing can be used to validate developer assumptions about user experience and effectiveness of pages for their specific purpose.
Ads shown per page compared to volume of times viewed
Next I calculate the average number of ads shows per page by dividing the total count of New Ad messages by page (this combines the two parameters, page name and the even New Ad) by the total count of the times the page is shown. TOTAL ADS SHOWN PER PAGE / TOTAL TIMES PAGE VISITED

I use the same ad duration interval across all of my pages – so this is actually another means of calculating how much time my users are spending on each page (this can be done with Runtime Intelligence alone, but in this case, I don’t have to do that).

The graph below shows the average number of ads shown per page and maps them to where they rank in terms of how often the page is visited.

Happily, the two core pages of my app also get the most ads (and are also where my users are stopping to spend time). I can also see that users spend more time on detailed pose descriptions than they do browsing – even though the browse more often than they drill down (which makes perfect sense).

Sadly, my upsell page is getting the least love – I definitely have to work on making this page more engaging.

Advantage: Ad frequency by page provides insight into where users spend their time. Calculating click-through rates by page identifies where users stop to look around and may be most open to suggestion.

Returning users and sessions per user
Another column in the CSV extract is the ANID – this is either the result of hashing the true ANID from a user’s phone (it is not the actual ANID value), or, if they opt-out of that, it will contain a GUID generated by our software and written to isolated storage. In either case, this value acts as a unique user identifier.

The ANID can be used to identify new and returning users. Dividing session count (ApplicationGroupId) by ANID gives the average number of sessions per user. The following bar chart takes the 10 ANIDs with the highest session counts and compares the resulting sessions per user value to the rest of the user base (whose count is roughly 500 other users).

What I see is that there is a core group of users that are heavily using my apps (YAY!). Now that I know who they are, I can zero in on their specific behaviors, how they relate to my ads, what features they use most heavily, etc.

Advantage: Segmenting users into new, returning, and power categories dramatically improves a developer’s ability to target, prioritize, and validate development, marketing, and support activities.
Conclusion

I hope to have shown how using Runtime Intelligence, developers can materially improve their ability to build more effective applications and refine their advertising strategy while coordinating that strategy with complimentary upsell strategies as well.

Advantage: Development!

Posted in DashO, Dotfuscator, Runtime Intelligence | No Comments »

« Older Entries