The Microsoft sponsored service for WP7 ends – the PreEmptive sponsored service debuts

November 29th, 2011 by Sebastian Holst

At 24:00 EST on December 9, 2011 the Microsoft sponsored protection and analytics service for Windows Phone 7 will be shut-off.

A different service fully and solely subsidized by PreEmptive Solutions will take its place. This service is materially different – please read the following notice carefully for information on how to continue to work with PreEmptive Solutions technology for Windows Phone.

Background

While Microsoft’s sponsorship expired on September 30, 2011, PreEmptive continued the service for an additional 60 days at our own expense while we explored a variety of options to continue our support for the Windows Phone development community. Microsoft’s sponsored service has ended, but our commitment and support for this community continues unabated.

PreEmptive’s challenge was to find an affordable means to support for the burgeoning WP7 development community without compromising the quality and capabilities unique to our protection and application analytics technologies; we believe the following provides both a valuable set of services at little and no cost for small WP7 development efforts with a smooth “on-ramp” for larger development projects and for organizations with more demanding service levels, governance or scalability requirements.

Summary

Obfuscation and Instrumentation continues at no cost through 12/31/2012.

Dotfuscator for Windows Phone, the post-compile tool that obfuscates and injects application instrumentation will continue to be offered to Windows Phone 7 developers at no cost through December 31, 2012.

Mobile analytics endpoint (wp7.runtimeintelligence.com) will be shut-off on 12/9/2011

The current analytics endpoint will be discontinued. However, developers have a number of options that they can consider;

· Subscribe to the PreEmptive Solutions commercial endpoint. This is a fee-based option that includes all of the features currently offered PLUS an advanced mobile portal, a RESTful API, and a higher service level (plus support beyond WP7). For more information, email sales@preemptive.com.

· License PreEmptive Analytics for TFS. This is also a fee-based option. This solution is an on-premises solution focused on exceptions rather than feature tracking. For more information, see Using Analytics for Windows Phone and Azure Exception Tracking - User Community Virtual Series and email sales@preemptive.com.

· Develop and host a homegrown endpoint. This is a no-fee option but development will be required. The CodePlex Runtime Intelligence Endpoint Starter Kit repository starter kit project may be of some help.

· Plan to migrate to the PreEmptive Analytics for TFS community edition to be included with Dev-11. This is a no-fee option but is NOT yet generally available from Microsoft. For more information, see the video A Lap Around PreEmptive Analytics for TFS with Justin Marks .

· Publish their app as a CodePlex project and utilize the CodePlex analytics endpoint (this is different than the option above). This is a no-fee option. For more information, see this tutorial (note that this assumes the developer is limited to the Community Edition of Dotfuscator – but the WP7 edition has full functionality).

The following feature summary table highlights the three principle options available to the Windows Phone 7 development community with a comparison to the discontinued Microsoft sponsored service. (click thumbnail to enlarge)

Click to enlarge

Click to enlarge

FAQ

Q: Will developers have to republish my WP7 app on or before December 9, 2011?

No

Q: Will users notice any difference in app behavior after December 9, 2011?

No

Q: Will I have to re-register my installation of Dotfuscator for Windows Phone 7?

No

Q: If I have only been using Dotfuscator for obfuscation, will I lose any functionality or will I have to do anything differently?

No

Q: Will developers have access to earlier runtime data generated by Runtime Intelligence for Windows Phone after December 9, 2011?

They will not.

Q: Where can developers ask additional questions regarding migration, upgrades or discontinuing use of PreEmptive Solutions technologies?

Post to the PreEmptive forum at http://www.preemptive.com/forum/index.php?f=26&sid=dfe90c2ba80de07692372dae962c58b2&rb_v=viewforum

PLEASE NOTE – this is NOT a moderated forum.

Q: Why would a development organization upgrade to a professional SKU of PreEmptive Analytics?

· Multi-platform (WP7, Android, JavaScript, all .NET and Java, native API…)

· Private endpoint (for large-scale enterprises with demanding scalability, governance, or other unique requirements).

· Analytics for TFS option (out-of-the-box integration with Microsoft Team Foundation Server).

· True application analytics such as custom data fields, development ownership of data, true SLA and support for developers, etc.

Conclusion

Of course we would have preferred that Microsoft had opted to extend their sponsorship for Runtime Intelligence for Windows Phone, but that was not the decision that they ultimately made. However, over the past 12 months, we had a front row seat watching a flood of innovative apps launch. We know that a good percentage of the WP7 development community relied upon PreEmptive Solutions for both analytics and protection (in a recent analysis of the marketplace, it was shown that 17% of all apps used either protection, analytics or both).

This experience combined with our confidence in the future of the Windows Phone platform has prompted us to extend free access to Dotfuscator for Windows Phone. We look forward to our continued support and participation in the growth and success of this exciting technology and marketplace.

Posted in Dotfuscator, Runtime Intelligence | No Comments »

60 days – déjà vu all over again?

October 10th, 2011 by Sebastian Holst

Hi all – today I sent out a notice to registered Runtime Intelligence for Windows Phone users letting them know that a change would be coming to our service on (or perhaps after) December 9th. I have already received messages from some rather annoyed developers who feel like we are playing some kind of Machiavellian pricing game; the basic flaw in this view is that it presumes we sit in the Prince’s chair – which we do not.

First, let me start by stating categorically that we remain excited and committed to the Windows Phone platform.

Our approach to analytics has always been squarely focused on development organizations rather than on marketing; this is not a “spin,” rather, this approach informs and influences our product’s architecture and business model.

I’ll drill into this distinction in a moment, but first, I want to point out that it is this distinction coupled with our belief in WP7 as a platform and the value of our approach to application analytics in general that led us to invest in building out the WP7-specific service. While I cannot speak to Microsoft’s motivations (in any context, let alone this one), it is public knowledge that they funded this service so that it could be offered at no charge to the WP7 dev community (but now that’s no longer the case).

Why was this even necessary? Why couldn’t PreEmptive just give away our analytics and protection service like Google does?

“Marketing-centric” analytic services make their money off of your data – but first they have to cover their costs; the “back end” or analytics portal is where the majority of that “cost” sits. Today, Google or Flurry (or any other similar service) each rely upon web and other mobile platforms to generate the data volume that ultimately funds their backend service. At this early stage, WP7 app data can fund client API’s (perhaps) but is nowhere near the volume-levels required to pay for an entire service.

But there’s more – focusing exclusively on resalable content (and by extension, divesting from any requirements that do not) have profound feature and architecture implications.

Given a marketing-centric focus, you can readily understand why these solutions all share the following traits:

· Custom data fields and data types are limited in scope and volume(unique/heterogeneous data points cannot be easily rolled-up, sold or used to better target users)

· No built-in enforcement for opt-out policies – this is left entirely to the developer (if you do not provide user data to these services, you are of no value)

· No investment in analytics of software away from the presentation layer(this data has no relevance to advertising, profiling etc.)

· Software-specific data is only marginally supported – unhandled exceptions can be captured, caught and thrown exceptions are out of scope *same as above – does not contribute to monetization model.

· They own your data (privacy rules notwithstanding – the have full rights to monetize your data in any way they see fit). This is the entire reason they are in business – if they don’t own the data, how can they make money?

· They can provide their analytics software for free – since they have built a monetization strategy based on your data and do not invest in any development that does not directly contribute to that strategy, they can afford to give you the picks and shovels for free in exchange for the gold you produce for them (that’s big of them don’t you think?)

Conversely, focusing on application stakeholders as we do, we invest in features that do not contribute to a roll-up across companies and apps to monetize user preferences – in fact, some of these features can materially impede that objective – yet they are equally as important for developers who want to build better, faster and more effective software…

For example:

· Supporting custom data (app-specific fields and states) and data types (other than strings)

· Built-in exception tracking (unhandled, caught and thrown) to not only send you stack traces but provide insight into user experience and runtime environments (hostile or otherwise)

· Built-in opt-out policy enforcement (local, regional, cultural, legal and regulatory rules are so complex – serious development organizations need to centrally manage and reliably enforce compliance – could you sell a gun without a safety?!)

· Analytics away from the presentation layer (want to know how your Azure or other distributed services are performing and behaving?)

· AND LAST BUT NOT LEAST, you own your data. (given the heterogeneity and specificity of each application’s data – it’s value is exponentially higher to development stakeholders and proportionately lower in aggregate)

So what’s the takeaway?

With Microsoft, we invested in this mobile platform because we believed (and now we know) that it has significant value.

Given our focus on development rather than advertising, we have to innovate on both the technical and business ends.

We had hoped that our previous relationship with MSFT Windows Phone division had lasted a bit longer, but it is what it is.

We have a number of very interesting scenarios in mind that we think will prove to be both exciting technically and innovative from a business perspective – but we are not ready to discuss that yet (as soon as we can – we will).

If you don’t care about these added value features and don’t mind the strings that come with web/mobile analytics services like Google – then you should certainly use them. They are not flawed – they are just fundamentally different.

If you value the services we have been offering or have ideas on how we can improve them even further – PLEASE LET US KNOW…

Registered users will be getting a survey in the next 24-48 hours – please let us know what you think and what’s working (and not) for you.

As I’ve already said, we remain excited and committed to the Windows Phone platform.

(now i have to break to prepare my wp7 training materials; i’ve got a crew from my son’s high school writing WP7 apps for independent study credit)

Posted in Runtime Intelligence | No Comments »

Runtime Intelligence and Dotfuscator for WP7 developers speak (Mikey likes it!)

September 26th, 2011 by Sebastian Holst

In my last post, I drew a correlation between apps that used Runtime Intelligence and their relative (positive) success as measured by user ratings and engagement. While it was fairly clear that developers who chose to use Runtime Intelligence built more successful apps than their counterparts, it really said nothing about a) Runtime Intelligence analytics’ contribution to their success or b) developer satisfaction with Runtime Intelligence overall.

Well, there’s really only one way to answer these questions …and that’s to ASK THEM.

I sent out an electronic survey starting on Monday of last week (September 19) and have received over 200 developer responses. Here is what they said…

Who is doing what?

  • 32% indicated that they are only using Dotfuscator to protect their application
  • 24.5% said they were only using Runtime Intelligence
  • And 43.4% indicated that they were using both.

Do these “smarter than the average bear” developers see the value?

In a word, YES.

Looking only at those developers who indicated that they already had their applications in the marketplace (representing over 100 development organizations):

  • 60% indicated that analytics set the wp7 platform apart from all other platforms or added significant value to the platform.
  • 68% indicated that protection set the wp7 platform apart from all other platforms or added significant value to the platform.

Analytics’ perceived value increases by 450% with developer experience

When looking at those developers that indicated that analytics and/or protection “set the WP7 development platform apart from all others,” analytics’ value actually increased by 450% (from 2% to 9%) as developers moved from no app, to less than four weeks to a ship date, to actually having an app in the marketplace (and getting analytics back). Interestingly, obfuscation (protection) peaked in value just prior to shipping.

So what’s the takeaway?

In my last post, we established the user of Runtime Intelligence were more successful than other WP7 developers. In this post, we see that these developers credit their success, to some material degree, to either Runtime Intelligence or Dotfuscator protection (or both).

In their own words
(selected - but unedited - responses to the open-ended question “what are you most excited about?”)

I like being able to get crash reports without much additional work.

It gives the developer ability to know about usage patterns in an application. Obviously code obfuscation is a necessity, especially for paid apps.

It offers a unique way to see how users interact with the application, and with the latest release it also has error reporting. Awesome!

I’m an excitable person.

Fabulous data provided by RIS to analyze the performance, usage and app demographics.

So I can know what is happening in my app and protect my code.

Used correctly, the analytics really let me see how and by whom my application is being used. I get more insight into this information than I could if I set up a usability lab or just did extensive user testing. There is no better way to observe than to do so in production.

The concept of attaching runtime analytics after the compilation process is very useful for us (standard software development, single application in various customer-specific configurations), since we are able to attach this on a per-customer basis and don’t have to manage it in code.

UI for parameterization

It gives detailed statistics about the usage of all parts of the software and helps to recognize the hot features of the software are and which parts are less used. This adds great value into the effort of making software better.

Really gives me insight into what my customers are doing with my applications. They help me to understand where I can enhance functionality and add value.

Quality of product

Analytics give me an idea on what I should work on next to improve my application

Runtime analytics is cool because there is no code to write.

I can collect the exact information i need.

It allows me to phase out or strengthen certain parts of my apps. I currently have seven apps and the instrumentation is crucial.

Because i can have a deep analysis of when and especially how my application is being used. If you add the fact that all these data are aggregated and presented in such a nice way by the portal, you end up with a great product

I produce libraries (DLL’s) that are handed to third parties, hence the need for obfuscation.

Kickass obfuscator.

It helps me keep track of any bugs. And it allows feature tracking. And it gives me the cool world map that shows where some of the users are.

Posted in Dotfuscator, Runtime Intelligence | No Comments »

WP7 apps with Runtime Intelligence have higher ratings and engagement rates

September 26th, 2011 by Sebastian Holst

In my last post, I lamented that only %2.5 of the apps in the marketplace were actively using Runtime Intelligence (RI). This begged an obvious question; were these developers leaders or laggards? Were RI-enabled apps more successful and effective and, by extension, worth emulating?

The short answer appears to be yes – to be clear, I am NOT saying that merely turning Runtime Intelligence on increases an app’s success. What I think the following data does show is that the developers who chose to include Runtime Intelligence as a component of their development process are indeed more successful than those that did not.

I took a second look at the 26,469 apps that we downloaded from the marketplace and compared those apps that were instrumented with Runtime Intelligence with those that were not.

Specifically, apps with Runtime Intelligence (analytics) are:

Ranked 25% higher: RI-enabled apps averaged roughly 4 stars (8.02 out of 10). Non-RI apps came in at 3 ½ stars (7.6 out of 10). NB: apps with no ratings whatsoever were excluded from this calculation.

Have a 50% higher rate of engagement: 73% of the apps using RI had at least one rating while only 48% of non-RI apps had any rating whatsoever.

As I’ve already said, simply turning on RI will not get you a 1/2 star bump in your ratings - but clearly, the kinds of developers who are achieving higher user satisfaction are the kinds of developers who are choosing to use RI.

Coming next: Users of RI and Dotfuscator for WP7 make themselves heard!

Posted in Runtime Intelligence | No Comments »

WP7 Marketplace share (or how we became a victim of our own success)

September 22nd, 2011 by Sebastian Holst
This post tells the story of how good faith estimates of our WP7 marketplace penetration were under-reported by 500%. This is not a “gotcha blog” – there are only good actors with the best of intentions in this story; but that’s why I think it’s a story worth telling.

You see, we don’t just obfuscate – we hide the fact that your app is obfuscated. We don’t just offer application instrumentation and monitoring, we inject that logic to simplify and streamline packaging and improve performance. …and therein lies the rub.

Something was in the air

While at Build soaking up the heady atmosphere of Windows 8 and all that’s coming with it, a few MVP’s and also the Lowdermilk brothers took a moment to ask me if I’d read this awesome blog post where a developer had downloaded and analyzed all of the XAPs in WP7 marketplace. Apparently the market insight was killer and covered everything from the most popular libraries to snagging cloned apps. No more guessing – the facts were all laid out. And they each mentioned to me how surprised and even a little worried they were by how few apps were being obfuscated. I didn’t think too much of it at the time as I am among the first to point out that not every app needs to be obfuscated – but I did make a mental note to be sure to check out the blog.

When I got back home, I finally had a chance to track this “all seeing blog” down – it was Justin Angel’s blog and the post was Windows Phone 7 Marketplace Statistics. And it really is a fascinating post with both initiative and insight; and then I got to the obfuscation section.

According to Justin’s analysis, only 3% of the apps in the market were obfuscated! And as I scanned down, there were comments to the effect of “gee, since no one else is doing it, perhaps I shouldn’t bother either.”

Even more surprising to me was the fact that our analytics (Runtime Intelligence) was not even listed in a very long list of third party tools – when I knew for a fact that we have nearly a thousand apps sending data.

This can’t be right! (and I was)

Given the nearly 6,000 downloads of Runtime Intelligence and Dotfuscator for WP7 and the activity that I had been seeing over the past year, these numbers just didn’t seem right. I wrote to Justin who was quick to share his detection logic (in fact he posted the source on his blog) and just as quick to invite any comments or refinements that I might have to offer.

To put a fine point on this, Justin was in no way defensive and was as interested in getting to the right answer as I was.

Hung by our own petard


Without going into a lot of detail, Justin’s approach was to bust open the XAP and examine the various files and manifests to separate Silverlight from XNA and to identify the presence of third party tools. This approach proved to be effective because frameworks, tools and components leave behind files and other telltale fingerprints as a matter of course. There is one limitation though; this approach cannot detect when an application is modified or extended through IL manipulation or injection. And that’s exactly what we do.
From Build to Bill’ed (new numbers and how)

As I like to say, my ideas only have to be good, they don’t have to be original (trademark and patent laws not withstanding) and so I did what I often do when confronted with a conundrum – I asked Bill Leach, our CTO, for help. He quickly (dare I say magically?) authored our own “marketplace crawler” that populated our own XAP repository. Rather than look at XAP contents at a component level, he wrote some code that examined the binaries themselves.
The first pass looked for the custom attribute DotfuscatorAttribute inside the binaries. This is a good way (but not an absolute way) to determine if a file has been processed by Dotfuscator (for either obfuscation or injection of analytics). It’s not infallible because developers can remove that attribute if they chose (to further cloak the fact that they have used Dotfuscator). Here is what we found:
We downloaded 26,159 XAP files and 14.5% to have been processed by Dotfuscator.
This is basically 5X as many apps as Justin’s analysis had found (and that does not include the developers who configured Dotfuscator to remove the attribute we were searching for – so the number is certainly a bit higher).
In fact, we were surprised that Justin had found any at all – where did his 3% come from? Upon inspection, we think it’s an unexpected side-effect of how XAP’s are assembled – there are some instances where the configuration file of Dotfuscator gets pulled into the XAP – this is unnecessary and should never happen. We will document this behavior and make sure that users know how to prevent this from happening. In short, his 3% showed the prevalence of a bug – not the use of Dotfuscator.
To determine if an application was instrumented (rather than obfuscated), we applied some heuristics that are less obvious but can be shared if someone is interested (we looked for the existence of some high-level classes).

2.6% of marketplace apps are instrumented.
From my perspective, this is a low number – but to put it in perspective (or let’s be honest – I’m looking for the silver lining) we have a larger share than Google Analytics and Admob but a slightly lower number than Flurry.
Attack of the clones
Just one more point to be made in this post. If one were to consider each family of cloned apps as essentially a single, re-skinned app – these numbers have the potential to change materially. We may take a look at that, but I think we have already gotten most of what we can from the static analysis of the marketplace.

So – is that the whole story? (Of course not)

Don’t give me (just) static

As interesting as the static analysis of the WP7 marketplace is (and it is), static analysis only gives us a backwards facing snapshot of what’s already been deployed. We get no insight into:

  • Best practices that we would want to replicate (which are different that common practices),
  • Developer motivations behind their development choices
  • Future trends especially when driven by new technology and market opportunities
In the context of Dotfuscator and Runtime Intelligence, I would want to know
  • If the developers who built the 14.5% of “Dotfuscator processed” apps leaders or laggards?
  • Do they have special requirements that set them apart?
  • In short, do they have anything to teach the rest of us?
Want to know more about the 14.5% apps and what they have to teach us?

Coming soon, WP7 Developer Survey Results. We’ve been running surveys since the WP7 launch last year (you can checkout survey 1 and survey 2). As part of this ongoing effort, we have just closed out our third survey in the series and I will be posting results in the next few days – stay tuned!

Posted in Runtime Intelligence | No Comments »

« Older Entries