Thursday, January 31, 2008

So, you want to deploy a J2ME app in the US?

We are about to launch a new mobile service in the US. Sounds simple, doesn't it? Well, it's not...

You have two ways of deploying a mobile service in the US, and for that matter anywhere in the world. One is to go hand in hand with the mobile operators and launch your application on their deck. You can gain marketing cooperation that way, and the fact that you're on their portal, guarantees a fair share of success.

Why not go that way? Well, for starters the deal cycle can take about 12-18 months. And not only that, you have to convince the operator that it is even worth talking to you... And while I hate to admit it, it's quite reasonable... Operators (or carriers as they are called in the US) get a huge amount of noise from companies wanting to launch their services. Content managers there have more hits than Google has... So, even if you're a start-up with a brilliant solution, they will have a hard time getting you in unless you have some proof of getting revenues or an impressive userbase or an impressive brand etc.

Oh, and there's also the fact that they will take about 50% of your revenues...

The other option, which is taken by most consumer-oriented start-ups when they begin is going off-deck without any business relationship with the opertaor. This is very suitable for free mobile 2.0 products that operators don't like very much (50% of 0 is...), especially if those can be used as a substitute for messaging and voice. However the operator gets data traffic (And the application provider can't touch it), so you would think that in the big picture, operators would like to increase data traffic as they always state.

However, in the reality of things today, application providers find along the way a lot of barriers when they try to deploy their apps off-deck, and the situation in the US is much worse than Europe in which operators already understand that they should encourage data usage.

Anyway, there is also a big difference between the various operators in the US, and during the last few weeks I have come to learn the nuances both via extensive testing we've done on handsets in US networks (Using Mobile Complete's Device Anywhere) and the help of my colleagues at Mobile Monday Silicon Valley.

And here are the results:...

There are 4 big operators in the US: AT&T (which purchased Cingular), Verzion, Sprint-Nextel and T-Mobile. There are also smaller and regional ones, that we'll elegantly ignore for simplicity (I want to get this post done by 2010...)


Verizon

Our application is in J2ME, which is de-facto standard in Europe and also other parts of the world. J2ME is also gaining dominance in the US, however one (big) operator doesn't support it (yet) - and that's Verizon.

So Verizon was out of the picture from the start for that phase, and this is how a lot of start-ups start: First a J2ME version that can cover Europe and about 65-70% of the US. BTW - The Java "version" will have to be ported to about every device in those markets (read my post on that), but Brew is a complete rewrite, so the effort is bigger.

In any case, though we didn't test on Verizon, it is about the most closed environment, so much that you can forget about releasing something off-deck.

T-Mobile

From some reason I had the notion that T-Mobile would be a very open network. Maybe it is because I know it from Europe, maybe it's because it was an early adopter for J2ME and maybe it's because it was rather open up until a few years ago. However, today the situation is quite the opposite.

Users are able to download J2ME apps (JAD+JAR), so if you have an offline app, like a standalone mobile game, you should be fine. But if your app requires an internet connection - you'll be surprised to find out that that is not trivial at all. In the bottom line users will be able to access the internet if:
1) The application was signed with a T-Mobile certificate, or
2) The user has a $20 "total internet" plan instead of the regular $6 T-Zones one, or
3) The handset was not bought through T-Mobile.

Now, as you can understand (2) and (3) are major problems, since the majority of users do not have a $20 data plan and buy their handset from their operator, so you can't rely on those.

As for (1), in order to get a certificate from T-Mobile, you have to have some sort of a commercial agreement with them, which brings us back to plan A (on-deck) that can take 12-18 months as explained above.

Now you might have heard of 3rd party certificates (Verisign, the Java Verified program) - those will not do you any good here. T-Mobile alters the Java security domains and permissions on the handsets it sells in such a way that only a certificate by T-Mobile is considered to be trusted, and any thing that is not trusted does not have access at all to the network as well as to other APIs (SMS, Bluetooth etc - see the exact list here).

This is why users who bought their handsets through other channels have devices that are less restricted and do not block any API even in the untrusted domain, but rather prompt the user whenever there's an attempt to access resources (BTW - on the network side, these users may still have to configure a proxy to use the $6 plan, which is not trivial at all, but at least these menus are not blocked like they are on handsets sold by TMO...)

In short, if you are aiming for a mass-market consumer application, and not just one targeted at business users or tech-savvy users, your hands are pretty much tied... The only way to acheive that goal is to go on-deck with T-Mobile.


AT&T

AT&T appears to be (at least now, these things tend to change) a little less strict than T-Mobile. You can download J2ME applications from anywhere, and they have network access (HTTP), which is what we needed. However, it seems that although we didn't bump into a wall with AT&T, other applications that want to use socket communication, access to the file system, address book and messaging (SMS/MMS) will probably be blocked (See Hartti Suomela Forum Nokia Blog and the exact domains/permissions in AT&T. Hartti has done some research on the subject and answered my nags on Mobile Monday... Thanks Hartti...)

The blockng of those resources/API is done on the same way as with T-Mobile, by altering security domains on handsets sold by the operator. And again even if the application is signed by a third party, it won't make much difference, but at least in AT&T in this case SMS will be allowed in addition to network access.

So it seems that for unsigned applications "just" HTTP network access is granted compared to T-Mobile, but this is a huge difference, since most online applications need just that (sockets communication is less popular also because some handsets don't support it as well as other networks in the world). For example, GMail mobile, Google Maps and Opera Mini will run on AT&T but not on T-Mobile.

However, more advanced mobile 2.0 applications that interact with the content on the handset itself (address book, pictures, other files) will have a hard time running on both networks (hard time=they wouldn't run at all...)

And one more note about unsigned applications (which are considered "untrusted"), even if the operator allows them to install and access HTTP like in AT&T, the handset still gives the user several security alerts, but that's common procedure in J2ME.

Note that those alerts are sometimes quite nasty (For example, in Nokia's newest devices the message can be "This application may cause damage to your handset"...) so you might consider getting a certification from a third party such as Verisign.


Sprint-Nextel

Even though Sprint purchased nextel some time ago, there are still 2 seperate networks with different technologies. They may both be marketed under the Sprint brand, but since there are differences in their policy, I'll address them separately:

Sprint (CMDA)

Sprint is a CDMA network, which usually doesn't combine with J2ME. Traditionally GSM networks supported J2ME and CDMA supported Brew. However, you can understand that this doesn't have to be the situation since J2ME/Brew are high-level technologies that can be implemented over several OSs, while GSM/CDMA are the low-level transmission methods. It's like saying that Java works only on ADSL while .NET works only on WiFi....

The reason for this traditional coupling may be related to commercial aspects or simply to the fact that once handset vendors made specific devices for specific networks it became a standard in those networks.

In any case, while Sprint is a CDMA network it does support a wide variety of J2ME enabled devices, and as expected it has similar issues like its GSM "sisters". Again with the security domains and permissions alteration, but closer to AT&T: you don't have to sign your apps for network access, but other APIs are restricted and can only be used with a Sprint certificate (see the full details here).

Sprint does make it easier for developers and allow them to obtain a developers certificate so they can test their apps before even speaking with Sprint. This certificate is available at the custom WTK (Wireless Toolkit) that is available at Sprint's developer portal (This certificate is only good for testing). In addition I was told that it is much easier to obtain their production certificate, and it doesn't necessarily involve a marketing cooperation, so this process may be quicker.

And one more technical tip you should know: some of Sprint's devices are more sensitive when it comes to JAD definitions. At first we thought that the download itself is being blocked, but it's not. If you can't download an app, go over the JAD definitions and check the MIME type your server send JAD files as.

Nextel (iDEN)

The iDEN network in Sprint-Nextel (which I'll refer to as Nextel, since it was just that originally) is even trickier, but not only because of the operator's restrictions, but rather due to the technology itself.

iDEN handsets do not allow users to download J2ME apps unless it is from the carrier's portal (which means you have to get on-deck). However, you can download the application to your computer and from there transfer it to the handset using a data cable and a tool called openJAL. This is of course not an optional delivery for the mass market, as just tech-savvy users (or geeks as they are more commonly referred to...) can do that.

Note that this is even before we got to the permissions "bridge" - no application even if it's a standalone offline application can be downloaded over the air if it's not on the operator's portal/deck, and in that perspective it's the worst restriction set, but again it has also a lot to do with the technology itself.

So again, you are left with the choice of either getting a deal with the operator or deploying your app to users that can hack their way through (That would be me and some of the guys in the office here... definitely not a very promising demographics...)

Summary

So that's about it. As I said there are other operators in the states but these are the big four. As you can see the current state makes it difficult to deploy applications off the deck, and that's slowing down the industry in my opinion.

The operators themselves claim that the restrictions are for security reasons, and I can understand that partly, since no carrier wants its users to download apps that access their private data or sends messages/connects the internet without permission. But isn't the customer smart enough? I mean what's wrong with the normal security domains in which untrusted (unsigned) applications can access resources only after a system prompt to the user? These messages are scary enough that a user that doesn't understand will click "No", but those who do, can enjoy great apps, and the application providers can enjoy a growth in their revenues, as well as the operators in the end.

So get moving guys and unlock your networks!

20 comments:

Anonymous said...

a great analysis of the US carriers' apps market. Thank you!

yael

Anonymous said...

great analysis. Thanks for sharing this with the developers community and savine many of us lots of time trying to do the impossible.

I only hope the US carriers will quickly realize that it is a mutual interest to open their walled gardens to benefit both users and to increase the overall pie to be shared between carriers and ISVs

Anonymous said...

Great article - interesting to see the US being left in the dark ages. Looks like the next wave of internet entrepreneuralship will occur in Europe and the East with 'the rise and fall of the US empire' already on it's downward slope.

Anonymous said...

I would add that you probably misinterpret the motivation for the operators in the US. they're not really all that intent on driving higher data usage. They're more intent on driving people to pay for specific services. So, say the got someone to pay for an all-you-can-eat data plan: they'd rather you paid for it and didn't use it than pay for it and have you use it (requiring them to up the hardware requirement). Better yet, they'd rather you pay for it individually as part of application access, rather than as a bundled service. They don't want you to treat them as an ISP.

Anonymous said...

Great discussion and real one for US market.

For both desktop and mobile version of opera to succeed in USA environment/corporate culture /user level, opera need to cut a deal with big companies. No matter what it takes, how long it takes but opera has to do it. there is no other way. Quality does not always rule in USA.

Ofir Leitner said...

Thanks guys for the feedback. Here are my thoughts on the comments so far:

1. The US carriers are behind of their European counterparts, but bear in mind that european carriers also have their own quirks, and while they are more open than the US ones, they are still not the most "open-sourced" environment. For example, take a look at my post on WiFi calls which details how operators in the UK also altered handsets software to their benefit. In fact, the whole walled-garden/data blocking was very common in Europe a few years back. So I wouldn't say that the "US empire" is "falling", it's just that the carriers market there is a few years behind - but eventually it will catch up.

2. As for the motivation of operators in the US - I know that currently not all of them think in the mindset of how to increase data usage - but that's my point exactly: If you sell an "unlimited" dataplan for $6 and the user finds out he can't use it for most of his apps - users are going to disconnect, or even switch networks. So in the long run it is beneficial for the operator to drive not only data revenue but also data usage. Now maybe some carriers still don't think that, but their mindset is bound to change in the next couple of years.

Anonymous said...

Good stuff - thanks. But what about mobile internet/wap sites? What are the quirks and hurdles that US Operators can put us through even using only WAP?

ps I run flirtomatic a successful UK based flirting service on both mobile and web....

Ofir Leitner said...

Thanks Mark. As for WAP, I didn't get the chance to experience the quirks first hand, so I can't write an in-depth analysis, but I am sure that in the near future I'll get there too...

However, we did need to download the JARs we tried from simple WAP pages, and this seemed to be okay with all carriers we tested (AT&T, T-Mobile and Sprint), but again these do not encompass the whole array of things you can do with WAP.

One other subject that I am going to write about soon is the SMS regulations in the US, as this is something I've experienced personally.

Anonymous said...

Very enlightening. Well done. Thank you.

Ron Edwards said...

Great post!

Anonymous said...

the BREW/CDMA connection is because BREW is a Qualcomm product, and they make most (all?) CDMA chipsets.

Anonymous said...

Any insight along the same lines for the Indian market? I assume the economics of selling an app might be diffferent, but I understand the market there is potentially HUGE!

Anonymous said...

WAP or XHTML mobile profile sites are also subject to limitations, lesser functionality enabled and other quirks. For instance you get warned that this is an 'untrusted domain' if you haven't been whitelisted, which scares people. You cannot enable downloads to the phone (save/use this image/sound file etc. simply doesn't work) so people can view the image, but not make it their wallpaper and many other wonderful things that seem strange limitations.
But things are changing fast, and the promised opening of networks is already starting to have its effect.

Anonymous said...

Ofir,

Well done. That is a good read.

Do you have much experience with the operators in France ? I believe the 3 main ops there are SFR, Orange and Bouygues. Are they walled-gardens as well ?

Gavin

Alan said...

The point about:

2) The user has a $20 "total internet" plan instead of the regular $6 T-Zones one

is not correct. If the user has purchased the phone from T-Mobile, even upgrading to the $20 "total internet" plan does not fix the problem with unsigned apps not being able to access the Internet.

More here:
http://asteele.net/post/5597866/about-that-last-twitter

Ofir Leitner said...

grok2 - I am not familiar with the Indian operators's policies. It would be interesting to see whether their policies resemble Europe, the US or the fareast. If anyone knows, please post a comment here.

Gavin - As for operators in France and in Europe in general, they are usually more relaxed than their US counterparts. It doesn't mean they're completely open, but without checking the specifics in these operators, I would say they are all better than the ones I mentioned...

Alan - sorry to here that... It appears the situation is even worse.. However, my guess is that you have a phone bought from T-mobile, hence they messed your Java security domains... Trying an open phone (bought not from them) with the unlimited plan should allow you to access the net from Java.

al said...

Great article - just want to mention that with an unlocked Nokia N73, I can get HTTP access with J2ME apps with the TMobile $6 plan. I don't know if other phones work though.

Anonymous said...

I've had conversations with hi-level T-Mobile techs about this very topic and T-Mobile signing midlets. They said they really don't support that activity anymore and this site, http://j2medeveloper.t-mobile.com/devportal/index.jsp, should be taken down.

Anonymous said...

How about J2ME games on "Carrier locked" devices?

I recently downloaded 3rd party java games to ATT LG VU but interesting enuff the phone would not even display the jar files leave alone running it.

I could see the files when the I attempt to move files under that folder so they are there you know.

Mr. Reliable said...

T-Mobile now has their more open partner network program available http://developer.t-mobile.com. It has a 70% revenue share to developers.