Are transcoding proxies needed on the mobile?

Most users have never used the mobile web, in some cases they have downloaded a few ringtones, a wallpaper or a java game. The operator portals are the still the first and probably only site that the average user has ever visited. Sometimes some brave individual looks for adventures and surpasses the borders of his operator portal. These users either know exactly what they want and know the URL or don’t have any idea of what they can find. Users who don’t know what they can find will go to some search engine like Google or Yahoo! (hardly the mobile versions) and type a keyword such as “tennis”. They will get a list of links and follow them. There are very good chances that they will reach a web site that doesn’t even take into consideration the possibility that a user with a mobile phone will visit and is not doing any device detection or content adaptation.
If this is case, and unfortunately still is for most sites, there are two chances, either the user has a really smart browser like Opera Mini or Nokia’s webKit-based browser, or he will download a huge page that he will not be able to read. After 2 or 3 different sites have been tested, users will be discouraged and will not take the risk of going out of the operator’s portal ever again.

Operators should be happy about this and try to disincentivate browsing outside of the portal. On the other hand, browsing means traffic, traffic means money, so accessing the “full web” can be a positive thing for mobile operators too. If browsing can be a revenue stream you don’t want your users to be discouraged, you want them to have a taste of what they can get on their mobiles.
Opera Mini is one possible solution, transcoding is the other. Google offers transcoding automatically, operators might choose to get their own transcoding software. The result is that users get what they want (the tennis site), operators get what they want, money from data traffic. Looks like a win-win solution.

Within some limits, this is effectively a win-win solution. The limits are defined by the user experience that the transcoding engine can provide. It is proven that transcoding a web page and fitting it into the small screen of a mobile device will not provide the best user experience, nevertheless it is still better than a kick in the teeth (like the average rendering of a full web site on a Motorola V3).
Transcoding is like a shortcut to get more users on the mobile web. Major web sites should consider developing a mobile version quickly simply because mobile is the future. At the same time there is a number of sites that will never be converted (think of all the geocities pages!) and there are many minor sites that will consider mobile only in 3 or 5 years from now. We do want users to be able to find information on the mobile today, don’t we?
Transcoding engines must be smart enough to recognize sites that are made for the mobiles and leave the content as is and convert the content only when this will not fit the features of the mobile device (or set-top-box). Transcoding engines should be smart and act when needed and never get in the way, they must be an extra tool, not a mousetrap that gets in the way of the developers and designers that are sweating to produce content that is optimized for mobiles. Transcoding the content of a site that was made for mobiles will most likely break the usability and optimizations that the designer put in place.

At the same time, users should be educated that there is a lot of information for them on the internet and that they should be looking and asking for mobile versions. We should not expect the average user to understand mark-ups, XHTML-MP [PDF], transcoding, image rescaling, but they do understand when a page is not usable on their mobile and if they can’t find anything interesting, they won’t come back.

There isn’t just ONE WebKit

Reporting from Surfin’ Safari Blog:

[W]e have a WebKit detection script that properly checks for the WebKit engine (not just Safari) and properly detects versions.

This script has now been updated to support iPhone and the new iPod touch. You can try a live version to see what kinds of results you’ll get on different browsers and devices.

For all those that believed that mobile devices are the same as desktop PC’s.

Read the full article here: WebKit Detect script updated for iPhone and iPod touch.

Is it good to hide the user-agent in mobile?

Recently Vodafone plugged a new infrastructure in their mobile network. In two words, it is a proxy that is also capable of transcoding web page contents to be suitable for mobile devices.
While I don’t know the marketing plan behind this, it is obvious that the main driver is to make the entire web available on mobile devices, even if the source content was not designed for use on mobile devices. This is good, of course, because it allows all Vodafone (UK-only at this time) customers to access all the valuable information that you can find on the web normally, all that information you’re used to (and I’m addicted to, I’d say). But how is this software working? It is a proxy sitting sitting in the middle between the mobile device and the open internet, all contents requested by Vodafone-UK users are received by the proxy that performs the requests, gets the page, elaborates it (or maybe simplifies it) and delivers a new page to the mobile browser. While processing the page, the proxy takes out all contents that might create a problem to the mobile browser, tries to clean up the markup if there’s any tagsoup and sends something that is optimized for the mobile.

All this sounds perfect, doesn’t it? Well, not to me.
There are a few glitches here and there that I think are breaking the original plan of improving the user experience. First of all the proxy sends a unique user-agent HTTP header string that hides the original device behind a generic Mozilla-compatible browser. Remote servers will think that the requesting browser is a desktop PC and send the “appropriate” content, unfortunately sometimes even the remote applications have an adaptation engine that will provide content for desktop PC that is not meant for the mobile. Let’s think of a company selling ringtones and contents for mobile phones, when visited by a mobile device will let the user pick a content and download it directly, but if a desktop PC is recognized, it will ask for the mobile device model, show the available contents, ask for the phone number and then send a wappush to let the user download the content. If the proxy is hiding the mobile device and pretending it’s a desktop PC the original plan is failing.
The answer to this would be to have a protocol that describes how the remote web application could tell the Vodafone proxy that it IS capable of providing appropriate content for mobile and stop the adaptation, unfortunately, such a protocol does not exist, as of today.
Is the user-agent SO important? At this time IT IS so important, there isn’t in fact, a technology that lets a web server know the full capabilities of a mobile device. The OMA and the W3C are both working on this topic, but the technology will not be ready before 1 year or so and then we will have to wait for it to be implemented in mobile devices and later to become a common feature: you can easily imagine the timeframe. The solution proposed by Vodafone is that sites that are able to provide mobile content should contact them and provide all the URL’s. Sorry, but this doesn’t seem very scalable.

Another tiny issue is that the above plan makes most sense if you’re thinking of old WML browser that would not be able to parse an HTML or XHTML page, but what about the recent WebKit implementation that Nokia is delivering pre-installed in their Symbian phones? What about Safari on the iPhone? What about Opera Mini (that I have to admit is a quantum leap in browsing as it provides very smart browsing features to all J2ME devices)? All these browsers provide TODAY (and even 1 year ago if you think of Opera and the first devices running the WebKit-based browser) a technology to adapt content that was meant for desktop PC’s with big screens to be viewed and used decently on a mobile device. Unfortunately, again, there isn’t a technology that will tell the Vodafone proxy if the device is running a smart browser…. Oh, wait… The user-agent string DOES tell it. So let me recap: Vodafone reads your browser’s user-agent string, tries to understand how smart your mobile browser is and then hides the very same information to the remote server! Is this a dog biting its own tail?
But, isn’t Opera Mini doing the same, using a unique user-agent for all mobile devices? Yes of course, but the difference is that Opera mini is installed by the user, the user knows about it and also has a chance to set a few options to make the software behave as she better likes it and, by the way, if she doesn’t like it, she can use the native browser or install another one, if available. Vodafone’s proxy is doing this without asking anything to the user and without allowing any personalization and configuration. It looks like Vodafone tries to be smarter than the web designers, developers and users…. All at once. I think that a lot of smart people work for Vodafone, but I suspect they can hardly address ALL needs or ALL users. Offering a choice is always the smartest solution. Vodafone should make a choice, let the user know and let her pick the best option for him.

I think that what Vodafone is missing is the context. From a user perspective Vodafone will never know the context in which the user is and from a technical point of view it is not allowing remote web applications to know the little that the user-agents and normal HTTP headers provide.
I think that this is a very good plan in theory, that unfortunately is not matching the reality. Transcoding without letting the user be in control is so much 1999 and WAP 1. Reminds me of WAP Gateways, proxies that were created to optimize network usage (and did it well), but were removed to open to gates to HTML and open browsing. Vodafone is implementing a new WAP Gateway that seems to be doing the wrong thing at the wrong time.
If and when there will be a technology that lets the user be in control and user a proxy to optimize network usage and resources, then I’ll be in line to use it. For the time being I’m happy to use Opera Mini for some sites and the native browser for other sites.

Some links worth to mention on the topic:
Novarra (the company that developed the transcoding proxy
Vodafone Mobile Internet and Content Services
Opera Mini
Barbara Ballard’s more than just a pretty face
Mobile Web Best Practice Working Group – Content Transformation Task Force
Client-Specific Web Services by Using User Agent Attributes

Video in Flashlite

YouTube was a massive success, I guess it doesn’t take a genius to think that there are many companies trying to get the same success on the mobile and I actually think that on the mobile it would be an even bigger success, especially once every mobile will be able to upload video files.

Well, while we wait for all the mobile browsers to reach that point (and some are already capable of this!), it’s probably good to get started with Flashlite. Here’s a free seminar that will be available online, only on 18 September 2007.

Embedding Video and Sound in Flash Lite

Mobile Web Book

After 10 months of work, a lot of research, a lot of testing, Cameron Moll‘s book is eventually almost ready to go live.

I’m happy to report the announcement from Cameron’s site. The book is really worth reading. The target audience is very wide, all those who are wondering if they should start building contents for mobile or who have started recently.
The book is full of good links, references and suggestions. I think the real value of this book is that it explains very simply what are the problems you might face, some of the solutions and how to get things to work.

Worth the money it costs.

Calling the mobile old-school!

If you were there in the early days of WAP and miss the initial experiences of WAP 1 development, you probably need a doctor. If the doctor can’t do anything, you have a chance to expose all your feelings to the community and get a nice post by James.

So make sure you take out all your inner feelings about WAP 1, why you loved it and why you miss it, answer the call: Calling the mobile old-school!

Submit Mobile Websites and Win $100 Cash

Ryan at MobileMammoth just sent me a link to a contest they launched. The contest is quite simple, provide your favorite top 5 sites, the best list of sites will get 100 USD. It’s not a fortune but I like the idea of sharing the best sites, so let’s post them!

Submit Mobile Websites and Win $100 Cash

Also, I suggest to the visit the homepage because they provide links and comments of mobile sites regularly.

First impressions about Nokia MOSH

I had just posted about Nokia MOSH and it’s a restricted beta, luckily I already got my password to access it. I created an account and uploaded my first content. I was really testing the site in parallel on my Mac and on my mobile phone.

Very first impression is that it’s an interesting new social site. It has all the common features such as upload a photo, invite a friend, exchange messages. The first question that comes to my mind is “So what’s new?”. Not very much I’d say, if you don’t consider that it’s made for mobile devices first and ALSO features a web interface. The main concept here is to be able to upload contents from your mobile device. This is nice and probably Nokia’s commitment is promising, but I don’t see it SO different from what Flickr Mobile has been offering for a while or even ShoZu.

But the very first question that came to my mind even before completing the registration was how they would recognize devices. The e-mail clearly stated that they will do their best to support all devices. Well, look at the image below and think…

I believe they are using WURFL. Not very hard to guess since it’s free and very well supported by many developers, but at the same time you might expect a different approach from Nokia. Are they using WURFL as-is? Did they patch and optimize the Nokia devices? Are they going to give back to the community?
How can I say it’s WURFL? If you look at the screenshot you can see a few things that hinted this to me such as “Research in Motion Ltd” instead of “RIM” or “Blackberry” that are much better known names than the complete company name. Another thing is the duplicated “Vitelcom” and “VITELCOM for Telefonica Movistar”, I remember adding those values and wondering if they should be merged or not. Then there are a few brands that you would not expect to be possible to be picked from a list of devices such as “W3C”, “WAPUniverse” and “WinWAP Technologies” (the first one is obviously not a device manufacturer and the other two are companies selling a browsing software). Also, if you look at the list of Sony Ericsson devices, you can see the “W810”, “W810i” and “W810c”, but the “W810” does not exist, it’s a virtual that device we defined in WURFL and the different localized versions (i for Europe, c for China and a for Americas) inherit from it most of their capabilities if not all.

Going back to the service, I liked that once I completed the registration it suggested to point the browser of my mobile device to a mosh.nokia.mobi. Accessing the site via your desktop browser provides the very same interface… Since they have a version optimized for desktop PC’s it would be good to be automatically redirected to the other version OR have a link. Nokia, if you are reading, I suggest you use is_wireless_device from WURFL.

One bug that I already found was that when trying to upload from the mobile, I could specify the tags. There was no mention of how to do it, so I used commas as on blogger.com, but this did not work, so the upload was not successful. Since I was using Opera, I could not pick a file from the filesystem, but I had to use the camera, so I had to take another picture and, even worse, some, but not all of the information I had provided was lost such as the tags (of course) and the content title. This one needs to be fixed.

Questions still open are:

  • is the site able to recognize the device while browsing and let me know which contents will be appropriate? Using Opera Mini it did not seem to pick it up correctly (but the pages looked very good).
  • how will they be able to keep out pirated contents. I have uploaded one image and it is now waiting for approval, but still I can’t imaging people at Nokia testing all the applications on any possible device.