Google’s GWT

Transcoding is a hot topic these days in the mobile community. Google’s GWT has come up a couple of times in some conversations I had in the last 2-3 weeks. I never wondered how they decide to transcode or not, anyway.
I gave it a shot today using cURL and an emulator to get access to some devices quickly.

How I did it
I got on http://www.google.com/m and searched for “WURFL test”; I left the default “Web” setting. As expected my http://t.wurfl.com was the first link. I clicked on it and got to the page, the layout was unchanged, but that’s not a surprise, it’s so simple. Scrolling down I noticed that the user-agent string is not my phone’s, but an ugly generic MSIE user-agent string. Scrolling down a bit more I noticed the standard chrome provided by GWT that lets you disable images or request the page in the original source.

The solution
I sent an e-mail to Sean Owen, who is in the W3C MWI with me. Surprisingly he replied on a Sunday and was very helpful. He explained me how their crawler is able to mark mobile sites and make sure the GWT will not transcode anything. The problem is that at the moment this magic feature is available in the US only and will be rolled out to the world soon. Since I left “Web” in my search I got the web version of my site (t.wurfl.com has a very basic device detection that provides different markup based on the Accept headers). US users should already get the mobile version of t.wurfl.com. Anyway, Sean also suggested a trick that now provides the mobile version to all mobile users without any other update required. I added the following tag in the head of the page:
<link rel=”alternate” media=”handheld” href=”http://t.wurfl.com/index.php”>

Seconds after the update I requested again the page with the emulator and got the mobile version.

PS: if you want to get a mobile page using cURL you can do this:
curl -D – -A “Nokia6600” “http://www.google.com/gwt/n?u=t.wurfl.com/”

You will see the page source and the HTTP headers sent by the server. Read the manual for more commands, cURL is super-powerful!

Are transcoding proxies needed on the mobile?

Most users have never used the mobile web, in some cases they have downloaded a few ringtones, a wallpaper or a java game. The operator portals are the still the first and probably only site that the average user has ever visited. Sometimes some brave individual looks for adventures and surpasses the borders of his operator portal. These users either know exactly what they want and know the URL or don’t have any idea of what they can find. Users who don’t know what they can find will go to some search engine like Google or Yahoo! (hardly the mobile versions) and type a keyword such as “tennis”. They will get a list of links and follow them. There are very good chances that they will reach a web site that doesn’t even take into consideration the possibility that a user with a mobile phone will visit and is not doing any device detection or content adaptation.
If this is case, and unfortunately still is for most sites, there are two chances, either the user has a really smart browser like Opera Mini or Nokia’s webKit-based browser, or he will download a huge page that he will not be able to read. After 2 or 3 different sites have been tested, users will be discouraged and will not take the risk of going out of the operator’s portal ever again.

Operators should be happy about this and try to disincentivate browsing outside of the portal. On the other hand, browsing means traffic, traffic means money, so accessing the “full web” can be a positive thing for mobile operators too. If browsing can be a revenue stream you don’t want your users to be discouraged, you want them to have a taste of what they can get on their mobiles.
Opera Mini is one possible solution, transcoding is the other. Google offers transcoding automatically, operators might choose to get their own transcoding software. The result is that users get what they want (the tennis site), operators get what they want, money from data traffic. Looks like a win-win solution.

Within some limits, this is effectively a win-win solution. The limits are defined by the user experience that the transcoding engine can provide. It is proven that transcoding a web page and fitting it into the small screen of a mobile device will not provide the best user experience, nevertheless it is still better than a kick in the teeth (like the average rendering of a full web site on a Motorola V3).
Transcoding is like a shortcut to get more users on the mobile web. Major web sites should consider developing a mobile version quickly simply because mobile is the future. At the same time there is a number of sites that will never be converted (think of all the geocities pages!) and there are many minor sites that will consider mobile only in 3 or 5 years from now. We do want users to be able to find information on the mobile today, don’t we?
Transcoding engines must be smart enough to recognize sites that are made for the mobiles and leave the content as is and convert the content only when this will not fit the features of the mobile device (or set-top-box). Transcoding engines should be smart and act when needed and never get in the way, they must be an extra tool, not a mousetrap that gets in the way of the developers and designers that are sweating to produce content that is optimized for mobiles. Transcoding the content of a site that was made for mobiles will most likely break the usability and optimizations that the designer put in place.

At the same time, users should be educated that there is a lot of information for them on the internet and that they should be looking and asking for mobile versions. We should not expect the average user to understand mark-ups, XHTML-MP [PDF], transcoding, image rescaling, but they do understand when a page is not usable on their mobile and if they can’t find anything interesting, they won’t come back.

Room with a view

Usually when people go on holiday and reserve a hotel room, they expect a sea-view or the country-side, normally something beautiful, relaxing, and everything you don’t get when you open your window at home, in the big cities where many of us live.

I often travel for pleasure, luckily, but I also often travel for business.
This week my usual hotels in Dublin were fully booked, so this is the special view they provided with my room.

I can assure you it’s not an effect of the digital camera, the trains are really about 3 or 4 meters from my window.

Milano, Malpensa and Alitalia

Alitalia is short of money and this is no surprise since their flights are expensive and the quality is low. To give you an example, I flew to Miami in May and my luggage was broken during the flight. I immediately filed a complaint in Miami, as suggested in Miami I kept my copy and waited to be back in Milan. Once in back in Malpensa 2 weeks after I went to the “lost and found” desk to seek for some extra support. They gave me a fax number and told me to send me boarding card a copy of the filed complaint and a copy of my ID. I did so on Wed, June 6th (5 days later). On Sept, 18 I received a letter from Alitalia’s attorneys saying that unfortunately they may not accept my request because more than 7 days had gone from when I filed the complaint and when I sent them a copy via FAX. I obviously did not think of taking the trouble of faxing them while on holiday and did it as soon as I got back home. I had 7 days to let them know. They took 3 months to answer to me. Sounds a bit unbalanced. I fly quite often these days and Alitalia’s airplanes are the oldest and dirtiest.

Nevertheless they are short of money and this is probably because the company is led by corporations and politicians and not people who know about business.
They think Malpensa is a source of costs more than of revenues. They are now planning to cancel about 150 routes.

The industry has a reply to this move: EasyJet has doubled the flights to and from Malpensa, RyanAir wants to spend 850M Euro over 5 years to expand their routes adding *80* new to the existing ones.
I wonder if EasyJet and RyanAir would invest money on an airport that would not assure them to fill every single flight.

Is Apple aware of exchange rates?

Apple computers and all the accessories have always been more expensive in Europe than in the US. Once Apple computers were way more expensive than an unbranded PC, but they used different technologies and it was hard to make a real comparison. Today Apple computers are running Intel chips, IDE hard drives, have USB ports and most of the hardware is the same. Prices in the US have dropped and are not aligned with Dell and HP.
In Italy and in Europe they are still way more expensive. Prices are the apparently the same, what in the US costs 1199 USD, will cost you 1199 Euro in Europe (see the iMac’s). What strikes me is that today, 1 Euro is worth almost 1.4 US Dollars, so something that costs 1000 Dollars in the US should cost less than 750 Euro here!
The latest example of this is the iPhone that was just launched in the UK, it costs 269 Pounds that are about 390 Euro and also about 545 USD! Once again, the price in USD and Euro is the same it’s just that they are not the same currency.

Your Own (Free/Open!) Adserver for Mobile

A very good post by Mike Rowehl: Your Own (Free/Open!) Adserver for Mobile

How to map this in a device DB?

Gregg pointed me in the direction of this very interesting Chinese device. I suspect we will have to add some new properties in the DDWG Core Vocabulary in order to properly describe this.

This is a screenshot:

Taken from this site: http://mobile.139shop.com/mobile/75/5673.htm

We’re thinking of giving a bounty to the first person who can explain what the added “gadget” on the left side of the device is for.
Is this some special equipment for ice skaters?

There isn’t just ONE WebKit

Reporting from Surfin’ Safari Blog:

[W]e have a WebKit detection script that properly checks for the WebKit engine (not just Safari) and properly detects versions.

This script has now been updated to support iPhone and the new iPod touch. You can try a live version to see what kinds of results you’ll get on different browsers and devices.

For all those that believed that mobile devices are the same as desktop PC’s.

Read the full article here: WebKit Detect script updated for iPhone and iPod touch.

Is it good to hide the user-agent in mobile?

Recently Vodafone plugged a new infrastructure in their mobile network. In two words, it is a proxy that is also capable of transcoding web page contents to be suitable for mobile devices.
While I don’t know the marketing plan behind this, it is obvious that the main driver is to make the entire web available on mobile devices, even if the source content was not designed for use on mobile devices. This is good, of course, because it allows all Vodafone (UK-only at this time) customers to access all the valuable information that you can find on the web normally, all that information you’re used to (and I’m addicted to, I’d say). But how is this software working? It is a proxy sitting sitting in the middle between the mobile device and the open internet, all contents requested by Vodafone-UK users are received by the proxy that performs the requests, gets the page, elaborates it (or maybe simplifies it) and delivers a new page to the mobile browser. While processing the page, the proxy takes out all contents that might create a problem to the mobile browser, tries to clean up the markup if there’s any tagsoup and sends something that is optimized for the mobile.

All this sounds perfect, doesn’t it? Well, not to me.
There are a few glitches here and there that I think are breaking the original plan of improving the user experience. First of all the proxy sends a unique user-agent HTTP header string that hides the original device behind a generic Mozilla-compatible browser. Remote servers will think that the requesting browser is a desktop PC and send the “appropriate” content, unfortunately sometimes even the remote applications have an adaptation engine that will provide content for desktop PC that is not meant for the mobile. Let’s think of a company selling ringtones and contents for mobile phones, when visited by a mobile device will let the user pick a content and download it directly, but if a desktop PC is recognized, it will ask for the mobile device model, show the available contents, ask for the phone number and then send a wappush to let the user download the content. If the proxy is hiding the mobile device and pretending it’s a desktop PC the original plan is failing.
The answer to this would be to have a protocol that describes how the remote web application could tell the Vodafone proxy that it IS capable of providing appropriate content for mobile and stop the adaptation, unfortunately, such a protocol does not exist, as of today.
Is the user-agent SO important? At this time IT IS so important, there isn’t in fact, a technology that lets a web server know the full capabilities of a mobile device. The OMA and the W3C are both working on this topic, but the technology will not be ready before 1 year or so and then we will have to wait for it to be implemented in mobile devices and later to become a common feature: you can easily imagine the timeframe. The solution proposed by Vodafone is that sites that are able to provide mobile content should contact them and provide all the URL’s. Sorry, but this doesn’t seem very scalable.

Another tiny issue is that the above plan makes most sense if you’re thinking of old WML browser that would not be able to parse an HTML or XHTML page, but what about the recent WebKit implementation that Nokia is delivering pre-installed in their Symbian phones? What about Safari on the iPhone? What about Opera Mini (that I have to admit is a quantum leap in browsing as it provides very smart browsing features to all J2ME devices)? All these browsers provide TODAY (and even 1 year ago if you think of Opera and the first devices running the WebKit-based browser) a technology to adapt content that was meant for desktop PC’s with big screens to be viewed and used decently on a mobile device. Unfortunately, again, there isn’t a technology that will tell the Vodafone proxy if the device is running a smart browser…. Oh, wait… The user-agent string DOES tell it. So let me recap: Vodafone reads your browser’s user-agent string, tries to understand how smart your mobile browser is and then hides the very same information to the remote server! Is this a dog biting its own tail?
But, isn’t Opera Mini doing the same, using a unique user-agent for all mobile devices? Yes of course, but the difference is that Opera mini is installed by the user, the user knows about it and also has a chance to set a few options to make the software behave as she better likes it and, by the way, if she doesn’t like it, she can use the native browser or install another one, if available. Vodafone’s proxy is doing this without asking anything to the user and without allowing any personalization and configuration. It looks like Vodafone tries to be smarter than the web designers, developers and users…. All at once. I think that a lot of smart people work for Vodafone, but I suspect they can hardly address ALL needs or ALL users. Offering a choice is always the smartest solution. Vodafone should make a choice, let the user know and let her pick the best option for him.

I think that what Vodafone is missing is the context. From a user perspective Vodafone will never know the context in which the user is and from a technical point of view it is not allowing remote web applications to know the little that the user-agents and normal HTTP headers provide.
I think that this is a very good plan in theory, that unfortunately is not matching the reality. Transcoding without letting the user be in control is so much 1999 and WAP 1. Reminds me of WAP Gateways, proxies that were created to optimize network usage (and did it well), but were removed to open to gates to HTML and open browsing. Vodafone is implementing a new WAP Gateway that seems to be doing the wrong thing at the wrong time.
If and when there will be a technology that lets the user be in control and user a proxy to optimize network usage and resources, then I’ll be in line to use it. For the time being I’m happy to use Opera Mini for some sites and the native browser for other sites.

Some links worth to mention on the topic:
Novarra (the company that developed the transcoding proxy
Vodafone Mobile Internet and Content Services
Opera Mini
Barbara Ballard’s more than just a pretty face
Mobile Web Best Practice Working Group – Content Transformation Task Force
Client-Specific Web Services by Using User Agent Attributes

Code snippet of the day

Sometimes working late has some side-effects. The other day I had a problem with a PHP script that should have taken 1 or 2 seconds to run and instead seemed to hang the webserver. Eventually I found the issue:


while ( $variable ) {
   if ( $a == $b )
     echo "do something";
}

Ermm… for some reason I never changed the value of $variable and of course the loop became infinite.