It is very common that over the years something gets more and more cluttered, until at some point someone comes in and decides it’s time to clean up and start fresh, based on current and up-to-date needs. User-Agent strings, part of the HTTP request headers, are no different and between desktop and mobile browsers, the history is long and the amount of text (some might say useless text) has just kept growing. I am going to try and write down what is important today and why.
A little introduction is probably needed. I have been the maintainer of WURFL, a device database, for about 5 years and then worked on another device database called DeviceAtlas for another 3 years. During these years I also worked with the W3C and in general with the mobile Web community to define how the mobile Web should look like and what developers needed to do their job better.
I am going to take four User-Agent strings for four platforms that are meaningful for mobile and analyse what is useful today, what they have currently and what they should have. I will also briefly jump into some other headers as they can help in a number of situations.
User-Agent string definition
A clarification here is probably needed. Let’s start from what the User-Agent string is, if you don’t know. When an HTTP client such as a Web browser makes a request to a server it sends a number of headers as part of that request. One of these headers is called User-Agent and it is meant to identify the client. This is how it is defined in the RFC-2616:
The User-Agent request-header field contains information about the user agent originating the request. This is for statistical purposes, the tracing of protocol violations, and automated recognition of user agents for the sake of tailoring responses to avoid particular user agent limitations. User agents SHOULD include this field with requests. The field can contain multiple product tokens (section 3.8) and comments identifying the agent and any subproducts which form a significant part of the user agent. By convention, the product tokensare listed in order of their significance for identifying the application.
User-Agent strings in mobile
In the mobile space specifically this string has historically been used to recognise devices. Since each device has very specific features and installing third party plug-ins has always been (and still is) nearly impossible, recognising a device easily is fundamental to the creation and provision of content that is not only optimised for a device, but often that it is simply supported. In the early 2000 the User-Agent string of a Nokia 7110 was “Nokia7110/1.0 (04.84)“. At that time it gave all the information that was needed. Over the years a lot of information has been added and eventually, recently, a lot of mobile browsers have started emulating the strings that were usually provided by desktop browsers, unfortunately picking up a lot of useless information and removing what was useful.
Here are my four sample User-Agent strings, an iOS 4.1, an HTC/Google Nexus One, a Nokia N8 and the latest Fennec on an Android device:
- Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-gb) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7
- Mozilla/5.0 (Linux; U; Android 2.2; en-gb; Nexus One Build/FRF83) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
- Mozilla/5.0 (Symbian/3; Series60/5.2 NokiaN8-00/010.016; Profile/MIDP-2.1 Configuration/CLDC-1.1 ) AppleWebKit/525 (KHTML, like Gecko) Version/3.0 BrowserNG/18.104.22.168 3gpp-gba
- Mozilla/5.0 (Android; Linux armv7l; rv:2.0b7pre) Gecko/20101006 Firefox/4.0b7pre Fennec/4.0b1
iPhone User-Agent string
Let me start with the first one, the iPhone. There are a number of things here of interest: that it’s an iPhone, the software version, the type of browser and possibly the browser version. I will get back to why the browser version may or may not be useful later. Do we care it’s “like Mac OS X“? Do we care that “KHTML (is), like Gecko“?
So what we could do is strip out everything that we don’t need and
Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_1 like Mac OS X; en-gb) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8B117 Safari/6531.22.7
If I was a VP at Apple and had any decision power I would also specify the iPhone model and it’s probably a good idea to also be explicit about the vendor (it happened that two vendors named two devices the same way). Here is a very short string that tells me everything I need:
Google Nexus One User-Agent string
Second on my list is the Nexus One, again a lot of information we don’t need. Who needs to know it’s Linux when we can’t really hack the kernel or install X.org (at least from the browser!)? Who cares about the language when there is an appropriate header for that called Accept-Language? So what we have is
Mozilla/5.0 (Linux; U; Android 2.2; en-gb; Nexus One Build/FRF83) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
but what we should have is
Nokia, with the N8 and a lot of other devices has taken the best of all worlds. On one side emulating a desktop browser and on the other side bringing all the extra information about J2ME, audio and video streaming and other details that have been meaningful in the mobile space at some point in time. The sad truth is that all these supported environments and protocols are so fragmented that the name alone is not useful and hence, just a burden. Let’s clean up the house, remove the information that the device supports MIDP-2.1, because anyone who has ever seen the build-chain of a J2ME application knows that you need to know a lot more about the specific JSRs that are available and of course the device-specific quirks. Also, don’t forget that this is just an example, Sony Ericsson, Samsung, LG and the other vendors are all more or less the same. So bring some fresh air, leave space for what is important and change this
Mozilla/5.0 (Symbian/3; Series60/5.2 NokiaN8-00/010.016; Profile/MIDP-2.1 Configuration/CLDC-1.1 ) AppleWebKit/525 (KHTML, like Gecko) Version/3.0 BrowserNG/22.214.171.124 3gpp-gba
Fennec on Android
Now on with my last example and I am picking Fennec on Android simply because it’s probably the newest kid on the block, but the exact same comment would be valid for Opera browser and other third party browsers that you can install on your mobile device. For starter, there is no reference to the device, there are a already more than 100 of Android devices, how will I know which one? How will I know if it’s Android 2.2 with Flash or not? So this is what we have today
Mozilla/5.0 (Android; Linux armv7l; rv:2.0b7pre) Gecko/20101006 Firefox/4.0b7pre Fennec/4.0b1
This is what we should have (and for the sake of this example I am assuming the device is a Nexus One):
Now, you might comment that I will know if Flash is available looking at some other headers. Well this would be true, if the browser sent the appropriate information. If you have been into mobile detection for a little while you will know that since day one you could not trust the Accept header, mostly because browser vendors have been very bullish about their ability to support content, to then generally disappoint their users (and most of all the _customers_ of mobile sites and content providers). For example the Accept header of this Fennec for Android is “text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8” (see the full headers recorded on logme.mobi). As you can see, this browser supports ANYTHING (the little */* towards the end), isn’t that great? Now, will it play a real-video? Probably not. Will it play Flash? MAYBE! Will it play an H.264 video? I don’t know. Oh, yes, these are the same Accept headers that Firefox for PC sends, but it is also true that on my PC or Mac I can download the file and play it with VLC or I might want to download a Word document and read it later. On mobile devices it’s slightly different and most mobile devices do not let you download and store files and even after I have downloaded it, my context is normally such that if I cannot read the file now I don’t care. Hence the server needs more information in order to provide the right content; the server might have the file in the right format, if it knew what is supported.
PS: In fairness Opera Mini adds extra headers to tell you the device it is running on, when they know it and this depends on how you installed the client.
I am not advocating overloading the headers, we have examples of other headers sent to the servers that are nearly completely useless. Looking at the N8 again, it sends a header about the Music shop version (X-Nokia-Musicshop-Version: 13.1012.15), do I care? Unlikely. Would I care to know the version of Ovi Maps installed so that I can provide a link that will launch the Maps application? More likely!
As you might have noticed, I have stripped the “Mozilla/” at the start. There is a myth that servers will not provide the Web site or will provide a very limited version of the site if that string is not there. I have asked many times and although everyone seems scared of this, I do not know of a single site that really does it today. If you know one, please let me know and most of all WHY is this site doing this?
So please, if you are browser vendor, take out what is not useful and keep what developers need.