Skip to content


Changing the headers for UIWebKit HTTP requests

Update
As of iOS 5, the iOS does no longer use the public API internally to set the UserAgent headers for all of the NSURLRequest objects that are created within UIWebView. So the method that is described here to change the UserAgent headers does no longer work. It relies on the public API for setting HTTP header fields. Therefore the only way left to change the UserAgent headers which works for iOS 5 as well is to write your own HTTP protocol header using the NSURLProtocol class. This is a little bit more complicated than the “method swizzling” approach.

I was asked several times, in which way the “User-Agent” header can be modified for the HTTP requests that are initiated from within the UIWebView object. iCabMobile is doing this, and also some other iPhone Apps, but the UIWebView API doesn’t provide anything which allows to modify the “User-Agent” information or any other HTTP header.

When you load a web page from the internet through UIWebView, you can provide a delegate which is called for each web page that is loaded. And in the method “webView:shouldStartLoadWithRequest:navigationType:” of the delegate, you’ll even get an NSURLRequest object you can look at, but unfortunately you can not modify this object. So there’s no way to change the default “User-Agent” information that is sent to the server, nor can you modify any other data.

When you’re loading data from the internet outside of UIWebView, you would probably use the NSURLConnection class. In this case you would create an NSURLRequest object (or the mutable counterpart NSMutableURLRequest) with all the HTTP headers for the request yourself (using the method “setValue:forHTTPHeaderField:”). You have full control over all of the HTTP headers you want to send to the server, including the “User-Agent” information.

When we assume that the UIWebView object will internally also use NSURLRequest or NSMutableURLRequest to create a HTTP request before this request is passed to the networking classes like NSURLConnection, we need a way to subclass or overwrite the method “setValue:forHTTPHeaderField:” of the NSMutableURLRequest class. Then we would be able to check for each HTTP header that is set for a NSMutableURLRequest, if this is the “User-Agent” header and if it is, we can modify it.

The only problem is that we can’t overwrite or subclass the NSMutableURLRequest class and force UIWebView to use our subclass instead of the original class. But iPhone Apps are written in Objective C and this programming language does allow exchanging and modifying classes, methods, variables etc. at runtime any time. So we can tell the Objective C runtime system that each time the method “setValue:forHTTPHeaderField:” of the “NSMutableURLRequest” class is called, our own method is called instead. This way it doesn’t matter that UIWebView will never call our method directly. Exchanging methods is called “Method Swizzling” and you can learn more about it on the CocoaDev page.

The method swizzling is very powerful, but it can be also very dangerous if you don’t know what you’re doing. So be very careful.

Now to the sources. I’ve implemented the method swizzling as a category of NSObject, so you can use it for all classes very easy (but as I said above, be careful, don’t use it if there are other options).

MethodSwizzling.h:

@interface NSObject (Swizzle)

+ (BOOL)swizzleMethod:(SEL)origSelector withMethod:(SEL)newSelector;

@end

MethodSwizzling.m:

#import "MethodSwizzling.h"

@implementation NSObject (Swizzle)

+ (BOOL)swizzleMethod:(SEL)origSelector withMethod:(SEL)newSelector
{
    Method origMethod = class_getInstanceMethod(self, origSelector);
    Method newMethod = class_getInstanceMethod(self, newSelector);

    if (origMethod && newMethod) {
        if (class_addMethod(self, origSelector, method_getImplementation(newMethod), method_getTypeEncoding(newMethod))) {
            class_replaceMethod(self, newSelector, method_getImplementation(origMethod), method_getTypeEncoding(origMethod));
        } else {
            method_exchangeImplementations(origMethod, newMethod);
        }
        return YES;
    }
    return NO;
}

@end

You can call “swizzleMethod:” for an object, passing in the selectors of the original and the new replacement methods. If the “swizzleMethod:” method returns with the result YES, each call of the original method will then call the replacement method and each call of the replacement method will call the original method. So within your replacement method you can still call the original method.

Here’s the implementation of the new replacement method for the NSMutableURLRequest class:

MyMutableURLRequest.h:

@interface NSMutableURLRequest (MyMutableURLRequest)

+ (void)setupUserAgentOverwrite;

@end

MyMutableURLRequest.m:

#import "MyMutableURLRequest.h"
#import "MethodSwizzling.h"

@implementation NSMutableURLRequest (MyMutableURLRequest)

- (void)newSetValue:(NSString *)value forHTTPHeaderField:(NSString *)field;
{
    if ([field isEqualToString:@"User-Agent"]) {
        value = @"The new User-Agent string";
    }
    [self newSetValue:value forHTTPHeaderField:field];
}

+ (void)setupUserAgentOverwrite
{
    [self swizzleMethod:@selector(setValue:forHTTPHeaderField:)
            withMethod:@selector(newSetValue:forHTTPHeaderField:)];
}

@end

This new method is implemented as a category, we don’t need to subclass. The replacement method for “setValue:forHTTPHeaderField:” is called “newSetValue:forHTTPHeaderField:” and it is simply checking if the “field” variable is equal to “User-Agent”. If it is, the value is modified. Afterwards the original method is called.
Please note: because the method swizzling exchanges the original and replacement methods, we have to call “newSetValue:forHTTPHeaderField:” to call the original method “setValue:forHTTPHeaderField:”. This looks confusing, but this is the way you can give control back to the original method.

The method “setupUserAgentOverwrite” has to be called once after the App is launched (for example in the Application delegate in the “applicationDidFinishLaunching:” method, or even in “main()”).

   [NSMutableURLRequest setupUserAgentOverwrite];

This should be done before any UIWebView objects are created to make sure that the “User-Agent” is modified for all requests.

You can also use this approach when you need to modify other HTTP headers.

Posted in iCab, iPhone & iPod Touch, Programming.

Tagged with , , , , .


66 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.

  1. Alex says

    —————————————–
    As I noted above in one comment, I assume the only “legal” way to change the UserAgent that also works under iOS 5 is to implement your own HTTP protocol handler using NSURLProtocol, which is a public API.
    —————————————–
    @Alexander: I did use NSURLProtocol to hook the request of UIWebView, but I found a problem: some webpages, when user clicks a link, will popup a “window”, and the content of the “window” is fetched by ajax. if i use NSURLProtocol to hook the request, the “window” won’t popup, I don’t know why.

    sorry for my poor English -_-!!!

  2. Alexander Alexander says

    @Alex
    I’m not sure if the AJAX issue is really related to NSURLProtocol. There’s a general unsolvable problem with popup windows in the iOS: The UIWebView API does not support multiple windows, so it does not have any method that deals with popup windows. And this means that all attempts of a web page in opening a new window will be silently ignored by UIWebView. No window will ever open.

    So in order to get popup windows or links open in new windows or tabs, you as a developer need to modify the web page, modify link targets, overwrite the “window.open” property of JavaScript to get notified when a web page tries to open a new window, and then open the window yourself outside of the context of the UIWebView. This works find in most cases, but it fails if a web page is using the reference to the new window that is normally returned by the “window.open” call. This is because even if you’ve opened a new
    window and created a new instance of UIWebView for this “window-open” call, you can not pass back the reference to the new window as return value of this “window.open” call. So whenever a web page tries to use a window references that was returned by window.open, it will fail, because this can never be a valid reference to the new window.

    And web pages might use this reference to pass new content to the new window. And this will fail.

  3. Alex says

    @Alexander: sorry I didn’t explain clearly, the “window” is not a real window, just a region that covers the part of the original page, and it looks like a “window”, this region may rendered by CSS or some else.

    when user click the link, it won’t open a new page, just show this “window”.

  4. Alexander Alexander says

    @Alex
    OK, maybe you should use a network sniffer to check what exactly is going on here. Please note that there can be also HTTP redirections, which you also need to process in the NSURLProtocol. If you don’t, the UserAgent header might be missing for redirections and if the server doesn’t like this, you might see some unexpected results.

  5. Alex says

    Hi, @Alexander,
    I found a problem, I use NSURLProtocol to filter the request from UIWebView, but I found a webpage contains this JS code:

     $.ajax({
            async: false, cache: false, type: "get", dataType: "text",
            url: "/Order/OrderTrackLine/" + formcode + "?province=" + escape(province) + "&status=" + status,
            data: "",
            beforeSend: function () { $("#info" + formcode).html("正在读取,请稍候..."); },
            success: function (data) {
    
                $("#info" + formcode).html(data);
            },
            error: function (XMLHttpRequest) { },
            complete: function (XMLHttpRequest) { }
        });
    

    NSURLProtocol can capture this request, but after -connectionDidFinishLoading: called, nothing continue, that’s say, -stopLoading method isn’t called, and the content that fetched by this ajax request can’t be shown in the web page. I don’t know why.

  6. Alexander Alexander says

    @Alex
    In general this should work just fine. I’ve checked this myself again, and all the methods (including stopLoading) are called as expected. Is there a special reason why you’re using a synchronous Ajax call? Normally you should avoid synchronous Ajax requests whenever possible.

    But in case your still using iOS 6.0.0, you should definitely update, because iOS 6.0.0 has some serious bugs which caused that an App should not always find out if the page loading has finished (even the “isLoading” method of UIWebView did return “YES” when the page load was definitely finished).

    Also please note that when showing a Javascript “alert” (for example for debugging) within the context of the XMLHttpRequest, the request is stopped until the alert is dismissed. And this also means that “stopLoading” is not called until the alert box is dismissed.

  7. Alex says

    @Alexander:

    Yes! you’re right! The problem is caused by sync ajax.(JQuery v1.9, but not happened in v1.4), and stopLoading is not called before I close the “popup win” in the web page.

    But is there any way to notify the iOS to call “stopLoading” after data received?
    Thanks!

  8. Alexander Alexander says

    @Alex
    You have to avoid showing the Alert within(!) the AJAX context. As long as the alert box is shown (an alert box is a modal window), the Javascript execution stops until the user has dismissed it. And therefore the AJAX execution also stops. Which means the notification about the end of the AJAX context also has to wait.

    So maybe if you need to shot an alert box t the user, open it after the AJAX call has ended, maybe open the alert box via timer, so it is not directly linked to the AJAX context anymore.

  9. Max Litteral says

    This isn’t working great for me on iOS 8 (maybe 7 as well). This actually seems to make no difference than the code I found for a custom NSURLProtocol (found at http://eng.kifi.com/customizing-uiwebview-requests-with-nsurlprotocol/), Without the protocol the field is never equal to User-Agent. The custom protocol works sometimes, but is very flakey and i can’t figure out why. Like DuckDuckGo.com won’t load with the custom user agent, Google.com won’t either, but a web search on google works. CNN works if directly loaded but i don’t think it works if loaded by a google search a user has said. Have any idea whats going on and why the user agent doesn’t change every time? Logging and using the Safari debugger for UIWebView shows that User-Agent is set to the system default

  10. Alexander Alexander says

    @Max Litteral
    The custom protocol should usually work just fine. But because the „User-Agent“ is not only sent via HTTP to the server, but instead a web site can also check this via JavaScript (navigation.userAgent), it is often not enough just set the User-Agent for HTTP in your own custom URLProtocol handler. You also need to overwrite the JavaScript property, so that web sites checking the user Agent via JavaScript would also get the “correct“ User-Agent.
    I guess this is the problem you experience, it’s probably not the custom URLProtocol.

  11. Max Litteral says

    Ah, I didn’t know that. Some websites (Duckduckgo) didn’t work, but my semi-solution was to also check if the user agent matches the custom user-agent and if not set it again, this fixed it for a lot of websites, DDG looks the same on every device, but didn’t see the buttons that are shown on Mac on the search engine page. Ill try setting it with javascript and see if it works, thanks!

  12. Alexander Alexander says

    @Max Litteral
    Please note that overwriting the „navigation.userAgent“ in JavaScript can be challenging.

    You need to replace the whole „navigator“ object with your own custom one. And in order to get this working on all sites, you even need to make sure that you overwrite it before the web site has a chance to do anything (some sites do check the UserAgent before they have finished loading). So when using UIWebView, the „didFinish“ delegate method is far too late, while the „didStart“ method is too early.

    In iCab Mobile I solve this by intercepting the network traffic and inject my code into all HTML code that is received before the data is delivered to the UIWebView. So the first thing a web site is doing is to run my own JavasScript code to overwrite the navigator object.

    When using WKWebView there are methods to provide code which is run before the web site is loaded, so this is much easier here. But WKWebView has other limitations which can make it unusable.

    Another note: Not all web sites use the userAgent information to determine a mobile or desktop browser. Web sites can also check the screen resolution, check if touch gestures are available etc. This means a fake userAgent information doesn’t always work to force a web site to show the desktop version.

  13. Max Litteral says

    Do you know of any websites the only check the navigator? Also I found online that to change the user-agent I just call this, [self stringByEvaluatingJavaScriptFromString:[NSString stringWithFormat:@”window.navigator = {userAgent:’%@’}”, userAgent]]; and I found a w3schools page that displays the navigator user-agent, and it displays the correct user-agent but I set it in the webViewDidStartLoad and webViewDidFinishLoad (No reason I guess to call it in didFinish probably) and it seems to work since it does display the correct one I set? Anyways thank you for the help!

  14. Alexander Alexander says

    @Max Litteral
    The „navigator“ object has many other properties, besides „userAgent“, so I guess your solution will fail if a page tries to check one of the other properties as well, because you only define the „userAgent“ property for your own navigator object. So if you know that the other properties are not needed, your solution would be fine, otherwise you also need to define all the other properties.

    Sorry, with „didFinish“ I actually meant „webViewDidFinishLoad“, I was just to lazy ;-)

    If a web site ask for the userAgent after that page has loaded (like in the example of w3school), then it works fine if you just overwrite the userAgent in „webViewDidFinishLoad“. But if the page checks the userAgent while the page is still loading, then „webViewDidFinishLoad“ would be too late. And overwriting this in „webViewDidStartLoad“ seems to be very unreliable. It seems to work in a few cases, but not all.

  15. Max Litteral says

    ah, okay. Thank you for all the info! Ill add the other keys and find a better place for it then! :P

1 2

Continuing the Discussion

  1. UIWebView user-agent weirdness and how to change user-agent value programmatically | blog.sallarp.com linked to this post on December 5, 2010

    […] found the solution here, kudos to Alexander Clauss, and it’s called “Method Swizzling”. Because UIWebView […]



Some HTML is OK

or, reply to this post via trackback.