Customize the contextual menu of WKWebView on macOS

Under iOS the WKWebView class provides a delegate method which allows to customize the contextual menu of the web engine (the menu which opens when long-pressing a link). Unfortunately under macOS the WKWebView does not provide such a method for its contextual menu. This article explains how you can customize the contextual menu of WKWebView under macOS as well. It’s not that obvious how to do so, but it can be done.

Because the WKWebView API for macOS itself does not provide anything at all which deals with the contextual menu, we need to look elsewhere. WKWebView is a subclass of an NSView on the Mac and the NSView class has methods to deal with a contextual menu. These are    


  open func willOpenMenu(_ menu: NSMenu, with event: NSEvent)

which is called before the contextual menu of the view will open and


  open func didCloseMenu(_ menu: NSMenu, with event: NSEvent)

which is called after the contextual menu has closed.

So in order to customize the contextual menu of WKWebView, we could simply subclass WKWebView and override these methods in order to modify this menu. In the „willOpenMenu“ method we can add, modify or remove menu items and in „didCloseMenu“ we would reset anything that needs to be reverted back to normal.

In „willOpenMenu“ the parameter „menu“ represents the contextual menu, so we can easily inspect all the available menu items, remove what we don’t need and add all the new menu items we want to have in the menu. This sounds easy, but there are a few issues.

Because Apple does not officially provide a way to customize the contextual menu, nothing is officially documented. So some simple reverse engineering is required to find out the meaning of the default menu items and their actions.

  • The first thing we can find out is that the „identifier“ property of the menu items is very clear about the meaning, and these identifiers look very stable – they haven’t changed between different macOS releases.
  • The second thing we can find out is that if a link is clicked, we find menu items with the ID „WKMenuItemIdentifierOpenLinkInNewWindow“ and „WKMenuItemIdentifierDownloadLinkedFile“. If an Image was clicked we find „WKMenuItemIdentifierOpenImageInNewWindow“ and „WKMenuItemIdentifierDownloadImage“ and for videos „WKMenuItemIdentifierOpenMediaInNewWindow“ and „WKMenuItemIdentifierDownloadMedia
  • Another thing we can find out is that the menu and its items do not contain any information about the context, e.g. the link, image or other element on which the contextual menu was opened is totally unknown.

The first point makes it easy to remove all the menu items which deal with features we do not want to provide in out App. We just need to check the identifiers and remove items whose identifier match these features.

But the latter point is a problem, because if we want to add custom menu items which deal with the current context (the clicked link, image, etc), we need to know about this context.

So how we can solve this issue? The solution I’ve found is the following:

Because the context is unknown but the context can only be a link, an image, a frame or a video (something that can be clicked and which has a URL) and for all these contexts there’s already a default menu item which opens the context object in a new window, why not simply use these existing menu items and then intercept the default action for the menu item and replace it with our custom action. All the default actions to open an object in a new window will call the following method of the standard navigation delegate of WKWebView:


  func webView(_ webView: WKWebView, createWebViewWith configuration: WKWebViewConfiguration, for navigationAction: WKNavigationAction, windowFeatures: WKWindowFeatures) -> WKWebView?

And this delegate method is getting the required context via the navigationAction parameter, which means the URL of the object. The delegte method itself and its parameters are unable to provide any information about our custom action of the menu item, so we have to find another way to get this information there.

We already have overridden the WKWebView class in order to intercept the contexual menu, therefore we could simply add another property to this subclass where we save the custom action for the selected contextual menu item. In the navigation delegate method above it’s then easy to check this property and use it to decide wether to continue with the default action (creating a new window) or continue with the custom action.

We now have everything we need to implement our own custom menu items for the contextual menu.

At first we implement an enum for all our custom actions:


  enum ContextualMenuAction {
    case openInNewTab
    case addToBookmarks
    // add any other actions you want to have
  }

Now we implement the subclass for the WKWebView, including the property contextualMenuAction which will hold the custom action that was selected from within the contextual menu. This property is declared as optional, so it will be nil if a default menu item is selected.


  class MyWebView: WKWebView {

    var contextualMenuAction: ContextualMenuAction?

Now we override the willOpenMenu method, which will be called when the contextual menu opens and where we modify the menu according to our needs.


  override func willOpenMenu(_ menu: NSMenu, with event: NSEvent) {
    super.willOpenMenu(menu, with: event)
    
    var items = menu.items
    
    // In our example we don't need Download options, so we remove these menu items
    for idx in (0..<items.count).reversed() {
      if let id = items[idx].identifier?.rawValue {
        if id == "WKMenuItemIdentifierDownloadLinkedFile" ||
           id == "WKMenuItemIdentifierDownloadImage" ||
           id == "WKMenuItemIdentifierDownloadMedia" {
          items.remove(at:idx)
        }
      }
    }
    
    // For all default menu items which open a new Window, we add custom menu items
    // to open the object in a new Tab and to add them to the bookmarks.
    for idx in (0..<items.count).reversed() {
      if let id = items[idx].identifier?.rawValue {
        if id == "WKMenuItemIdentifierOpenLinkInNewWindow" ||
           id == "WKMenuItemIdentifierOpenImageInNewWindow" ||
           id == "WKMenuItemIdentifierOpenMediaInNewWindow" ||
           id == "WKMenuItemIdentifierOpenFrameInNewWindow" {
        
          let object:String
          if id == "WKMenuItemIdentifierOpenLinkInNewWindow" {
            object = "Link"
          } else if id == "WKMenuItemIdentifierOpenImageInNewWindow" {
            object = "Image"
          } else if id == "WKMenuItemIdentifierOpenMediaInNewWindow" {
            object = "Video"
          } else {
            object = "Frame"
          }
          
          let action = #selector(processMenuItem(_:))

          let title = "Open \(object) in new Tab"
          let tabMenuItem = NSMenuItem(title:title, action:action, keyEquivalent:"")
          tabMenuItem.identifier = NSUserInterfaceItemIdentifier("openInNewTab")
          tabMenuItem.target = self
          tabMenuItem.representedObject = items[idx]          
          items.insert(tabMenuItem, at: idx+1)
          
          let title = "Add \(object) to Bookmarks"
          let bookmarkMenuItem = NSMenuItem(title:title, action:action, keyEquivalent:"")
          bookmarkMenuItem.identifier = NSUserInterfaceItemIdentifier("addToBookmarks")
          bookmarkMenuItem.target = self
          bookmarkMenuItem.representedObject = items[idx]
          items.insert(bookmarkMenuItem, at: idx+2)
        }
      }
    }
    
    menu.items = items
  }

The first action is to call super so that the WKWebView can do everything it needs to do for the menu. Then the menu items will be modified to our needs. In this example all the download items will be removed from the contextual menu. The second step is to look for the default menu items which open an object (link, image video, frame) in a new window and then create a new custom menu item which is supposed to open this object in a new tab and a second menu item which is supposed to save the object into the bookmark (this article doesn’t cover how to do this, it’s just an example how to add such menu items and later how to detect and process the selection of these menu items). The new custom menu items use the „representedObject“ property to store the default menu item which is needed later when the user has selected our custom menu items. Our new custom menu items are then inserted after the original menu item within the contextual menu. Now the macOS will show the contextual menu with all our customizations.

The method which is called when selecting our custom menu items is implemented next.


  @objc func processMenuItem(_ menuItem: NSMenuItem) {
    self.contextualMenuAction = nil

    if let originalMenu = menuItem.representedObject as? NSMenuItem {
    
      if menuItem.identifier?.rawValue == "openInNewTab" {
        self.contextualMenuAction = .openInNewTab
      } else if menuItem.identifier?.rawValue == "addToBookmarks" {
        self.contextualMenuAction = .addToBookmarks
      }
      
      if let action = originalMenu.action {
        originalMenu.target?.perform(action, with: originalMenu)
      }
    }
  }

At first the property contextualMenuAction is initialized to nil. Then we check if the property representedObject contains a NSMenuItem, in which case this menu item is the original default menu item and we use its target and action to trigger its default action. But before that, we need to set the property contextualMenuAction to the action that is bound to our custom menu item, so when the navigation delegate of WKWebView is called to create a new window, we can check contextualMenuAction to find out the action that is really supposed to be triggered.

And finally we override didCloseMenu to make sure that the property contextualMenuAction is reset to nil after the menu has closed. The navigation delegate to create new windows can be also called by WKWebView without having any contextual menu involved (for example if JavaScript code of the web site creates new windows), therefore this property can’t be allowed to keep its last value until the contextual menu is used again.


  override func didCloseMenu(_ menu: NSMenu, with event: NSEvent?) {
    super.didCloseMenu(menu, with: event)

    DispatchQueue.main.asyncAfter(deadline: .now() + 0.7) {
      self.contextualMenuAction = nil
    }
  }

It is important set contextualMenuAction to nil after a delay, because the contextual menu action is processed asynchronously. Which means the menu can close before the action is actually triggered and the navigation delegate method is called. Theoretically you can also clear this property within the navigation delegate method, but in practice it can happen that this method is not called if something goes wrong while processing the menu action (like there’s no network connection, the URL is invalid etc). Therefore it’s better to clear the property after closing the menu here.

And as the final action we need to implement the navigation delegate method of WKWebView which is called to create a new window:


  func webView(_ webView: WKWebView, createWebViewWith configuration: WKWebViewConfiguration,
                  for navigationAction: WKNavigationAction,
                  windowFeatures: WKWindowFeatures) -> WKWebView? {

    if let customAction = (webView as? MyWebView)?.contextualMenuAction {
      let url = navigationAction.request.url

      if customAction == .openInNewTab {
        createNewTab(for:url)
      } else if customAction == .addToBookmarks {
        addBookmark(url:url)
      }
      return nil

    } else {
      return createWebView(with:configuration)
    }
  }

First we check if the  property contextualMenuAction is not nil. In this case one of the custom actions must be performed. The value of this property tells us, which action this is. Otherwise the default action (creating a new window) must be performed.

You see that it’s not too complicated to implement a custom contextual menu for WKWebView on the Mac. Basically we need to „hijack“ one of the existing default menu items to let the WKWebEngine pass the context (the URL of the object for which the contextual menu was opened) to the App which would be otherwise inaccessible. The solution is not totally „clean“ because we have to use undocumented identfiers of the default menu items, but because these are simple strings you can’t break anything. The worst thing that can happen is that these identifiers change in future macOS releases and then the custom menu items will be missing unless you add the new identifieres as well. But this is very unlikely.

„Classic“ MapKit on watchOS

If you want to show a native map view within your watchOS App you have currently two options: using either the „classic“ watchOS Interface elements or SwiftUI.

SwiftUI is relatively new and was introduced with watchOS 7 on the Apple Watch (and with iOS 14 on the iPhone/iPad). SwiftUI is still a work in progress, so there are limitations which will be resolved in the future. But when it comes to the MapKit it is already much more powerful than the „classic“ watchOS interface elements. So in case you don’t care about compatibility with older watchOS releases and don’t mind in creating your user interface in code, you should use SwiftUI.

But if your App should still work under watchOS 6 and older or you want to create your user interface graphically via Storyboard, you may still need to use the classic watchOS API. Unfortunytely the classic MapKit API is extremly limited. While the map view of SwiftUI lets you scroll and zoom the map out of the box, the classic watchOS API won’t support this at all. Even worse, tapping the Map will immediatelly quit your App and will launch Apple’s Maps App. Also while SwiftUI lets you place any number of annotations (for example pins, markers) on the map, the classic watchOS only supports up to 5 annotations.

This blog post will explain how you can implement scrolling and zooming capabilities with the classic watchOS APIs, so you can get a similar experience as with the new SwiftUI.

The topic of this blog post is based on the experiences of my iOS/watchOS Apps Pado (all about geo tracking for sports, vacation etc), and Wigemo „“ (a general Maps App where you can save and manage your favorite places, plan vacations etc, with a direct link to Wikipedia). In both Apps you have a MapView which can be scrolled and zoomed without restriction. In Pado the map will also show tracks (overlays) on the Map and in Wigemo the map can have any number of annotations (pins) and is even able to cluster these pins (combine multiple pins into one with a number if they are getting too close). Check out the video that is linked here, how this looks in Wigemo. Especially Pado was developeed many years before Apple introduced SwiftUI, therefore SwiftUI was never an option here.

Step One: Creating your Userinterface in a Storyboard

Since we create an App using the classic watchOS API, the user interface is build via Storyboard. In the Storyboard, just drag a Map (WKInterfaceMap) from the Library window into an Interface controller. Because we use the digital crown for zooming, also add a WKInterfacePicker into the Interface Controller in the Storyboard. Then connect both with the IBOutlet varaibales in the code file of the InterfaceController.

class InterfaceController: WKInterfaceController {

    @IBOutlet weak var mapView: WKInterfaceMap!
    @IBOutlet weak var pickerView: WKInterfacePicker!

Issue One: Taps are leaving the App

The first issue to solve would be that tapping the map would immediatelly quit the App and switch to Apple’s Map App. This task is an easy one, all that needs to be done is to remove the checkmark of the „enabled“ checkbox in the „Maps“ section of the „Attributes Inspector“ panel in the Storyboard in XCode. Then the map view will ignore taps and so the user won’t be kicked out of the App anymore.

Issue Two: Zooming

For the zoom feature we follow the convention which almost all popular map providers (OpenStreetMap, Google, Apple etc) are using: A zoom level of 0 covers the whole earth, adding 1 to the zoom level will divide the visible area by 2 (horizontally and vertically). With the length of the equator and the current zoom level it is then possible to find out how large the visible area is. Together with the center coordinate of this area we have all information to configure the mapView to show exactly this region.

Therefore we first define a few constants for the min and max zoom level and the equator length (in meters). The zoom level is initialized with a useful default value and will later change whenever the user uses the digital crown to zoom in or out. The visible distance of the Map can be directly calculated by the equator length and the zoom level. The location is the current center coordinate of the map, initialized with any coordinate you like (in this example this is simply a fixed one, but you can also use the Location Manager to ask for the user location) and it is also later changed by the user when scrolling the map.


    let minZoomLevel = 4.0
    let maxZoomLevel = 18.0
    let equatorLength = 40075000.0

    var zoomLevel = 10.0
    var visibleDistance: Double { return equatorLength / pow(2, zoomLevel) }
    var location = CLLocationCoordinate2D(latitude: 49, longitude: 9)

The next step will be to setup the Picker control which is used to zoom in and out. The first step is to connect the IBAction „zoomAction“ in the code with the picker control in the storyboard. „zoomAction” will be called whenever the digital crown is rotated.

In the „awake“ method we initialize the Picker control. In order to get the zooming more smooth, each step of the digital crown should increase or decrease the zoom level by 0.5, therefore the picker control needs twice as many picker items as zoom levels are available (you can add more items per zoom level to get an even more smooth experience, but this also has the disadvantage that zooming fully in or out requires more steps and needs more time which can be less comfortable to use – using steps of 0.5 is a good compromise). These PickerItems don’t need a title because they are invisble anyways, so we simply reuse a single PickerItem and add it multiple times into the Picker control. Then we set the currently selected Picker item to the one which represents the current (in this case the default) zoom level. And last, because the Picker Control should be invisible, its height is set to 0. The „zoomAction“ method is called whenever the crown is rotated which will then calulate the new zoom level and then update the map. As you can see, zooming can be implemented in just a few lines of code and is not very compicated.


    override func awake(withContext context: Any?) {
        let count = Int(maxZoomLevel-minZoomLevel)*2
        let pickerItem = WKPickerItem()
        let pickerItems = [WKPickerItem](repeatElement(pickerItem, count:count))
        pickerView.setItems(pickerItems)
        let idx = Int((zoomLevel - minZoomLevel) * 2)
        pickerView.setSelectedItemIndex(idx)
        pickerView.setHeight(0)
    }

    @IBAction func zoomAction(_ value: Int) {
        zoomLevel = Double(value) / 2 + minZoomLevel
        setupMap()
    }

Issue Three: Scrolling

In order to scroll the visible map region we can use a standard PanGestureRecognizer. Therefore simply add one into the maps view in the Storyboard and link it to the IBAction „scrollAction“ in the InterfaceController code file. Whenever the user moves his finger on the map, the „scrollAction” method will be called where we can then scroll the map accordingly.

The pan gesture recognizer will tell us the position of the finger on the screen, so to scroll the map we need the horizontal and vertical distance in pixels the finger has moved between calls of this method and transform these values into changes of the geographical coordinate. This calculation can be a little bit complicated because the earth is not flat, but projected on a flat screen. So moving the finger a certain distance on the screen does not translate in the same changes for latitude and longitude when beeing near the poles or near the equator. But fortunytely we do not need a perfect solution, an approximation is fine. More information about the details of the approximation which I’ve used can be found on this web page. Based on the movement of the finger in pixels this method calculates the changes for the latitude (dY) and longitude (dX), normalizes or „clips” these if they are getting to large or small and then use these to update the new center coordinate and finally update the map. We only accept longitude values between -180 and 180 and latitude values between -75 and 75. Though valid latitude values lie between -90 and 90, the limit to -75 to 75 makes life a lot of easier for this sample project. These values are used for the center of the visible map view, therefore the total visible range can be much larger. And because the map view can not handle a visible area which goes beyond the -90/90 boundary this limit will prevent any issues. If you need to inspect the poles as well, you need to add some more checks, which is omitted here to keep the example code simple.


    var touchPoint = CGPoint()

    @IBAction func scrollAction(_ sender: WKPanGestureRecognizer) {
        let pt = sender.locationInObject()
        if sender.state == .began {
            touchPoint = pt

        } else {
            let screenSize = WKInterfaceDevice.current().screenBounds.size
            let distance = visibleDistance

            let x = (touchPoint.x - pt.x) / screenSize.width * distance
            let y = (touchPoint.y - pt.y) / screenSize.width * distance

            let R:CGFloat = 111111
            var dY = y/R
            var dX = x/(R*cos(location.latitude/180 * Double.pi))

            while location.longitude + dX < -180 { dX += 360 }
            while location.longitude + dX > 180 { dX -= 360 }

            if location.latitude - dY > 75 { dY = location.latitude - 75 }
            if location.latitude - dY < -75 { dY = location.latitude + 75 }

            touchPoint = pt

            location.longitude += dX
            location.latitude -= dY
            setupMap()
        }
    }

 

Last Step

The very last step would be to update the map each time the zoom level has changed or the map is scrolled. This method is very simple again, based on the current center location and the visible distance (calculated from the zoom level) the region can be calculated using a MapKit method and then it will be passed to the map view which will then display the new region on the screen.


func setupMap() {
    let distance = visibleDistance
    let region = MKCoordinateRegion(center: location, latitudinalMeters: distance, longitudinalMeters: distance)
    mapView.setRegion(region)
}

Overall there’s not that much code necessary to implement scrolling and zooming for the classic watchOS API.

The complete source code of this App (XCode project) is also available for download.

Hello World! – Introduction

This blog is all about developing apps and applications for the iOS platform (iPhone, iPad, iPod Touch), watchOS (Apple Watch) and the macOS (Mac). The blog posts will usually use Swift as programming language for all code samples. Most topics are based on my own experiences and issues I encountered while developing my own Apps, so hopefully they can help other developers who run into similar problems.

There’s still my old blog from 2009-2013 available through the „Archives“ link at the top of the blog site. Most of these old blog posts are from the „classic“ ages of the iOS (up to iOS 5) and therefore kind of outdated and no longer really useful.

Feel free to ask for topics I should write about. I’d be glad if  I get feedback and comments.