Staying safe and having fun with ARKit

A lot of people in the world are worrying about coronavirus and staying safe in the current situation. Let’s try to use ARKit to tell others how important it is to wear a mask.

Let’s dive in. Create new iOS App with Storyboard-based UI. But before coding download a 3D-model: https://www.turbosquid.com/3d-models/n95-mask-3d-model-1574740 (don’t worry — it’s free!).

Add new file to the project a new SceneKit Catalog and name it Models.scnassets. Then, add there .mtl, .obj and .png files. Leave png file name as it is!

In storyboard Document outline select View and in Identity Inspector change class to ARSCNView (like on a screenshot above). Immediately after that connect this view to the ViewController.

So, now it’s time to code. Open ViewController.swift and add ARKit on the top:

import ARKit

Then implement viewDidLoad(), viewWillAppear() and viewWillDisappear() methods to add a support of ARKit:

class ViewController: UIViewController {
@IBOutlet weak var sceneView: ARSCNView!
override func viewDidLoad() {
  super.viewDidLoad()
  if !ARFaceTrackingConfiguration.isSupported {
    fatalError("Face tracking is not supported")
  }
  sceneView.delegate = self
}
override func viewWillAppear(_ animated: Bool) {
  super.viewWillAppear(animated)
  sceneView.session.run(ARFaceTrackingConfiguration())
}
override func viewWillDisappear(_ animated: Bool) {
  super.viewWillDisappear(animated)
  sceneView.session.pause()
  }
}

As you can see the app checks availability of face tracking API and sets our ViewController as a delegate of a sceneView. Then, it controls the session run and pause.

Of course, that’s not all. As we told app that this ViewController will be a delegate for sceneView, let’s implement it too:

extension WearMaskViewController: ARSCNViewDelegate {
  func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    if let url = Bundle.main.url(forResource: "mask", withExtension: "obj", subdirectory: "Models.scnassets"),
    let overlay = SCNReferenceNode(url: url) {
      overlay.load()
      let node = SCNNode()
      node.addChildNode(overlay)
      return node
    } else {
      return nil
    }
  }
}

The renderer(_: nodeFor:) method of ARSCNDelegate applies mask to your face. Our simplest version of the app is almost ready. The only thing left is to adjust scale and position of mask. Your can do it by changing these properties of mask.obj file:

You can see that my object has an offset, but Xcode doesn’t show it. In my experience the scale should be about 0.02 for each axis and also a little bit to the right and bottom

Think it works? Yes. But no. Remember: Apple respects our privacy, so we should add a description to the Privacy — Camera Usage Description key of Info.plist file. And… that’s it!

Hit run, have fun, improve it, stay safe and wear a mask! ✌️😷

P.S. All code is available on GitHub: https://github.com/antonoff/WearMask