fbpx

Programmatic ARKit Positioning

Currently iOS does not support markers or targets in a similar fashion to Vuforia (at the moment of this writing iOS 11.3 has a similar feature but it’s not released). So what is the best way to drop an animation on a target? Assuming you’re dealing with a rectangular object we can use the Vision framework’s VNDetectRectanglesRequest

  // viewController 
  let pixelBuffer      = self.sceneView.session.currentFrame?.capturedImage 
  let ciImage          = CIImage(cvImageBuffer: pixelBuffer!) 
  let handler          = VNImageRequestHandler(ciImage: ciImage) 
  let rectService      = RectangleDetectionService(sceneView: self.sceneView)
  let rectangleRequest = VNDetectRectanglesRequest(completionHandler: rectService.handleRectangles) 
  do { 
    try handler.perform([rectangleRequest]) 
  } catch { 
    log.error(error) 
  } 

Above we’ve transformed the pixel buffer being received from ARKit (sceneView) into a ciImage that we then pass to the Vision Framework’s request handler. Then we create a new instance of our custom , then defined the completion handler once a rectangle is detected.

// RectangleDetectionService (rectService)

func handleRectangles(request: VNRequest, error: Error?) {
  guard let observations = request.results as? [VNRectangleObservation] else {
    return
  }
 
  // get the highest confidence rectangle observation
  let highConfidenceObservation = observations.max { a, b in a.confidence < b.confidence }
 
   guard let highestConfidenceObservation = highConfidenceObservation else {
    return
   }
 
   // Calculates the position of the 4 corners
   let points = [highConfidenceObservation?.topLeft,
                 highConfidenceObservation?.topRight,
                 highConfidenceObservation?.bottomRight,
                 highConfidenceObservation?.bottomLeft]
 
   // Calculates center point
   let center         = CGPoint(x: (highConfidenceObservation?.boundingBox.midX)!,
                                y: (highConfidenceObservation?.boundingBox.midY)!)

   // create a hit test simulating a user touch at the point within the sceneview
   let hitTestResults = self.sceneView.hitTest(center, types: [.existingPlaneUsingExtent, .featurePoint])
   guard let result = hitTestResults.first else {
     print("Rectangle detection error")
     return
   }
  
    // now that we've calculated a center point and created a hit test in the arView
    anchor = ARAnchor(transform: result.worldTransform)
    self.sceneView.session.add(anchor: anchor)
}

You should now have an ARAnchor at the center point of the detected rectangle

Dropbox & Webpack

I’ve had a major issue with Angular 2/Webpack for the past few months. I store all my development files in Dropbox as a second layer of backup and an issue came up that would make me bounce the server every time I changed a file. Not cool.

The office joke is “it’s Tmux’s fault” and we eliminated Tmux, uninstalled Node, NPM, Yarn, etc. Nothing worked. My colleague Alex pointed out the repo was in Dropbox so I moved it to another folder and magically it started working again.

So take note, if you’re having live reload or similar issues and it’s in Dropbox, take it out.

CSS & SVG Based Tooltips

While working on a quick cleanup sprint I ran into an issue with a tooltip-box. The original design was using a raster-based image as a CSS background and it wasn’t meeting our needs. Googling revealed the problem had been solved but still didn’t give me a shadow on the arrow tip. Below is the quick and dirty structure of a CSS & SVG based tooltip box.

See the Pen CSS-Only Tooltip w/ Shadow by Ryan Stone (@ryanstone) on CodePen.