View hidden infrastructure in AR

View on GitHub

Sample viewer app

Visualize hidden infrastructure in its real-world location using augmented reality.

Add pipe infrastructure to the map View hidden infrastructure in AR

Use case

You can use AR to "x-ray" the ground to see pipes, wiring, or other infrastructure that isn't otherwise visible. For example, you could use this feature to trace the flow of water through a building to help identify the source of a leak.

How to use the sample

When the sample is launched, you'll see a map centered on your current location. Tap "Add" to launch the sketch editor and draw pipes around your location. After drawing the pipes, input an elevation value to place the drawn infrastructure above or below ground. When you're ready, tap the camera button to view the infrastructure you drew in AR.

There are two calibration modes in the sample: roaming and local. In roaming calibration mode, your position is updated automatically from the location data source every second. Because of that, you can only adjust heading, not position or elevation. This mode is best when working in a large area, where you would travel beyond the useful range of ARKit.

When you're ready to take a more precise look at the infrastructure, switch to local calibration mode. In local calibration mode, you can make fine adjustments to location, elevation, and heading to ensure the content is exactly where it should be.

How it works

Draw pipes on the map. See more in the "Sketch on map" sample to learn how to use the sketch editor for creating graphics.
When you start the AR visualization experience, create and show the ArcGISARView.
Access the sceneView property of the AR View and set the space effect none and the atmosphere effect to transparent.
Create an elevation source and set it as the scene's base surface. Set the navigation constraint to none to allow going underground if needed.
Listen to ARKit events with ARSCNViewDelegate. Provide feedback on ARKit tracking as needed.
- Note: ARKit feedback should only be provided when the location data source is not continuously updating, i.e. in "local" mode.
- When the location is continuously being updated, ARKit tracking never has time to reach the "normal" state, so feedback is not useful.
Configure a graphics overlay and renderer for showing the drawn pipes. This sample uses an AGSSolidStrokeSymbolLayer with an AGSMultilayerPolylineSymbol to draw the pipes as tubes. Add the drawn pipes to the overlay.
Configure the calibration experience.
- When in "roaming" (continuous location update) mode, only heading calibration should be enabled. In continuous update mode, the user's calibration is overwritten by sensor-based values every second.
- When in "local" mode, the user needs to be able to adjust the heading, elevation, and position; position adjustment is achieved by panning.
- This sample uses a basemap as a reference during calibration; consider how you will support your user's calibration efforts. A basemap-oriented approach won't work indoors or in areas without readily visible, unchanging features like roads.

Relevant API

AGSGraphicsOverlay
AGSMultilayerPolylineSymbol
AGSSketchEditor
AGSSolidStrokeSymbolLayer
AGSSurface
ArcGISARView

About the data

This sample uses Esri's world elevation service to ensure that the infrastructure you create is accurately placed beneath the ground.

Real-scale AR relies on having data in real-world locations near the user. It isn't practical to provide pre-made data like other Runtime samples, so you must draw your own nearby sample "pipe infrastructure" prior to starting the AR experience.

Additional information

This sample requires a device that is compatible with ARKit.

Note that unlike other scene samples, a basemap isn't shown most of the time, because the real world provides the context. Only while calibrating is the basemap displayed at 50% opacity, to give the user a visual reference to compare to.

You may notice that pipes you draw underground appear to float more than you would expect. That floating is a normal result of the parallax effect that looks unnatural because you're not used to being able to see underground/obscured objects. Compare the behavior of underground pipes with equivalent pipes drawn above the surface - the behavior is the same, but probably feels more natural above ground because you see similar scenes day-to-day (e.g. utility wires).

World-scale AR is one of three main patterns for working with geographic information in augmented reality. Augmented reality is made possible with the ArcGIS Runtime Toolkit. See Augmented reality in the guide for more information about augmented reality and adding it to your app.

This sample uses a combination of two location data source modes: continuous update and one-time update, presented as "roaming" and "local" calibration modes in the app. The error in the position provided by ARKit increases as you move further from the origin, resulting in a poor experience when you move more than a few meters away. The location provided by GPS is more useful over large areas, but not good enough for a convincing AR experience on a small scale. With this sample, you can use "roaming" mode to maintain good enough accuracy for basic context while navigating a large area. When you want to see a more precise visualization, you can switch to "local" (ARKit-only) mode and manually calibrate for best results.

Sample Code

ViewHiddenInfrastructureARPipePlacingViewController.swift

ViewHiddenInfrastructureARExplorerViewController.swift

ViewHiddenInfrastructureARCalibrationViewController.swift

Use dark colors for code blocksCopy
// Copyright 2020 Esri
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//   http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import UIKit
import ArcGIS

class ViewHiddenInfrastructureARPipePlacingViewController: UIViewController {
    // MARK: Storyboard views

    /// The label to display pipe infrastructure planning status.
    @IBOutlet var statusLabel: UILabel!
    /// The bar button to add a new geometry.
    @IBOutlet var sketchBarButtonItem: UIBarButtonItem! {
        didSet {
            sketchBarButtonItem.possibleTitles = ["Add", "Done"]
        }
    }
    /// The bar button to remove all geometries.
    @IBOutlet var trashBarButtonItem: UIBarButtonItem!
    /// The bar button to launch the AR viewer.
    @IBOutlet var cameraBarButtonItem: UIBarButtonItem!

    /// The map view managed by the view controller.
    @IBOutlet var mapView: AGSMapView! {
        didSet {
            mapView.map = AGSMap(basemapStyle: .arcGISImagery)
            mapView.graphicsOverlays.add(pipeGraphicsOverlay)
            mapView.sketchEditor = AGSSketchEditor()
        }
    }

    // MARK: Properties

    /// A graphics overlay for showing the pipes.
    let pipeGraphicsOverlay: AGSGraphicsOverlay = {
        let overlay = AGSGraphicsOverlay()
        overlay.renderer = AGSSimpleRenderer(
            symbol: AGSSimpleLineSymbol(style: .solid, color: .red, width: 2)
        )
        return overlay
    }()

    /// A KVO on the graphics array of the graphics overlay.
    var graphicsObservation: NSKeyValueObservation?
    /// The data source to track device location and provide updates to location display.
    let locationDataSource = AGSCLLocationDataSource()
    /// The elevation source with elevation service URL.
    let elevationSource = AGSArcGISTiledElevationSource(url: URL(string: "https://elevation3d.arcgis.com/arcgis/rest/services/WorldElevation3D/Terrain3D/ImageServer")!)
    /// The elevation surface for drawing pipe graphics relative to groud level.
    let elevationSurface = AGSSurface()

    // MARK: Methods

    /// Add a graphic from the geometry of the sketch editor on current map view.
    ///
    /// - Parameters:
    ///   - polyline: A polyline geometry created by the sketch editor.
    ///   - elevationOffset: An offset added to the current elevation surface,
    ///                      to place the polyline (pipes) above or below the ground.
    func addGraphicsFromSketchEditor(polyline: AGSPolyline, elevationOffset: NSNumber) {
        guard let firstpoint = polyline.parts.array().first?.startPoint else { return }
        elevationSurface.elevation(for: firstpoint) { [weak self] (elevation: Double, error: Error?) in
            guard let self = self else { return }
            let graphic: AGSGraphic
            if error != nil {
                graphic = AGSGraphic(geometry: polyline, symbol: nil)
                self.setStatus(message: "Pipe added without elevation.")
            } else {
                let elevatedPolyline = AGSGeometryEngine.geometry(bySettingZ: elevation + elevationOffset.doubleValue, in: polyline)
                graphic = AGSGraphic(geometry: elevatedPolyline, symbol: nil, attributes: ["ElevationOffset": elevationOffset.doubleValue])
                if elevationOffset.intValue < 0 {
                    self.setStatus(message: "Pipe added \(elevationOffset.stringValue) meter(s) below surface.")
                } else if elevationOffset.intValue == 0 {
                    self.setStatus(message: "Pipe added at ground level.")
                } else {
                    self.setStatus(message: "Pipe added \(elevationOffset.stringValue) meter(s) above surface.")
                }
            }
            self.pipeGraphicsOverlay.graphics.add(graphic)
        }
    }

    // MARK: Actions

    @IBAction func sketchBarButtonTapped(_ sender: UIBarButtonItem) {
        guard let sketchEditor = mapView.sketchEditor else { return }
        switch sketchEditor.isStarted {
        case true:
            // Stop the sketch editor and create graphics when "Done" is tapped.
            if let polyline = sketchEditor.geometry as? AGSPolyline, polyline.parts.array().contains(where: { $0.pointCount >= 2 }) {
                // Let user provide an elevation if the geometry is a valid polyline.
                presentElevationAlert { [weak self] elevation in
                    self?.addGraphicsFromSketchEditor(polyline: polyline, elevationOffset: elevation)
                }
            } else {
                setStatus(message: "No pipe added.")
            }
            sketchEditor.stop()
            sketchEditor.clearGeometry()
            sender.title = "Add"
        case false:
            // Start the sketch editor when "Add" is tapped.
            sketchEditor.start(with: nil, creationMode: .polyline)
            setStatus(message: "Tap on the map to add geometry.")
            sender.title = "Done"
        }
    }

    @IBAction func trashBarButtonTapped(_ sender: UIBarButtonItem) {
        pipeGraphicsOverlay.graphics.removeAllObjects()
        setStatus(message: "Tap add button to add pipes.")
    }

    // MARK: UI

    func setStatus(message: String) {
        statusLabel.text = message
    }

    func presentElevationAlert(completion: @escaping (NSNumber) -> Void) {
        let alert = UIAlertController(title: "Provide an elevation", message: "Between -10 and 10 meters", preferredStyle: .alert)
        alert.addTextField { textField in
            textField.keyboardType = .numbersAndPunctuation
            textField.placeholder = "3"
        }
        let cancelAction = UIAlertAction(title: "Cancel", style: .cancel)
        let doneAction = UIAlertAction(title: "Done", style: .default) { [textField = alert.textFields?.first] _ in
            let distanceFormatter = NumberFormatter()
            // Format the string to an integer.
            distanceFormatter.maximumFractionDigits = 0

            // Ensure the elevation value is valid.
            guard let text = textField?.text,
                !text.isEmpty,
                let elevation = distanceFormatter.number(from: text),
                elevation.intValue >= -10,
                elevation.intValue <= 10 else { return }
            // Pass back the elevation value.
            completion(elevation)
        }
        alert.addAction(cancelAction)
        alert.addAction(doneAction)
        alert.preferredAction = doneAction
        present(alert, animated: true)
    }

    // MARK: UIViewController

    override func prepare(for segue: UIStoryboardSegue, sender: Any?) {
        if segue.identifier == "showViewer",
            let controller = segue.destination as? ViewHiddenInfrastructureARExplorerViewController,
            let graphics = pipeGraphicsOverlay.graphics as? [AGSGraphic] {
            controller.pipeGraphics = graphics
        }
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        // Add the source code button item to the right of navigation bar.
        (navigationItem.rightBarButtonItem as? SourceCodeBarButtonItem)?.filenames = [
            "ViewHiddenInfrastructureARPipePlacingViewController",
            "ViewHiddenInfrastructureARExplorerViewController",
            "ViewHiddenInfrastructureARCalibrationViewController"
        ]

        // Configure the elevation surface used to place drawn graphics relative to the ground.
        elevationSurface.elevationSources.append(elevationSource)
        elevationSource.load { [weak self] error in
            guard let self = self else { return }
            if let error = error {
                self.presentAlert(error: error)
            }
        }

        // Add a KVO to update the button states.
        graphicsObservation = pipeGraphicsOverlay.observe(\.graphics, options: .initial) { [weak self] overlay, _ in
            guard let self = self else { return }
            // 'NSMutableArray' has no member 'isEmpty'; check its count instead.
            let graphicsCount = overlay.graphics.count
            let hasGraphics = graphicsCount > 0
            self.trashBarButtonItem.isEnabled = hasGraphics
            self.cameraBarButtonItem.isEnabled = hasGraphics
        }

        // Set location display.
        setStatus(message: "Adjusting to your current location…")
        locationDataSource.locationChangeHandlerDelegate = self
        locationDataSource.start { [weak self] error in
            guard let self = self else { return }
            if let error = error {
                self.presentAlert(error: error)
            }
        }
    }
}

// MARK: - Location change handler

extension ViewHiddenInfrastructureARPipePlacingViewController: AGSLocationChangeHandlerDelegate {
    func locationDataSource(_ locationDataSource: AGSLocationDataSource, locationDidChange location: AGSLocation) {
        // Stop the location adjustment immediately after the first location update is received.
        let newViewpoint = AGSViewpoint(center: location.position!, scale: 1000)
        mapView.setViewpoint(newViewpoint, completion: nil)
        setStatus(message: "Tap add button to add pipes.")
        sketchBarButtonItem.isEnabled = true
        locationDataSource.locationChangeHandlerDelegate = nil
        locationDataSource.stop(completion: nil)
    }
}