Feature Service

A feature service is a data service that stores and hosts spatial and nonspatial data online. In a feature service, spatial datasets are feature layers and non-spatial datasets are tables. You can query, edit and analyze data in a feature service. Feature service URLs are typically in the formats shown below.

URL for a feature service layer or table hosted in ArcGIS Online: https://<host>/<uniqueID>/ArcGIS/rest/services/<serviceName>/FeatureServer/<Layer or Table ID>

URL for a feature service layer or table hosted in ArcGIS Enterprise: https://<host>/<webadaptor>/rest/services/Hosted/<serviceName>/FeatureServer/<Layer or Table ID>.

Refer to the feature service documentation for ArcGIS Online and ArcGIS Enterprise for more details.

Register a GIS

Authentication will be required when reading or saving a secured feature service hosted by ArcGIS Online or ArcGIS Enterprise. GeoAnalytics Engine supports authentication by registering a GIS (i.e. a connection to ArcGIS Online or ArcGIS Enterprise) with the register_gis function.

The default GIS in register_gis is ArcGIS Online. The following example shows registering ArcGIS Online as a GIS with a username and password for a built-in account.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import geoanalytics
geoanalytics.register_gis("myGIS", username="User", password="p@ssw0rd")

To connect to ArcGIS Enterprise, specify the portal URL as shown in the following example for a built-in account.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import geoanalytics
geoanalytics.register_gis("myPortal", "https://example.com/portal", username="User", password="p@ssw0rd")

Each GIS name can only be registered once. To remove a GIS, use the unregister_gis function as shown in the following example.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import geoanalytics
geoanalytics.unregister_gis("myGIS")

In addition to built-in accounts, the following authentication schemes are supported:

  • Web-tier authentication secured with PKI
  • User authentication with OAuth 2.0

Web-tier authentication secured with PKI

To connect to a PKI protected ArcGIS Enterprise, you can use a PKCS12 formatted certificate file (for example .pfx or .p12) and password when logging into your ArcGIS Enterprise using PKI-based client certificate authentication.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
geoanalytics.register_gis("myPortal", "https://example.com/portal",
                          cert_file=r"\\path\to\mycert.pfx", cert_password="p@ssw0rd")

User authentication with OAuth 2.0

GeoAnalytics Engine supports OAuth 2.0 as an authentication method and acts as a serverless native application when using OAuth 2.0 authorization with ArcGIS Online or ArcGIS Enterprise. This mode of authorization requires a client id. If you don't have a client id, follow the steps below to obtain one by registering a new application with your GIS. Only one client id is required, so if your GIS has one already, you may use it instead of creating a new application.

  1. Log in to your ArcGIS Online or ArcGIS Enterprise organization.
  2. Go to the Content tab.
  3. Click New item, then select Application.
  4. On the New item dialog, select Other application and click Next.
  5. Type in a title for the application item. Optionally, specify the folder, tags, and summary.
  6. Click Save to add the item.
  7. On the item details page of this newly created application, browse to the Credentials section and find the Client ID. The string listed is the value that you will pass in as the client_id parameter when registering a GIS

You can register an OAuth 2.0 protected GIS with your client id and an authorization code. To generate an authorization code, call register_gis with only the name, url, and client_id parameters specified.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
geoanalytics.register_gis("myPortal", "https://example.com/portal", client_id="drg5d5r575rfb")
Result
Use dark colors for code blocksCopy
1
2
IllegalArgumentException: 'authorization_code' is required when 'client_id' is provided. You can generate an
authorization code through the following URL: ...

Open the url provided in the exception in a browser (you may need to sign in) and copy the authorization code. You can now register your GIS as shown below.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
geoanalytics.register_gis("myPortal", "https://example.com/portal",
                          client_id="drg5d5r575rfb", authorization_code="3464736")

Read from feature services

GeoAnalytics Engine supports loading data from feature services into a Spark DataFrame for use with any operations supported on a DataFrame in GeoAnalytics Engine or PySpark.

The following example shows the Python syntax for loading a feature service into a Spark DataFrame, where URL is the URL to a feature service layer or table. URL is not required if specified using the DataFrameReader option listed in the table below.

Use dark colors for code blocksCopy
1
spark.read.format("feature-service").option().load(URL)

Below is a table of options that GeoAnalytics Engine supports when loading a feature service with Spark DataFrameReader.

DataFrameReader optionExampleDescription
url.option("url", "https://services.arcgis.com/P3ePLMYs2RVChkJx/ArcGIS/rest/services/USA_Counties/FeatureServer/0")The URL of the feature service layer.
gis.option("gis", "myGIS")Reference to a registered GIS. Required for secured feature services if token is not provided.
token.option("token", "myToken")ArcGIS Online or portal token. Required for secured feature services if gis is not provided.
timeout.option("timeout", 60)Timeout in seconds for loading a feature service.
connectTimeout.option("connectTimeout", 60)Timeout in seconds for connecting to the GIS.
numRetries.option("numRetries", 5)The total number of retries that will be made after failing to read from a feature service layer. The default is 2.
extent.option("extent", "-90.0, 30.0, 90, 50")Filters the feature service layer by the spatial extent specified using the format "<x min>, <y min>, <x max>, <y max>". Values should be defined in the same units as the spatial reference of the feature layer.

When loading a public feature service layer or table, you can pass the URL without options 'gis' or 'token':

Use dark colors for code blocksCopy
1
spark.read.format("feature-service").load(URL)

When loading a secured feature service layer or table, a GIS connection or token will be required. For example:

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import geoanalytics
geoanalytics.register_gis("myGIS", username="User", password="p@ssw0rd")
spark.read.format("feature-service").option("gis", "myGIS").load(URL)

For more examples on how to load a feature service layer, refer to the Read from feature services tutorial.

Write to feature services

GeoAnalytics Engine extends Spark and supports saving data in feature services. You can serve and share feature services over the internet. You can also access, visualize, or edit the published feature services in an ArcGIS Online or Enterprise portal.

The following script shows the Python syntax of saving feature services that GeoAnalytics Engine currently supports:

Use dark colors for code blocksCopy
1
df.write.format("feature-service").option().save()

Below is a table of options and modes that GeoAnalytics Engine supports when saving a feature service with Spark DataFrameWriter. The default mode is to create a new feature service.

DataFrameReader optionExampleDescription
append.mode("append")Append to existing layer in a feature service.
overwrite.mode("overwrite")Overwrite existing layer in a feature service.
gis.option("gis", "myGIS")Reference to a registered GIS. Required if token is not provided.
token.option("token", myToken)ArcGIS Online or portal token. Required if gis is not provided.
portalUrl.option("portalUrl", "https://arcgis.com")The Arcgis Online or Portal URL for saving new feature services. Required if token is provided.
serviceName.option("serviceName", "newService")The name of the feature service that will be created if writing to a new feature service.
layerId.option("layerId", 0)The ID of the layer within a feature service. Required when using append or overwrite mode if layerName is not provided.
layerName.option("layerName", "newLayer")The name of the layer within a feature service. Required when using append or overwrite mode if layerId is not provided.
serviceUrl.option("serviceUrl", serviceUrl)The URL of the feature service to append or overwrite to. Required when using append or overwrite mode.
path.option("path", "newLayer")Alternative method for specifying the layerName.
truncate.option("truncate", True)Remove all records in the layer before appending. The default is False.
maxParallelism.option("maxParallelism", 6)Maximum number of writers that will be used asynchronously when writing to a layer. The default is 12. A larger number can be specified if the data store you're writing to has the capacity to process more asynchronous requests.
description.option("description", "my gis data")Item description for a new feature service.
tag.option("tag", "flood")Item tags for a new feature service.
snippet.option("snippet", "flood")Item snippet for a new feature service.
datasourceType.option("datasourceType", "spatiotemporal")Optional when writing to ArcGIS Enterprise. Choose from spatiotemporal or relational for the data store type. The default is relational.
maxStringLength.option("maxStringLength", 20)Specify the maximum allowed length for a string field.
renameFields.option("renameFields", True)Rename the field by replacing the space in the field name with an underscore. The default is False.
upsertMatchingField.option("upsertMatchingField", "id")The layer field to be used when matching features with upsert. Any field that has a unique index can be used. When writing to a new layer and specifying the upsertMatchingField, a unique index will be created on the specified field.

When saving a feature service, either a registered GIS or a token is required. Below is an example of registering a GIS and saving a feature service.

Python
Use dark colors for code blocksCopy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
import geoanalytics
geoanalytics.register_gis("myGIS", username="User", password="p@ssw0rd")

data = [
    ('{"x": -0.126,"y": 51.504,"m": 187.73}', 43.26),
    ('{"x": -0.126,"y": 51.504}', 97.22)
]
df = spark.createDataFrame(data, ["esri_json", "value"]) \
          .withColumn("point", ST.point_from_esri_json("esri_json", 4326))

df.write.format("feature-service") \
        .option("gis", "myGIS") \
        .option("serviceName", "esri_json_output") \
        .save()

For more examples on how to save feature services, refer to the Write to feature services tutorial.

Usage notes

  • A layer or table ID must be included in the feature service URL when loading.
  • Secured feature services can be read by either providing a registered GIS or a token.
  • When loading a feature service layer with geometries, the geometry field will be included in the result DataFrame automatically as as a point, multipoint, line, or polygon column.
  • A registered GIS or a token and Portal URL are required when saving a feature service.
  • When saving a Spark DataFrame with a geometry column, a feature layer will be created. For a Spark DataFrame without a geometry column, a table will be created in the new feature service. A spatial reference is required to be set on the geometry column when saving to a feature layer.
  • The generic geometry type is not supported when saving a feature service. The geometry type should be one of the following types: point, multipoint, line, or polygon.
  • Any 64-bit signed integer columns (LongType in Spark) will be cast to DoubleType and written to feature service as esriFieldTypeDouble.
  • Beginning with version 1.2.0, esriFieldTypeBigInteger fields can be read from feature services as LongType columns. Reading esriFieldTypeBigInteger fields is not supported in previous versions.
  • The option layerId can only be used to overwrite or append to an existing layer. Use the layerName option when creating a new layer.
  • The option serviceName should only be used when writing to a new feature service, and the service name needs to be unique within your ArcGIS Enterprise or ArcGIS Online organization. To overwrite or append to an existing layer, use serviceUrl.
  • When providing serviceUrl, layerId, or layerName must also be specified.
  • When saving a feature service, the hasZ and hasM properties will always be set to false.

Your browser is no longer supported. Please upgrade your browser for the best experience. See our browser deprecation post for more details.