Introduction
One request the Python API team has heard repeatedly from Web GIS administrators: What's the best way to move my content from a development Enterprise or organization to staging to production?
This Guide provides one possible roadmap for how to get content from one Web GIS to another - a template for demonstrating basic software concepts when transferring content so the workflow can be modified for other types of content and tailored for an administrator's particular need.
The clone_items()
function on the ContentManger aims to create an exact duplicate of one item with all its uses and functionality from an original organizational deployment, whether ArcGIS Online, Enteprise, or Kubernetes to a second organizational deployment. The relationship between any supported, related, or resource items to the items being cloned must be considered when cloning an item.
When talking about cloning items, let's establish some basic terminology. Cloning can refer to the transfer of items between any organizational deployment, from either ArcGIS Enterprise, ArcGIS Online, or ArcGIS Enterprise for Kubernetes to any other deployment type. Rather than tangling ourselves up when trying to distinguish between these portal types while writing about them, we'll refer to organization where the content originates as the source regardless of deployment type. We'll consider the organization to which we'll clone content as the target.
This document illustrates the workflow of gathering source items using a source administrator account, and cloning them into a target administrator account. More specifically, this guide walks through an approach of cloning hosted feature layers
and web maps
. Web Maps
can consume these layer items and their source services as operational layer building blocks. For information on each of these item types, please click the hyperlinks for each type.
This is an administrator workflow. Using administrator accounts guarantees all necessary privileges within the source to access the item and any dependencies and within the target to create the services and items. The target administrator subsequently can create user accounts and reassign item ownership and/or group membership according to need. This might also be the simplest workflow when the portal has users managed by an external identity store.
Supported Items
As originally written, Python API developers designed the clone_items()
function for transferring the following item types:
- Hosted Web Applications built with Web AppBuilder or shared using Configurable App Templates
- Web Maps
- Hosted Feature Layers
- Hosted Feature Layer Views
- Feature Collections
- Survey123 Forms
- Workforce Projects
- StoryMaps
- Operation Views
- Dashboards
- QuickCapture Projects
- ArcGIS Notebooks
- Simple Types
- Those items with a download option (see Data files for items that may be in a Web GIS and available for download), including zipped file geodatabases and shapefiles, code samples, zip files, and packages amongst others.
clone_items()
clones the dependencies for the more complex items listed above. For example, cloning an existing web application, clones the web map and all hosted feature layers referenced in the map.
clone_items()
will not clone map services and image services. Since these services can be published to servers other than the hosted server in a configuration, it's impossible for the function to determine where to publish them in the target. As a result, these items will copy over, but will continue to point back to the original source URL.
Let's work through examples of cloning individual items and inspect the results.
First, let's import the necessary libraries and connect to our source and target GIS:
Import libraries
from pathlib import Path
import sys
import pandas as pd
from arcgis.gis import GIS, Item
from arcgis.env import active_gis
from arcgis.features import FeatureLayerCollection
from arcgis.mapping import WebMap
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions. Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Connect to source and target portals
Let's start from a discovery position as an administrator. We'll simulate an administrator who's been testing and developing using an ArcGIS Online deployment as a development environment and wants to clone items into an on-premise ArcGIS Enterprise for further development.
source = GIS(profile="your_online_admin_profile")
print(source)
GIS @ https://geosaurus.maps.arcgis.com version:2023.3
target = GIS(profile="your_data_owner_profile")
print(target)
GIS @ https://example.url.com/portal version:10.3
Get started with cloning
Let's start with an immediate demonstration of what clone_items()
can do. We're logged in as an administrator and we'll get()
a specific hosted feature layer item owned by one of the users in the source. We'll then clone it into the target while utilizing the owner
parameter to specify a particular User
in the target to own the cloned content. We can quickly examine the resulting url
of the cloned feature layer to confirm the new item.
hosted_flyr = source.content.get("3213ff60f81c46a0a970ec31dde368ac")
hosted_flyr
hosted_flyr.url
'https://services7.arcgis.com/JEwYeAy2cc8qOe3o/arcgis/rest/services/Arkansas_Hospitals/FeatureServer'
cloned_flyr = target.content.clone_items(items=[hosted_flyr],
owner="api_data_owner",
folder="cloning_guide")
cloned_flyr[0]
cloned_flyr[0].url
'https://pythonapi.playground.esri.com/server/rest/services/Hosted/Arkansas_Hospitals/FeatureServer'
We can see that the clone_items()
function returns a list containing the cloned items. Indexing the list we can see the operation created a new hosted feature layer in the target organization owned by the api_data_owner
user we entered as the owner
argument.
Cloning multiple items simultaneously
We demonstrated above passing one item in the items
parameter list. Now let's see how clone_items()
will clone every item in the list. First let's search()
the source function for items owned by a particular user and clone the list items into our target all at the same time.
tester_content = source.content.search(f"tags:disaster_testing AND owner:api_data_owner")
tester_content
[<Item title:"Earthquake Damage Map" type:Web Map owner:api_data_owner>, <Item title:"earthquakes_2" type:CSV owner:api_data_owner>, <Item title:"earthquakes_sample_data" type:CSV owner:api_data_owner>, <Item title:"earthquakes_2" type:Feature Layer Collection owner:api_data_owner>]
cloned_items = target.content.clone_items(items=tester_content,
folder=output_folder.name)
cloned_items
[<Item title:"earthquakes_2" type:Feature Layer Collection owner:api_data_owner>, <Item title:"earthquakes_2" type:CSV owner:api_data_owner>, <Item title:"earthquakes_sample_data" type:CSV owner:api_data_owner>, <Item title:"Earthquake Damage Map" type:Web Map owner:api_data_owner>]
We can see that the clone_items()
function cloned all the items in the list we passed to the items
argument. We can also see that by not passing in an owner
argument, the items are owned by the logged in user that ran clone_items()
.
target.users.me.username
'api_data_owner'
There's a quick demonstration of how the clone_items()
function replicates items from a source to a target. Now let's move onto iterating through a list of hosted feature layers and working with some additonal parameters to clone information products that consume them.
The cloning process
A very important parameter in the clone_items()
function impacts its output:
search_existing_items
- The possible values areTrue
orFalse
.
The default value is True
. Let's describe what happens when the value is set to True
. When any source item
is cloned into the target, the clone_items()
function assigns the cloned item
all the essential item type typeKeywords
(See hyperlink for typeKeywords
automatically assigned to items) plus an additional typeKeyword of source-<source_item_id_value>
. For example, if clone_items()
clones a feature layer item from source item item id
d879c7d972b1d989b97d037c7a7737d6
, the resulting feature layer item in the target will have a typeKeyword
of source-d879c7d972b1d989b97d037c7a7737d6
in addition to all essential typeKeywords
. Before any actual clone, clone_items()
searches the target for any item with a typeKeyword
matching that pattern and if it finds one uses the existing target item rather than cloning the item again.
If the argument is set to False
, the specified item(s) and its dependent items will be cloned into the target no matter whether they already exist.
Web Maps
We can create a list of the web map items using the advanced_search() method. First we'll get a list of all items, and then convert it to a Pandas DataFrame to filter out for our web maps:
# Get all items owned by the user
owner_items = source.content.advanced_search(query=f"owner:{source.users.me.username}",
max_items=-1)["results"]
# Convert list to a Pandas DataFrame
owner_items_df = pd.DataFrame(owner_items)
Use Pandas groupby()
to create individual group objects of each item type owned by the user. Then use the get_group()
method to return all the Web Maps.
wm_item_df = owner_items_df.groupby("type").get_group("Web Map")
wm_item_df
id | owner | created | isOrgItem | modified | guid | name | title | type | typeKeywords | ... | tokenExpirationDate | token2ExpirationDate | contentOrigin | lastViewed | size | commentsEnabled | itemControl | layers | tables | contentStatus | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
590 | 935b9032e6c8492ebcd5ff3d9c8131e9 | ArcGISPyAPIBot | 1685983591000 | True | 1685983595000 | None | None | Case 02773785 220 Mbs | Web Map | [ArcGIS Online, Collector, Data Editing, Explo... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
701 | 549169f28ce9424d94f7c5f237d33840 | ArcGISPyAPIBot | 1607547954000 | True | 1612904683000 | None | None | Chicago_Libraries_High_Checkout | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1305 | e4ba5816764748f1b36ead1a1327c37f | ArcGISPyAPIBot | 1692895691000 | True | 1693252353000 | None | None | set1_cities_webmap | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1709020800000 | NaN | NaN | NaN | NaN | NaN | NaN |
1359 | 061b561058484cbda73eea08dffe43bf | ArcGISPyAPIBot | 1698259114000 | True | 1698259116000 | None | None | UC Location | Web Map | [ArcGIS API for JavaScript, ArcGIS Online, Exp... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1360 | 6b1f7ea22e2543aa9849e34097d05570 | ArcGISPyAPIBot | 1708463115000 | True | 1708463115000 | None | None | US Power Plant Data | Web Map | [ArcGIS API for JavaScript, ArcGIS Online, Exp... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1364 | 437df2865e0f4147853a98d883d93c13 | ArcGISPyAPIBot | 1582579501000 | True | 1582590674000 | None | None | Vietnam for Testing | Web Map | [ArcGIS Online, Explorer Web Map, Map, Offline... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1382 | 8596074ae0af45039ec50e86c83b17e0 | ArcGISPyAPIBot | 1656553353000 | True | 1656553353000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1383 | 0c787e9cba574a5eb773c9846d90a8c6 | ArcGISPyAPIBot | 1654023844000 | True | 1654023844000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1384 | 3f7a064e12254f6b80a3f9179b37b8b7 | ArcGISPyAPIBot | 1656553355000 | True | 1656553355000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1385 | 55f662afe71145fcbf107e783710c0bf | ArcGISPyAPIBot | 1656553356000 | True | 1656553356000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1386 | dde150ba99454b00895225d06bdc7473 | ArcGISPyAPIBot | 1654023846000 | True | 1654023846000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1387 | 21f421cb5817471ab65185a53febc8ab | ArcGISPyAPIBot | 1654023843000 | True | 1654023843000 | None | None | erase_me_map | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1420 | 62f50f03e1c846d9a547c0480dc768e0 | ArcGISPyAPIBot | 1684115932000 | True | 1684115935000 | None | None | Giraffes | Web Map | [ArcGIS Online, Explorer Web Map, FieldMapsDis... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1421 | 6d8e89f6fb0a4c8a8bb5e059846425d9 | ArcGISPyAPIBot | 1601408274000 | True | 1684115692000 | None | None | Giraffes Original | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1433 | bc2e103d1c81422bb117a2a6f2491db3 | ArcGISPyAPIBot | 1573859849000 | True | 1573859860000 | None | None | mult_lyr_webmap | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
1446 | 84e125b2bb9a413aa250c8bd1a27d4a7 | ArcGISPyAPIBot | 1694650220000 | True | 1694650235000 | None | None | pansharpened_landsat_times_3 | Web Map | [ArcGIS Online, Explorer Web Map, Map, Online ... | ... | -1 | -1 | self | 1708671600000 | NaN | NaN | NaN | NaN | NaN | NaN |
16 rows × 53 columns
The resulting DataFrame's index uses the position value of each web map in the original dataframe for the row index, so we'll reset it to a zero-based index:
wm_item_df.index
Index([1081, 1092, 1250, 1251, 1255, 1363, 1364, 1365, 1366, 1367, 1368, 1401, 1402, 1414, 1428, 1454], dtype='int64')
web_maps = wm_item_df.reset_index(drop=True)
web_maps.index
RangeIndex(start=0, stop=16, step=1)
Let's retrieve a web map using a tag search so we can clone it into our target:
def check_wm(tag_list):
return "power_plants" in tag_list
power_plant_df = web_maps[web_maps.tags.apply(check_wm)]
power_plant_df.index
Index([3], dtype='int64')
power_plant_df.loc[3].id
'6b1f7ea22e2543aa9849e34097d05570'
power_plant_wm_item = source.content.get(power_plant_df.loc[3].id)
power_plant_wm_item
Let's use the WebMap object to print information about each of the layers in the Web Map. We'll look at the layer name and the url, specifically paying attention to the beginning of the path componenet of the url, which contains the organization id of the host of the layer.
wm_obj = WebMap(power_plant_wm_item)
for wm_lyr in wm_obj.definition["operationalLayers"]:
print(f"{wm_lyr['title']}\n{' '*2}{wm_lyr['url']}")
print(f"{' ' * 2}Host Organization id: {wm_lyr['url'].split('/')[3]}")
print("\n")
US Power Plants https://services7.arcgis.com/JEwYeAy2cc8qOe3o/arcgis/rest/services/US_Power_Plants/FeatureServer/0 Host Organization id: JEwYeAy2cc8qOe3o
Printing out our source organization id, we can see that the layer in the web map is hosted in the source.
print(f"{source.properties.id}")
JEwYeAy2cc8qOe3o
We can use the search_existing_items
parameter to handle how we want the items consumed in our web map
to behave upon cloning. Given what we've learned about this parameter, we can rely on it to detect whether any of the item ids
consumed in our web maps match any of the typeKeywords
already existing in our target. If the function finds an existing item in the target, it will swizzle the appropriate values in the new web map
definition. If the function does not find an existing item based on the typeKeyword
, it will clone the item
. For non-hosted items, it will recreate those items in the resulting web map
.
In this case, we'll set it to False
since we know we've not cloned this web map before. We want any hosted layers to also clone in our target.
cloned_wm = target.content.clone_items(items=[power_plant_wm_item],
folder=output_folder.name,
search_existing_items=False)
Let's examine the resulting items in the target:
cloned_wm
[<Item title:"US Power Plants" type:Feature Layer Collection owner:api_data_owner>, <Item title:"US Power Plant Data" type:Web Map owner:api_data_owner>]
cloned_wm_obj = WebMap(cloned_wm[1])
for wm_lyr in cloned_wm_obj.definition["operationalLayers"]:
print(f"{wm_lyr['title']}\n{' '*2}{wm_lyr['url']}")
print(f"{' ' * 2}Host Organization id: {wm_lyr['url'].split('/')[3]}")
print("\n")
US Power Plants https://pythonapi.playground.esri.com/server/rest/services/Hosted/US_Power_Plants/FeatureServer/0 Host Organization id: server
We can see that clone_items()
successfully recreated the web map and cloned the operational layer, and configured the resulting web map to use the cloned feature layer. We can see from the netscheme and path components of the layer URL that the resulting layer is using an ArcGIS Enterprise rather than an ArcGIS Online organization.
Let's visually compare our webmaps:
source
wm_obj
target
cloned_wm_obj
We can see visually that the maps appear to contain the same layers. You can repeat this process for any other web map
source/target pair from the wm_map
dictionary to make a visual comparison.
ArcGIS Dashboards
See the Cloning and Troubleshooting Complex Items guide for details on cloning ArcGIS Dashboards.
Story Maps
See the Cloning and Troubleshooting Complex Items guide for details on cloning ArcGIS StoryMaps.
Conclusion
This guide demonstrated one workflow for cloning items between a source Web GIS and a target Web GIS. We connected to each GIS as an administrator and cloned an individual item, and then a set of items. We then cloned a Web Map item, demonstrating how a hosted feature layer within the web map is also cloned and the cloned web map is configured to read the cloned feature layer which is hosted in the target organization.