Cloning and Troubleshooting Complex Items

The ArcGIS Platform provides you an almost endless variety of items and applications to serve the GIS community. Given the diversity of individual items combined with the complexity created by items containing other items, cloning between organizational deployments puts up a unique set of hurdles that cannot always be scaled in a straightfoward manner. We continually work to provide the cleanest method possible for cloning, but when the clone_items() method does not accomodate specific items, there are some creative ways we can address these situations.

We're going to look at a few things in this guide:

How to clone different complex applications
How to dive into an item's structure
Strategies for remapping data, including the new remap_data() function added in version 2.4.0

Updated Cloning Types

First and foremost, we thought that we'd mention a few item types that have become cloneable or had past obstructions to cloning cleared. Some items may have cloned partially or been recreated in some capacity that may have been incomplete, but now these should clone without a hitch through a call to clone_items(). The most notable ones fixed in the last few recent versions:

Experience Builder Experiences
StoryMaps
Web Scenes
ArcGIS Dashboards

'Complex Items' and `search_existing_items`

What do we mean when we say "complex items?" In this context, we're referring to items that may contain other items within them. One of the most important facets of cloning these complex items is making sure that all of their internal items are properly accounted for in the target GIS organization. That can mean making sure those items are also cloned, replaced with other items that already exist in the new GIS, or come from sources accessible by any GIS (such as Living Atlas items). Regardless of the method, the underlying structure of the newly cloned item must refer to items that can be used in the new GIS. This guide will cover a handful of strategies for cloning item dependencies and confirming they're being incorporated properly.

One of the most critical tools when cloning these items is the clone_items() function's search_existing_items argument. When this is set to True (which it is by default), the function searches to see if all of the items set to be cloned (this includes all items in the items argument and their internal data sources) already exist in the target GIS organization. When an item is cloned, it gets given a specific typeKeyword indicating its source item, so the function searches for items with those matching keywords. This means that items with matching names and data types will not get used if they aren't a clone of the item in question. When search_existing_items is set to False, all items will get cloned (even if clones of them already exist in the target), unless they're not cloneable or covered in a dictionary passed in as the item_mapping argument (more on this later).

Web Experiences

For those of you that work a lot with the Experience Builder, 2.2.0 is the update for you. This release includes a number of highly requested additions for cloning and managing experiences, which we'll briefly cover here.

First and foremost, Experience Builder items already existing in a portal will now be cloned properly when passed into clone_items(). Additionally we've added an entire new class for working with Web Experiences in general- we'll have online resources exhibiting the full functionality of the class, but we'll touch upon a couple useful cloning-related bits here.

First, we'll connect to our Web GIS organizations. For this example, we'll connect to an ArcGIS Enterprise deployment, and two ArcGIS Online organizations. Works with any deployment type!

from arcgis.gis import GIS
gis1 = GIS(profile="your_enterprise_profile")
gis2 = GIS(profile="your_online_profile")
gis3 = GIS(profile="another_online_profile")

As a quick refresher, here's how simple it is to clone a Web Map. You'll see throughout this guide that cloning complex items is ultimately still just the same calls as you'll see here.

test = gis2.content.get("7b066a3cccaf4c929808a6e384319ba5")
test

UC Location x3333

Web Map by online_user
Last Modified: July 20, 2023
0 comments, 5 views

gis3.content.clone_items(items = [test])

[<Item title:"UC Location x3333" type:Web Map owner:another_online_user>]

Now, let's do it with an Experience, and one that has a Web Map in it to boot.

exp_item = gis1.content.get("66b1f05151a8442999e67a8d6a1938ed")
exp_item

Fancy, Sophisticated Experience

Web Experience by ent_user
Last Modified: July 12, 2023
0 comments, 17 views

cloned_list = gis2.content.clone_items(items = [exp_item], search_existing_items = False, folder = "test_exp")
cloned_list

[<Item title:"Fancy, Sophisticated Experience" type:Web Experience owner:online_user>,
 <Item title:"UC Location" type:Web Map owner:online_user>]

cloned_list[0]

Fancy, Sophisticated Experience

Web Experience by online_user
Last Modified: July 13, 2023
0 comments, 0 views

Using the new WebExperience class, we can verify directly inline that our cloned experience looks good, without even having to open up our portal.

from arcgis.apps.expbuilder import WebExperience
exp = WebExperience(cloned_list[0])
exp.preview()

Local/Developer Edition

Now, we know that some of you might work with the Developer Edition of Experience Builder. For those unfamiliar, those experiences are created and stored locally, and don't exist in a portal (even though their data is tied to a particular portal). This might make it pretty tricky if you want to bring your experiences to a different established portal, especially if the data sources might be different there (think changing from development -> staging -> production type scenarios). The new WebExperience class lets you work with local experiences, and has a function called upload that allows you to easily add them to different portals and remap their data. A more detailed explanation of this workflow can be found in the Experience Builder guide.

First, download your local experience, then get your file path.

Note: For this unfamiliar with this workflow, this download option lets you get a zip file containing all of the data of your local (not portal) Experience. It is only available when working with a locally deployed instance of the Developer Edition Experience Builder.

Once you've downloaded your Experience, unzip the zip file, and open the newly created folder. Under "cdn/1" (or a different number if you've downloaded it multiple times), there should be a "config.json" file. This file contains the structure of your Experience, which is what we need to add it to our Web GIS.

Once you have your path to this file, instantiate a WebExperience object using it, and specify a GIS.

local_path = '/computer/path/to/config.json'
local_exp = WebExperience(path = local_path, gis = gis2)

As shown below, the WebExperience module lets you easily view your item's datasources through a class property.

local_exp.datasources

{'dataSource_1': {'id': 'dataSource_1',
  'type': 'WEB_MAP',
  'sourceLabel': 'UC Location',
  'itemId': '29218750833a43a5844d6686c9f428c5',
  'portalUrl': 'https://geosaurus.maps.arcgis.com'},
 'dataSource_4': {'id': 'dataSource_4',
  'type': 'WEB_MAP',
  'sourceLabel': 'Ugly Map',
  'itemId': '6fdd91f8eb754d01bdbaf4b8b4419719',
  'portalUrl': 'https://geosaurus.maps.arcgis.com'}}

The upload() function allows you to transfer your local experience to a portal (and remap data in a couple different ways if need be).

local_clone = local_exp.upload(
    gis = gis3, 
    title = "High Tech Clone", 
    publish = True, 
    auto_remap = True
)

local_clone

High Tech Clone

Web Experience by another_online_user
Last Modified: July 13, 2023
0 comments, 0 views

local_clone.get_data()["dataSources"]

{'dataSource_1': {'id': 'dataSource_1',
  'type': 'WEB_MAP',
  'sourceLabel': 'UC Location',
  'itemId': 'a39a060de7724041a3fb7b529456b075',
  'portalUrl': 'https://pythonapi.maps.arcgis.com'},
 'dataSource_4': {'id': 'dataSource_4',
  'type': 'WEB_MAP',
  'sourceLabel': 'Ugly Map',
  'itemId': 'f9827ea6779440d196806b84017f0c33',
  'portalUrl': 'https://pythonapi.maps.arcgis.com'}}

There we have it- uploaded into a new portal with remapped data sources.

StoryMaps, Web Scenes, and Item Remapping

Recent releases have also smoothed over cloning for another popular item type- ArcGIS StoryMaps. StoryMaps can contain numerous media types and different types of ArcGIS items, which clone_items() should cover- we're going to show how users can examine the items in their StoryMap and confirm that both the StoryMap and all of the internal items were cloned properly.

We'll start by grabbing a StoryMap using the portal item ID, and then use a couple methods to examine the content within the item: using the ArcGIS API for Python's StoryMap class, and accessing the item's internal JSON structure.

storymap = gis2.content.get("ebd88b8426db4384bb3d328ebcbc93f5")
storymap

How Tall is Mt. Everest Really?
We've all been wondering.

StoryMap by online_user
Last Modified: June 23, 2023
0 comments, 59 views

Using the Python API's StoryMap module, we can get a general overview of what's in the StoryMap with the content list property.

from arcgis.apps.storymap import StoryMap
sm_obj = StoryMap(storymap)
sm_obj.content_list

['Storycover', 'Navigation', Web Scene, 'Credits']

As you can see, we now have a general idea of what's in the StoryMap. We see that we have a Web Scene in there, which will also need to be cloned over. We can get even more information on the contained items and media of a StoryMap by examining the JSON structure of the application. There are two ways to do this: accessing the item's resources (contains the draft), and the get_data() function (contains the published StoryMap JSON data, if applicable). We'll take a look at the draft in the resources.

storymap.resources.list()

[{'resource': 'draft_1687545240201.json',
  'created': 1687545240000,
  'size': 3426,
  'properties': '{"editInfo":{"editor":"online_user","modified":1687545240420,"id":"WXJiWd8s3GH_cECUc2JVz"}}',
  'access': 'private'},
 {'resource': 'published_data.json',
  'created': 1687545244000,
  'size': 3426,
  'properties': '{"editInfo":{"editor":"online_user","modified":1687545244158,"id":"sGS7aK_uxhWaQW0rScbdi"}}',
  'access': 'inherit'}]

The draft is the one with "draft" in the name. We'll access that one by passing its name into resources.get(), and then take a look at all of the items in the StoryMap by checking out the resources section of the resulting dictionary.

sm_draft = storymap.resources.get("draft_1687545240201.json")
sm_draft["resources"]

{'r-994302e6d3034727adcdc8d26bc0af55': {'type': 'webmap',
  'data': {'extent': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'xmin': 9550102.425298018,
    'ymin': 3424521.4326146077,
    'xmax': 9806207.373133773,
    'ymax': 3568554.30289715},
   'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'x': 9678154.89921606,
    'y': 3496537.867755944,
    'z': 0},
   'zoom': 9.59591002782806,
   'viewpoint': {'rotation': 359.50339529759475,
    'scale': 764563.6394107993,
    'targetGeometry': {'spatialReference': {'latestWkid': 3857,
      'wkid': 102100},
     'x': 9678154.89921606,
     'y': 3496537.867755944,
     'z': 0},
    'camera': {'position': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9675713.003402857,
      'y': 3217737.9754389315,
      'z': 9553.995877388865},
     'heading': 0.4966047024052563,
     'tilt': 86.6645594945896}},
   'lightingDate': 1678859100000,
   'directShadows': True,
   'weather': {'type': 'snowy',
    'cloudCover': 0.48,
    'precipitation': 0.36,
    'snowCover': 'disabled'},
   'ground': {'opacity': 1},
   'visibleLayers': [{'id': 'globalElevation'}],
   'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
      'opacity': 1,
      'title': 'World Imagery',
      'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
      'visibility': True,
      'layerType': 'ArcGISTiledMapServiceLayer'}]},
   'mapLayers': [],
   'itemId': '994302e6d3034727adcdc8d26bc0af55',
   'itemType': 'Web Scene',
   'type': 'default'}},
 'r-4uVRzh': {'type': 'story-theme',
  'data': {'themeId': 'summit', 'themeBaseVariableOverrides': {}}}}

Above you'll see all of the info about what's contained within the StoryMap. All that's in there currently is a Web Scene, which simplifies things; we just know that Web Scene needs to be cloned across. The draft JSON of the cloned StoryMap should match this, except with a select few things adjusted to the new organization, such as itemId.

We're going to assign the output of the clone_items() call to a variable called clone_list. This will be a list of the ArcGIS items that got cloned over.

clone_list = gis3.content.clone_items(items = [storymap], search_existing_items=False, folder = "test_story")

clone_list

[<Item title:"Big Mountain" type:Web Scene owner:another_online_user>,
 <Item title:"How Tall is Mt. Everest Really?" type:StoryMap owner:another_online_user>]

cloned_story = clone_list[1]
cloned_story

How Tall is Mt. Everest Really?
We've all been wondering.

StoryMap by another_online_user
Last Modified: October 27, 2023
0 comments, 0 views

As you can see, both items cloned and the StoryMap is viewable in the new organization. Let's take a look at the structure in the same way we did with the original StoryMap and confirm that the right components are there.

StoryMap(cloned_story).content_list

['Storycover', 'Navigation', Web Scene, 'Credits']

cloned_story.resources.get("draft_1687545240201.json")['resources']

{'r-54d6bced2e7242f6af3a219f69d8a5bd': {'type': 'webmap',
  'data': {'extent': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'xmin': 9550102.425298018,
    'ymin': 3424521.4326146077,
    'xmax': 9806207.373133773,
    'ymax': 3568554.30289715},
   'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'x': 9678154.89921606,
    'y': 3496537.867755944,
    'z': 0},
   'zoom': 9.59591002782806,
   'viewpoint': {'rotation': 359.50339529759475,
    'scale': 764563.6394107993,
    'targetGeometry': {'spatialReference': {'latestWkid': 3857,
      'wkid': 102100},
     'x': 9678154.89921606,
     'y': 3496537.867755944,
     'z': 0},
    'camera': {'position': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9675713.003402857,
      'y': 3217737.9754389315,
      'z': 9553.995877388865},
     'heading': 0.4966047024052563,
     'tilt': 86.6645594945896}},
   'lightingDate': 1678859100000,
   'directShadows': True,
   'weather': {'type': 'snowy',
    'cloudCover': 0.48,
    'precipitation': 0.36,
    'snowCover': 'disabled'},
   'ground': {'opacity': 1},
   'visibleLayers': [{'id': 'globalElevation'}],
   'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
      'opacity': 1,
      'title': 'World Imagery',
      'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
      'visibility': True,
      'layerType': 'ArcGISTiledMapServiceLayer'}]},
   'mapLayers': [],
   'itemId': '54d6bced2e7242f6af3a219f69d8a5bd',
   'itemType': 'Web Scene',
   'type': 'default'}},
 'r-4uVRzh': {'type': 'story-theme',
  'data': {'themeId': 'summit', 'themeBaseVariableOverrides': {}}}}

As we can see, the results are essentially the same, minus one difference: the key of the resource and the itemId are different. The Web Scene got cloned over to the new organization, and our cloned StoryMap properly points to the new one.

Item Remapping

Now, remember the bit about search_existing_items earlier? Well there's something we should think about. What if there are items/data in our target organization that we want to use, but exist under a different name? Well, that's where item_mapping comes in. item_mapping is a dictionary you can pass to explicitly remap your data sources to particular items.

Say, for example, that you have the same Web Map in your staging and production environments- however, in your staging environment it's called "wm_staging", and in your production environment it's called "wm_production". Because of the disparity in naming and typeKeywords, search_existing_items won't see that they're the same; if you clone an item that contains the webmap, clone_items() will still want to clone the webmap. Instead, we can just add an entry into the item_mapping dictionary: the item ID of "wm_staging" is the key, and the item ID of "wm_production" is the value. This will go into the item's structure and replace the "wm_staging" info with the "wm_production" info, making it so the new item that contains the webmap will point to and render the production map.

We'll take a look at the before and after effects of remapping with this way. Let's go back to our StoryMap- say we want to remap that Web Scene to a different one representing Mt. Everest. We'll show how easy that is!

As we mentioned earlier, one handy way to get an insight into the structure of an item is by using the get_data() function. For most item types, it will show you the underlying JSON constructing the item. In the context of StoryMaps, get_data() will show you the structure of the published StoryMap.

storymap.get_data()

{'root': 'n-soHyee',
 'nodes': {'n-iUbraW': {'type': 'storycover',
   'data': {'type': 'minimal',
    'title': 'How Tall is Mt. Everest Really?',
    'summary': "We've all been wondering.",
    'byline': 'Noah P',
    'titlePanelPosition': 'start'}},
  'n-iYFoSo': {'type': 'navigation',
   'data': {'links': []},
   'config': {'isHidden': True}},
  'n-wi5FDm': {'type': 'webmap',
   'data': {'caption': 'About yay tall',
    'map': 'r-994302e6d3034727adcdc8d26bc0af55',
    'timeSlider': False,
    'extent': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
     'xmin': 9662516.888750484,
     'ymin': 3239699.759121931,
     'xmax': 9689421.53504091,
     'ymax': 3254830.96571359},
    'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
     'x': 9675969.21189586,
     'y': 3247265.3624178204,
     'z': 8084.227762622759},
    'zoom': 12.84669399057029,
    'viewpoint': {'rotation': 359.503395297594,
     'scale': 303047.43474156555,
     'targetGeometry': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9676680.192470033,
      'y': 3328868.1625311403,
      'z': 4594.240628777072},
     'camera': {'position': {'spatialReference': {'latestWkid': 3857,
        'wkid': 102100},
       'x': 9675713.003402857,
       'y': 3217737.9754389315,
       'z': 9553.995877388865},
      'heading': 0.4966047024059902,
      'tilt': 86.66455949458957}},
    'lightingDate': 1678859100000,
    'directShadows': True,
    'weather': {'type': 'snowy',
     'cloudCover': 0.48,
     'precipitation': 0.36,
     'snowCover': 'disabled'},
    'ground': {'opacity': 1},
    'visibleLayers': [{'id': 'globalElevation'}],
    'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
       'opacity': 1,
       'title': 'World Imagery',
       'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
       'visibility': True,
       'layerType': 'ArcGISTiledMapServiceLayer'}]}}},
  'n-ybu8E6': {'type': 'text', 'data': {'text': '', 'type': 'h4'}},
  'n-ldJhKZ': {'type': 'text', 'data': {'text': '', 'type': 'paragraph'}},
  'n-jipfrd': {'type': 'attribution',
   'data': {'content': '', 'attribution': ''}},
  'n-4tdmtS': {'type': 'credits',
   'children': ['n-ybu8E6', 'n-ldJhKZ', 'n-jipfrd']},
  'n-soHyee': {'type': 'story',
   'data': {'storyTheme': 'r-4uVRzh'},
   'config': {'coverDate': 'first-published'},
   'children': ['n-iUbraW', 'n-iYFoSo', 'n-wi5FDm', 'n-4tdmtS']}},
 'resources': {'r-994302e6d3034727adcdc8d26bc0af55': {'type': 'webmap',
   'data': {'extent': {'spatialReference': {'latestWkid': 3857,
      'wkid': 102100},
     'xmin': 9550102.425298018,
     'ymin': 3424521.4326146077,
     'xmax': 9806207.373133773,
     'ymax': 3568554.30289715},
    'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
     'x': 9678154.89921606,
     'y': 3496537.867755944,
     'z': 0},
    'zoom': 9.59591002782806,
    'viewpoint': {'rotation': 359.50339529759475,
     'scale': 764563.6394107993,
     'targetGeometry': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9678154.89921606,
      'y': 3496537.867755944,
      'z': 0},
     'camera': {'position': {'spatialReference': {'latestWkid': 3857,
        'wkid': 102100},
       'x': 9675713.003402857,
       'y': 3217737.9754389315,
       'z': 9553.995877388865},
      'heading': 0.4966047024052563,
      'tilt': 86.6645594945896}},
    'lightingDate': 1678859100000,
    'directShadows': True,
    'weather': {'type': 'snowy',
     'cloudCover': 0.48,
     'precipitation': 0.36,
     'snowCover': 'disabled'},
    'ground': {'opacity': 1},
    'visibleLayers': [{'id': 'globalElevation'}],
    'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
       'opacity': 1,
       'title': 'World Imagery',
       'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
       'visibility': True,
       'layerType': 'ArcGISTiledMapServiceLayer'}]},
    'mapLayers': [],
    'itemId': '994302e6d3034727adcdc8d26bc0af55',
    'itemType': 'Web Scene',
    'type': 'default'}},
  'r-4uVRzh': {'type': 'story-theme',
   'data': {'themeId': 'summit', 'themeBaseVariableOverrides': {}}}}}

Like we did earlier, we're going to examine the data sources contained in the item. For all items with an underlying JSON structure, get_data() returns a dictionary with accessible key/value pairs.

storymap.get_data()['resources']

{'r-994302e6d3034727adcdc8d26bc0af55': {'type': 'webmap',
  'data': {'extent': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'xmin': 9550102.425298018,
    'ymin': 3424521.4326146077,
    'xmax': 9806207.373133773,
    'ymax': 3568554.30289715},
   'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'x': 9678154.89921606,
    'y': 3496537.867755944,
    'z': 0},
   'zoom': 9.59591002782806,
   'viewpoint': {'rotation': 359.50339529759475,
    'scale': 764563.6394107993,
    'targetGeometry': {'spatialReference': {'latestWkid': 3857,
      'wkid': 102100},
     'x': 9678154.89921606,
     'y': 3496537.867755944,
     'z': 0},
    'camera': {'position': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9675713.003402857,
      'y': 3217737.9754389315,
      'z': 9553.995877388865},
     'heading': 0.4966047024052563,
     'tilt': 86.6645594945896}},
   'lightingDate': 1678859100000,
   'directShadows': True,
   'weather': {'type': 'snowy',
    'cloudCover': 0.48,
    'precipitation': 0.36,
    'snowCover': 'disabled'},
   'ground': {'opacity': 1},
   'visibleLayers': [{'id': 'globalElevation'}],
   'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
      'opacity': 1,
      'title': 'World Imagery',
      'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
      'visibility': True,
      'layerType': 'ArcGISTiledMapServiceLayer'}]},
   'mapLayers': [],
   'itemId': '994302e6d3034727adcdc8d26bc0af55',
   'itemType': 'Web Scene',
   'type': 'default'}},
 'r-4uVRzh': {'type': 'story-theme',
  'data': {'themeId': 'summit', 'themeBaseVariableOverrides': {}}}}

Using the itemId key in there, we can see exactly which item is being used in there. Say we already have an item in the target organization we want to replace that particular item with; we would construct an item_mapping dictionary with the original item's ID as the key and the target item's ID as the value. This dictionary can be contructed with as many appropriate item ID's as necessary.

Let's say we already have another Everest Web Scene in our target organization that we want to use instead; we would just have to find the item ID of that item, which is '502c4f1454d34766807e1268c09acf34'. If we we were to clone this StoryMap again, we could make our cloned StoryMap use this target item. We just pass in an item_mapping dictionary, and set the item ID of the source item we want to replace as the key, and set the item ID of the desired item in the target GIS as the value.

item_mapping = {"994302e6d3034727adcdc8d26bc0af55" : "502c4f1454d34766807e1268c09acf34"}
clone_list = gis3.content.clone_items(
    items = [storymap], 
    search_existing_items = False, 
    item_mapping = item_mapping,
    folder = "test_story"
)
clone_list

[<Item title:"How Tall is Mt. Everest Really?" type:StoryMap owner:another_online_user>]

We see that only the StoryMap got cloned, since the Web Scene within it didn't need to get cloned again.

clone_list[0]

How Tall is Mt. Everest Really?
We've all been wondering.

StoryMap by another_online_user
Last Modified: July 13, 2023
0 comments, 0 views

Now, if we look at the cloned item's resources, we'll see in the first line that the node got properly renamed using our target item ID, and that the value for "itemId" is also that target item ID.

clone_list[0].get_data()['resources']

{'r-502c4f1454d34766807e1268c09acf34': {'type': 'webmap',
  'data': {'extent': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'xmin': 9550102.425298018,
    'ymin': 3424521.4326146077,
    'xmax': 9806207.373133773,
    'ymax': 3568554.30289715},
   'center': {'spatialReference': {'latestWkid': 3857, 'wkid': 102100},
    'x': 9678154.89921606,
    'y': 3496537.867755944,
    'z': 0},
   'zoom': 9.59591002782806,
   'viewpoint': {'rotation': 359.50339529759475,
    'scale': 764563.6394107993,
    'targetGeometry': {'spatialReference': {'latestWkid': 3857,
      'wkid': 102100},
     'x': 9678154.89921606,
     'y': 3496537.867755944,
     'z': 0},
    'camera': {'position': {'spatialReference': {'latestWkid': 3857,
       'wkid': 102100},
      'x': 9675713.003402857,
      'y': 3217737.9754389315,
      'z': 9553.995877388865},
     'heading': 0.4966047024052563,
     'tilt': 86.6645594945896}},
   'lightingDate': 1678859100000,
   'directShadows': True,
   'weather': {'type': 'snowy',
    'cloudCover': 0.48,
    'precipitation': 0.36,
    'snowCover': 'disabled'},
   'ground': {'opacity': 1},
   'visibleLayers': [{'id': 'globalElevation'}],
   'baseMap': {'baseMapLayers': [{'id': 'World_Imagery_2017',
      'opacity': 1,
      'title': 'World Imagery',
      'url': 'https://services.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer',
      'visibility': True,
      'layerType': 'ArcGISTiledMapServiceLayer'}]},
   'mapLayers': [],
   'itemId': '502c4f1454d34766807e1268c09acf34',
   'itemType': 'Web Scene',
   'type': 'default'}},
 'r-4uVRzh': {'type': 'story-theme',
  'data': {'themeId': 'summit', 'themeBaseVariableOverrides': {}}}}

Dashboards, Editing Item Data, and `remap_data()`

As you've just seen, accessing an item's data gives you great insight into what's going on. Having direct access to the structure of the item's data also gives you a great tool to troubleshoot when something doesn't quite go right with your cloning (or if there's just any problem with an item in your organization).

However, what if the item's structure is particularly complicated, and you don't want to search through all of it? One big user request in recent times has been the ability to easily replace all of the broken data sources or components of an application without having to do it by hand or recreate the app. In version 2.4.0, we added a function that does just that: remap_data(), a method of the Item class.

ArcGIS Dashboards aren't actively supported with their own module in the Python API, but they're still cloneable. There are a couple caveats, such as item_mapping not being usable, but using the tricks we've covered in this guide, users can still move them across organizations and remap their data. This makes them the perfect example of how to troubleshoot something by hand using methods found in the gis.item module.

We'll start by grabbing a dashboard and cloning it, before looking at how to rework the data within.

dashboard = gis2.content.get("9172001dfdcf4e77a5731cb99b3a1c31")
dashboard

Really Exciting Dashboard

Dashboard by online_user
Last Modified: June 26, 2023
0 comments, 197 views

For reference, he's a picture of our source Dashboard. Exciting, isn't it?

dash_list = gis3.content.clone_items(items = [dashboard], search_existing_items = False)

dash_list

[<Item title:"clone_test_violet" type:Web Map owner:another_online_user>,
 <Item title:"CA/OR Airports" type:Feature Layer Collection owner:another_online_user>,
 <Item title:"Really Exciting Dashboard" type:Dashboard owner:another_online_user>]

So, there we have it, the Dashboard cloned. But what if we didn't want the violet map in our new GIS's Dashboard? What if this GIS is completely green themed, and we wanted a green map? Since we can't use item_mapping in this case, we'll look at other options. This is where remap_data() comes in.

`remap_data()`

As we mentioned earlier, remap_data() is a new function in the Item class added in version 2.4.0. This method was added in to address how users might try to fix broken items, or automate mass replacement of an item in other applications. One great example would be if you cloned a Dashboard, but wanted to remap the internal data items to new data items already existent in the target org. Another great example would be if you had a feature layer being used in a bunch of different applications, but the underlying service broke or became obsolete, and you want to automate replacing it in all of the apps that are using it. Basically, any scenario where you want to easily replace an item within another one, this method will be your best friend. See the docstring for it below.

Let's take a look at how we can use it, and what we're changing. We'll start by examining our cloned Dashboard. Let's call get_data() on it- this will show us the internal structure of our Dashboard.

cloned_dash = dash_list[2]
dash_structure = cloned_dash.get_data()
dash_structure

{'version': 55,
 'authoringApp': 'ArcGIS Dashboards',
 'authoringAppVersion': '4.27.0+629a18f319',
 'maxPaginationRecords': 50000,
 'mapOverrides': {'highlightColor': '#ff00ff',
  'trackedFeatureColor': '#0000ff',
  'trackedFeatureRadius': 100},
 'theme': 'light',
 'themeOverrides': {},
 'numberPrefixOverrides': [{'key': 'yotta', 'symbol': 'Y', 'enabled': True},
  {'key': 'zeta', 'symbol': 'Z', 'enabled': True},
  {'key': 'exa', 'symbol': 'E', 'enabled': True},
  {'key': 'peta', 'symbol': 'P', 'enabled': True},
  {'key': 'tera', 'symbol': 'T', 'enabled': True},
  {'key': 'giga', 'symbol': 'G', 'enabled': True},
  {'key': 'mega', 'symbol': 'M', 'enabled': True},
  {'key': 'kilo', 'symbol': 'k', 'enabled': True},
  {'key': 'base', 'symbol': '', 'enabled': True},
  {'key': 'deci', 'symbol': 'd', 'enabled': False},
  {'key': 'centi', 'symbol': 'c', 'enabled': False},
  {'key': 'milli', 'symbol': 'm', 'enabled': False},
  {'key': 'micro', 'symbol': 'µ', 'enabled': False},
  {'key': 'nano', 'symbol': 'n', 'enabled': False}],
 'desktopView': {'widgets': [{'id': '0827c801-8b7e-46f3-b214-9ba7d160307b',
    'name': 'clone_test_violet',
    'showLastUpdate': False,
    'noDataState': {'verticalAlignment': 'middle',
     'showCaption': True,
     'showDescription': True},
    'noFilterState': {'verticalAlignment': 'middle',
     'showCaption': True,
     'showDescription': True},
    'type': 'mapWidget',
    'flashRepeats': 3,
    'itemId': '43f9883010094f8ab07496608a9a8620',
    'mapTools': [],
    'showNavigation': False,
    'showLocate': False,
    'showCompass': False,
    'showPopup': True,
    'scalebarStyle': 'none',
    'groupSelect': 'none'},
   {'id': '66990c50-78d7-49c6-a8c8-7c7e840b7cda',
    'name': 'Pie chart (1)',
    'showLastUpdate': False,
    'noDataState': {'verticalAlignment': 'middle',
     'showCaption': True,
     'showDescription': True},
    'noFilterState': {'verticalAlignment': 'middle',
     'showCaption': True,
     'showDescription': True},
    'datasets': [{'type': 'serviceDataset',
      'name': 'main',
      'dataSource': {'type': 'layerDataSource',
       'itemId': '1a87ee96c67b431b8ea88cf7ade1ab58',
       'layerId': 0},
      'groupByFields': ['state'],
      'orderByFields': [],
      'statisticDefinitions': [{'onStatisticField': 'fid0',
        'outStatisticFieldName': 'value',
        'statisticType': 'count'}],
      'clientSideStatistics': False,
      'outFields': ['*'],
      'returnDistinctValues': False,
      'allowSourceDownload': False,
      'allowSummaryDownload': False}],
    'selectionMode': 'single',
    'categoryType': 'groupByValues',
    'type': 'pieChartWidget',
    'category': {'fieldName': 'state',
     'nullLabel': 'null',
     'blankLabel': 'blank',
     'defaultColor': '#d6d6d6',
     'nullColor': '#d6d6d6',
     'blankColor': '#d6d6d6',
     'sliceProperties': [{'key': 'CA', 'label': 'CA', 'color': '#ffaa00'},
      {'key': 'OR', 'label': 'OR', 'color': '#ffff00'}]},
    'pie': {'type': 'pie',
     'titleField': 'category',
     'valueField': 'absoluteValue',
     'alpha': 1,
     'outlineAlpha': 0,
     'outlineColor': '',
     'outlineThickness': 1,
     'innerRadius': 0,
     'labelsEnabled': True,
     'labelsFormat': 'percentage',
     'labelTickAlpha': 0.5,
     'labelTickColor': '',
     'maxLabelWidth': 100,
     'startAngle': 90,
     'groupedColor': '#d6d6d6'},
    'legend': {'enabled': False,
     'format': 'percentage',
     'position': 'bottom',
     'markerSize': 15,
     'markerType': 'circle',
     'align': 'center',
     'labelWidth': 100,
     'valueWidth': 50},
    'showBalloon': True,
    'valueFormat': {'name': 'value',
     'prefix': True,
     'style': 'decimal',
     'useGrouping': True,
     'maximumFractionDigits': 1},
    'percentageFormat': {'name': 'percentage',
     'prefix': False,
     'style': 'percent',
     'useGrouping': True,
     'maximumFractionDigits': 2}}],
  'layout': {'type': 'dockingLayout',
   'rootElement': {'width': 1,
    'height': 1,
    'id': 'a461757a-b9ee-46f7-a38b-d58040d20b10',
    'elements': [{'width': 1,
      'height': 1,
      'type': 'itemLayoutElement',
      'id': '0827c801-8b7e-46f3-b214-9ba7d160307b'},
     {'width': 1,
      'height': 1,
      'type': 'itemLayoutElement',
      'id': '66990c50-78d7-49c6-a8c8-7c7e840b7cda'}],
    'type': 'tabsLayoutElement'}},
  'type': 'desktop',
  'settings': {'allowElementResizing': False,
   'allowElementExpansion': True,
   'allowReset': False}},
 'elementMappings': {}}

We can see from this that there are multiple different item ID's contained inside this structure- instead of dissecting it down, we can just figure out what the item ID of the purple webmap that we want to replace is.

dash_list[0].id

'43f9883010094f8ab07496608a9a8620'

So there we have it- all we need to know is that. We don't even need to figure out where in the Dashboard structure it belongs, thanks to remap_data(). No matter what the structure of the data is or where the item information is stored, the function will automatically handle replacing it in the correct spots, meaning that any json-based application can be used with it. This includes, but isn't limited to, Web Maps/Scenes, StoryMaps, Experiences, Dashboards, and Hub Sites/Applications.

Let's grab our item we want to replace it with- we know the ID already.

# our green map
gis3.content.get("d61230b48c174b379a155d97eae11356")

clone_test_evergreen

Web Map by another_online_user
Last Modified: July 13, 2023
0 comments, 8 views

Just like we would with an item_mapping dictionary in clone_items(), we'll create a dictionary mapping the item ID we want to replace to the item ID with the new one. By default, remap_data() will do a check to make sure that the new item ID exists, and that it's of the same type as the original. However, if we set force to True, it will skip over the check- doing this is only recommended for cases where the original item ID may not correspond to an existent/accessible item, or if we want to replace something that's not an item ID (more on this later).

mapping = {"43f9883010094f8ab07496608a9a8620" : "d61230b48c174b379a155d97eae11356"}
cloned_dash.remap_data(mapping)

True

cloned_dash

Really Exciting Dashboard

Dashboard by another_online_user
Last Modified: July 13, 2023
0 comments, 3 views

And just like that, we've replaced it. We'll take a look at the final product of the cloned one to confirm it looks the same, but with a green map instead.

Let's take a look at the structure of the widget we just replaced.

cloned_dash.get_data()["desktopView"]["widgets"][0]

{'id': '0827c801-8b7e-46f3-b214-9ba7d160307b',
 'name': 'clone_test_violet',
 'showLastUpdate': False,
 'noDataState': {'verticalAlignment': 'middle',
  'showCaption': True,
  'showDescription': True},
 'noFilterState': {'verticalAlignment': 'middle',
  'showCaption': True,
  'showDescription': True},
 'type': 'mapWidget',
 'flashRepeats': 3,
 'itemId': 'd61230b48c174b379a155d97eae11356',
 'mapTools': [],
 'showNavigation': False,
 'showLocate': False,
 'showCompass': False,
 'showPopup': True,
 'scalebarStyle': 'none',
 'groupSelect': 'none'}

As we can see, we replaced the item ID, and now our Dashboard is showing the correct map. However, it didn't replace the name of the original map- it still says "clone_test_violet". In the future, we intend on adding the option to automatically replace all relevant information within the structure as well. But for now, if we want to replace the name (or any other string), we can do that manually through another call to remap_data()- this time with force set to True, which allows us to do replacement of any sort of string or sequence.

This time, we'll create another mapping dictionary, but this time we'll use that original map name as the key, and a new name for the map inside the widget as our value. For the sake of this Dashboard, it won't make much of a difference in what we display, but this functionality is especially powerful if the names of your internal items are being used in visible titles, or if you find a typo in your application that you want to fix throughout.

mapping = {"clone_test_violet" : "Super Cool Green Map That We Love"}
cloned_dash.remap_data(mapping, force = True)

True

cloned_dash.get_data()["desktopView"]["widgets"][0]

{'id': '0827c801-8b7e-46f3-b214-9ba7d160307b',
 'name': 'Super Cool Green Map That We Love',
 'showLastUpdate': False,
 'noDataState': {'verticalAlignment': 'middle',
  'showCaption': True,
  'showDescription': True},
 'noFilterState': {'verticalAlignment': 'middle',
  'showCaption': True,
  'showDescription': True},
 'type': 'mapWidget',
 'flashRepeats': 3,
 'itemId': 'd61230b48c174b379a155d97eae11356',
 'mapTools': [],
 'showNavigation': False,
 'showLocate': False,
 'showCompass': False,
 'showPopup': True,
 'scalebarStyle': 'none',
 'groupSelect': 'none'}

And there you have it! Our widget title has changed.

Now you see how many tools you have for moving your complex web apps from one portal to another, and making sure you can always clone your dependent items, point your apps to appropriate ones, or quickly fix other parts of the structure in need of changing.

Debugging using HTTP logging

Finally, here's a little bonus troubleshooting approach we can take when cloning isn't going as planned: the HTTP and error logs. What if something is going wrong with cloning, and we can't immediately tell that it's an error, or that an item isn't being remapped correctly? Say we've had an issue cloning a webmap to a particular Enterprise portal- let's enable logging, and then see what happens when we try it.

import http.client
import logging

# Debug logging

http.client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
req_log = logging.getLogger('requests.packages.urllib3')
req_log.setLevel(logging.DEBUG)
req_log.propagate = True

webmap = gis2.content.get("d788e23ff6ac43d9832dd28fe4125882")
gis1.content.clone_items(items = [webmap], search_existing_items = False)

DEBUG:urllib3.connectionpool:Resetting dropped connection: geosaurus.maps.arcgis.com
DEBUG:urllib3.connectionpool:https://geosaurus.maps.arcgis.com:443 "POST /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882 HTTP/1.1" 200 None

send: b'POST /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882 HTTP/1.1\r\nHost: geosaurus.maps.arcgis.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 6\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer Wt0mo_87PAwlUBwZe7OVFAXgG6PqR0PRmTOc5X1dPIaqFT9WgKlUWR5gbP0CJxA_g9sgMql8p_OJQCTcxv_MskUMveaq5xz3awty52BGNu-eRHGEhnxKYSzZpS0U0e7sJctCLKGhPdK15xgKU7OsxsHsZHdr1uo-_crjuuqwF45EymSEZd4JwgFMCID8teAQHxi4_gTyVSk1Wd4NuiyBDOAVR9dIziecTm2gE7pdWn0.\r\n\r\n'
send: b'f=json'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 13 Jul 2023 22:07:26 GMT
header: Content-Type: application/json;charset=utf-8
header: Transfer-Encoding: chunked
header: Connection: keep-alive
header: Vary: Origin
header: X-Content-Type-Options: nosniff
header: Cache-Control: no-cache
header: Pragma: no-cache
header: Expires: -1
header: Strict-Transport-Security: max-age=31536000
header: Vary: Accept-Encoding
header: Content-Encoding: gzip
send: b'POST /JEwYeAy2cc8qOe3o/arcgis/rest/services/ac5b86/FeatureServer HTTP/1.1\r\nHost: services7.arcgis.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 6\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer Wt0mo_87PAwlUBwZe7OVFAXgG6PqR0PRmTOc5X1dPIaqFT9WgKlUWR5gbP0CJxA_g9sgMql8p_OJQCTcxv_MskUMveaq5xz3awty52BGNu-eRHGEhnxKYSzZpS0U0e7sJctCLKGhPdK15xgKU7OsxsHsZHdr1uo-_crjuuqwF45EymSEZd4JwgFMCID8teAQHxi4_gTyVSk1Wd4NuiyBDOAVR9dIziecTm2gE7pdWn0.\r\n\r\n'
send: b'f=json'

DEBUG:urllib3.connectionpool:https://geosaurus.maps.arcgis.com:443 "GET /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882/data?f=json HTTP/1.1" 200 0

reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=utf-8
header: Content-Length: 969
header: Connection: keep-alive
header: Date: Thu, 13 Jul 2023 22:07:26 GMT
header: Cache-Control: private, max-age=30, s-maxage=30
header: Content-Encoding: br
header: Last-Modified: Wed, 12 Jul 2023 19:15:33 GMT
header: ETag: sds_1431908412
header: Request-Context: appId=cid-v1:e760746d-7077-47f1-a125-f6cc9f1bba28
header: x-arcgis-upstream: us1h07c00
header: X-ArcGIS-Trace-Id: ede182b0b85fffd63b87888f2071d38a
header: X-ArcGIS-Correlation-Id: 00-ede182b0b85fffd63b87888f2071d38a-5e8461b1e0c3daa4-00
header: X-ArcGIS-Instance: lf3xzxihb000002
header: Access-Control-Allow-Origin: *
header: X-Cache: Miss from cloudfront
header: Via: 1.1 f858657101c473fa0bfd3f708857c7c6.cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: LAX50-P2
header: X-Amz-Cf-Id: lqsU3i15lrSVwzUVa0EW1FsQd0I6lpp1CBzVdrRO9IacBFFULF5YZg==
send: b'GET /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882/data?f=json HTTP/1.1\r\nHost: geosaurus.maps.arcgis.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nX-Esri-Authorization: Bearer Wt0mo_87PAwlUBwZe7OVFAXgG6PqR0PRmTOc5X1dPIaqFT9WgKlUWR5gbP0CJxA_g9sgMql8p_OJQCTcxv_MskUMveaq5xz3awty52BGNu-eRHGEhnxKYSzZpS0U0e7sJctCLKGhPdK15xgKU7OsxsHsZHdr1uo-_crjuuqwF45EymSEZd4JwgFMCID8teAQHxi4_gTyVSk1Wd4NuiyBDOAVR9dIziecTm2gE7pdWn0.\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 13 Jul 2023 22:07:27 GMT
header: Content-Type: application/json;charset=utf-8
header: Content-Length: 0
header: Connection: keep-alive
header: Vary: Origin
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31536000
header: ETag: 1894b87a480
header: Cache-Control: max-age=0, must-revalidate
header: Vary: Accept-Encoding
send: b'POST /JEwYeAy2cc8qOe3o/arcgis/rest/services/ac5b86/FeatureServer/0 HTTP/1.1\r\nHost: services7.arcgis.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 6\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer Wt0mo_87PAwlUBwZe7OVFAXgG6PqR0PRmTOc5X1dPIaqFT9WgKlUWR5gbP0CJxA_g9sgMql8p_OJQCTcxv_MskUMveaq5xz3awty52BGNu-eRHGEhnxKYSzZpS0U0e7sJctCLKGhPdK15xgKU7OsxsHsZHdr1uo-_crjuuqwF45EymSEZd4JwgFMCID8teAQHxi4_gTyVSk1Wd4NuiyBDOAVR9dIziecTm2gE7pdWn0.\r\n\r\n'
send: b'f=json'

DEBUG:urllib3.connectionpool:https://geosaurus.maps.arcgis.com:443 "GET /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882/info/metadata/metadata.xml HTTP/1.1" 400 522

reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json; charset=utf-8
header: Content-Length: 4232
header: Connection: keep-alive
header: Date: Thu, 13 Jul 2023 22:07:27 GMT
header: Cache-Control: private, max-age=30, s-maxage=30
header: Content-Encoding: br
header: Last-Modified: Wed, 12 Jul 2023 19:15:33 GMT
header: ETag: 2143521137
header: Request-Context: appId=cid-v1:e760746d-7077-47f1-a125-f6cc9f1bba28
header: x-arcgis-upstream: us1h07c00
header: X-ArcGIS-Trace-Id: 48cd2a02fafd027a9ae78710669c2216
header: X-ArcGIS-Correlation-Id: 00-48cd2a02fafd027a9ae78710669c2216-c26e6f1bdcc6c663-00
header: X-ArcGIS-Instance: lf3xzxihb000003
header: Access-Control-Allow-Origin: *
header: X-Cache: Miss from cloudfront
header: Via: 1.1 f858657101c473fa0bfd3f708857c7c6.cloudfront.net (CloudFront)
header: X-Amz-Cf-Pop: LAX50-P2
header: X-Amz-Cf-Id: p4osHEU65qHZO081qmjvRrBNqnDwpPmcP_DBKJ5723IPLvj1U44LzA==
send: b'GET /sharing/rest/content/items/d788e23ff6ac43d9832dd28fe4125882/info/metadata/metadata.xml HTTP/1.1\r\nHost: geosaurus.maps.arcgis.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nX-Esri-Authorization: Bearer Wt0mo_87PAwlUBwZe7OVFAXgG6PqR0PRmTOc5X1dPIaqFT9WgKlUWR5gbP0CJxA_g9sgMql8p_OJQCTcxv_MskUMveaq5xz3awty52BGNu-eRHGEhnxKYSzZpS0U0e7sJctCLKGhPdK15xgKU7OsxsHsZHdr1uo-_crjuuqwF45EymSEZd4JwgFMCID8teAQHxi4_gTyVSk1Wd4NuiyBDOAVR9dIziecTm2gE7pdWn0.\r\n\r\n'
reply: 'HTTP/1.1 400 Bad Request\r\n'
header: Date: Thu, 13 Jul 2023 22:07:27 GMT
header: Content-Type: text/html;charset=utf-8
header: Content-Length: 522
header: Connection: keep-alive
header: Vary: Origin
header: X-XSS-Protection: 1; mode=block
header: X-Frame-Options: SAMEORIGIN
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31536000

INFO:root:Processing Level: 0

send: b'POST /portal/sharing/rest/community/users/ent_user HTTP/1.1\r\nHost: pythonapi.playground.esri.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 6\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer LuMO37z0gF4ymNoRvtHjs9q_ywmbAbWXZA8O9tEl9u0c7eiPCEThJgCA6B5xdhQIvYThmsi51Z9_g9LGDAsMz0sY4op6PAD9Np0g7r4MH72jiplo7WjYwRZlIeuJdh7p3QXyP2t9RDYFBw_JnXSDZW-yCD_4u18mv8amMWccjNNPfiTb8U9WPodzsVqcQcqeRqGDho_hPX1r7hz_Bq934PeEw84DgUYCDxotAr4RzPY.\r\n\r\n'
send: b'f=json'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 13 Jul 2023 22:07:27 GMT
header: Content-Type: application/json;charset=utf-8
header: Content-Length: 1369
header: Connection: keep-alive
header: Server: 
header: X-Content-Type-Options: nosniff
header: Pragma: no-cache
header: Cache-Control: no-cache
header: Content-Encoding: gzip
header: Vary: Origin
header: Expires: -1
header: X-XSS-Protection: 1; mode=block
send: b'POST /portal/sharing/rest/portals/self/isServiceNameAvailable HTTP/1.1\r\nHost: pythonapi.playground.esri.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 38\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer LuMO37z0gF4ymNoRvtHjs9q_ywmbAbWXZA8O9tEl9u0c7eiPCEThJgCA6B5xdhQIvYThmsi51Z9_g9LGDAsMz0sY4op6PAD9Np0g7r4MH72jiplo7WjYwRZlIeuJdh7p3QXyP2t9RDYFBw_JnXSDZW-yCD_4u18mv8amMWccjNNPfiTb8U9WPodzsVqcQcqeRqGDho_hPX1r7hz_Bq934PeEw84DgUYCDxotAr4RzPY.\r\n\r\n'
send: b'f=json&name=ac5b86&type=featureService'

DEBUG:urllib3.connectionpool:https://pythonapi.playground.esri.com:443 "POST /portal/sharing/rest/portals/self/isServiceNameAvailable HTTP/1.1" 200 38

reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 13 Jul 2023 22:07:27 GMT
header: Content-Type: application/json;charset=utf-8
header: Content-Length: 38
header: Connection: keep-alive
header: Server: 
header: X-Content-Type-Options: nosniff
header: Pragma: no-cache
header: Cache-Control: no-cache
header: Content-Encoding: gzip
header: Vary: Origin
header: Expires: -1
header: X-XSS-Protection: 1; mode=block
send: b'POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1\r\nHost: pythonapi.playground.esri.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 2577\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer LuMO37z0gF4ymNoRvtHjs9q_ywmbAbWXZA8O9tEl9u0c7eiPCEThJgCA6B5xdhQIvYThmsi51Z9_g9LGDAsMz0sY4op6PAD9Np0g7r4MH72jiplo7WjYwRZlIeuJdh7p3QXyP2t9RDYFBw_JnXSDZW-yCD_4u18mv8amMWccjNNPfiTb8U9WPodzsVqcQcqeRqGDho_hPX1r7hz_Bq934PeEw84DgUYCDxotAr4RzPY.\r\n\r\n'
send: b'f=json&createParameters=%7B%22currentVersion%22%3A+11.1%2C+%22serviceItemId%22%3A+%22d788e23ff6ac43d9832dd28fe4125882%22%2C+%22serviceDescription%22%3A+%22%22%2C+%22hasVersionedData%22%3A+false%2C+%22supportsDisconnectedEditing%22%3A+false%2C+%22hasStaticData%22%3A+true%2C+%22hasSharedDomains%22%3A+false%2C+%22maxRecordCount%22%3A+2000%2C+%22supportedQueryFormats%22%3A+%22JSON%22%2C+%22supportsVCSProjection%22%3A+false%2C+%22supportedExportFormats%22%3A+%22csv%2Cshapefile%2Csqlite%2CgeoPackage%2Cfilegdb%2CfeatureCollection%2Cgeojson%2Cexcel%22%2C+%22capabilities%22%3A+%22Query%22%2C+%22description%22%3A+%22%22%2C+%22copyrightText%22%3A+%22%22%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%2C+%22initialExtent%22%3A+%7B%22xmin%22%3A+-20023815.92308155%2C+%22ymin%22%3A+-8904526.928435927%2C+%22xmax%22%3A+20034803.279162966%2C+%22ymax%22%3A+17395888.112442996%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%7D%2C+%22allowGeometryUpdates%22%3A+true%2C+%22units%22%3A+%22esriMeters%22%2C+%22supportsAppend%22%3A+true%2C+%22supportsSharedDomains%22%3A+true%2C+%22supportsWebHooks%22%3A+true%2C+%22supportsTemporalLayers%22%3A+true%2C+%22layerOverridesEnabled%22%3A+true%2C+%22size%22%3A+2793472%2C+%22syncEnabled%22%3A+false%2C+%22supportsApplyEditsWithGlobalIds%22%3A+false%2C+%22supportsReturnDeleteResults%22%3A+true%2C+%22supportsLayerOverrides%22%3A+true%2C+%22supportsTilesAndBasicQueriesMode%22%3A+true%2C+%22supportsQueryContingentValues%22%3A+true%2C+%22supportedContingentValuesFormats%22%3A+%22JSON%2C+PBF%22%2C+%22supportsContingentValuesJson%22%3A+2%2C+%22advancedEditingCapabilities%22%3A+%7B%22supportsSplit%22%3A+false%2C+%22supportsReturnServiceEditsInSourceSR%22%3A+false%2C+%22supportsAsyncApplyEdits%22%3A+true%2C+%22supportsReturnEditResults%22%3A+true%2C+%22supportsApplyEditsbyUploadID%22%3A+true%2C+%22supportedApplyEditsUploadIDFormats%22%3A+%22JSON%22%7D%2C+%22editorTrackingInfo%22%3A+%7B%22enableEditorTracking%22%3A+false%2C+%22enableOwnershipAccessControl%22%3A+false%2C+%22allowOthersToQuery%22%3A+true%2C+%22allowOthersToUpdate%22%3A+true%2C+%22allowOthersToDelete%22%3A+false%2C+%22allowAnonymousToQuery%22%3A+true%2C+%22allowAnonymousToUpdate%22%3A+true%2C+%22allowAnonymousToDelete%22%3A+true%7D%2C+%22xssPreventionInfo%22%3A+%7B%22xssPreventionEnabled%22%3A+true%2C+%22xssPreventionRule%22%3A+%22InputOnly%22%2C+%22xssInputRule%22%3A+%22rejectInvalid%22%7D%2C+%22name%22%3A+%22ac5b86%22%2C+%22preserveLayerIds%22%3A+true%7D&outputType=featureService&isView=false'

DEBUG:urllib3.connectionpool:https://pythonapi.playground.esri.com:443 "POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1" 504 132
DEBUG:urllib3.util.retry:Incremented Retry for (url='/portal/sharing/rest/content/users/ent_user/createService'): Retry(total=3, connect=5, read=5, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Retry: /portal/sharing/rest/content/users/ent_user/createService

send: b'POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1\r\nHost: pythonapi.playground.esri.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 2577\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer LuMO37z0gF4ymNoRvtHjs9q_ywmbAbWXZA8O9tEl9u0c7eiPCEThJgCA6B5xdhQIvYThmsi51Z9_g9LGDAsMz0sY4op6PAD9Np0g7r4MH72jiplo7WjYwRZlIeuJdh7p3QXyP2t9RDYFBw_JnXSDZW-yCD_4u18mv8amMWccjNNPfiTb8U9WPodzsVqcQcqeRqGDho_hPX1r7hz_Bq934PeEw84DgUYCDxotAr4RzPY.\r\n\r\n'
send: b'f=json&createParameters=%7B%22currentVersion%22%3A+11.1%2C+%22serviceItemId%22%3A+%22d788e23ff6ac43d9832dd28fe4125882%22%2C+%22serviceDescription%22%3A+%22%22%2C+%22hasVersionedData%22%3A+false%2C+%22supportsDisconnectedEditing%22%3A+false%2C+%22hasStaticData%22%3A+true%2C+%22hasSharedDomains%22%3A+false%2C+%22maxRecordCount%22%3A+2000%2C+%22supportedQueryFormats%22%3A+%22JSON%22%2C+%22supportsVCSProjection%22%3A+false%2C+%22supportedExportFormats%22%3A+%22csv%2Cshapefile%2Csqlite%2CgeoPackage%2Cfilegdb%2CfeatureCollection%2Cgeojson%2Cexcel%22%2C+%22capabilities%22%3A+%22Query%22%2C+%22description%22%3A+%22%22%2C+%22copyrightText%22%3A+%22%22%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%2C+%22initialExtent%22%3A+%7B%22xmin%22%3A+-20023815.92308155%2C+%22ymin%22%3A+-8904526.928435927%2C+%22xmax%22%3A+20034803.279162966%2C+%22ymax%22%3A+17395888.112442996%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%7D%2C+%22allowGeometryUpdates%22%3A+true%2C+%22units%22%3A+%22esriMeters%22%2C+%22supportsAppend%22%3A+true%2C+%22supportsSharedDomains%22%3A+true%2C+%22supportsWebHooks%22%3A+true%2C+%22supportsTemporalLayers%22%3A+true%2C+%22layerOverridesEnabled%22%3A+true%2C+%22size%22%3A+2793472%2C+%22syncEnabled%22%3A+false%2C+%22supportsApplyEditsWithGlobalIds%22%3A+false%2C+%22supportsReturnDeleteResults%22%3A+true%2C+%22supportsLayerOverrides%22%3A+true%2C+%22supportsTilesAndBasicQueriesMode%22%3A+true%2C+%22supportsQueryContingentValues%22%3A+true%2C+%22supportedContingentValuesFormats%22%3A+%22JSON%2C+PBF%22%2C+%22supportsContingentValuesJson%22%3A+2%2C+%22advancedEditingCapabilities%22%3A+%7B%22supportsSplit%22%3A+false%2C+%22supportsReturnServiceEditsInSourceSR%22%3A+false%2C+%22supportsAsyncApplyEdits%22%3A+true%2C+%22supportsReturnEditResults%22%3A+true%2C+%22supportsApplyEditsbyUploadID%22%3A+true%2C+%22supportedApplyEditsUploadIDFormats%22%3A+%22JSON%22%7D%2C+%22editorTrackingInfo%22%3A+%7B%22enableEditorTracking%22%3A+false%2C+%22enableOwnershipAccessControl%22%3A+false%2C+%22allowOthersToQuery%22%3A+true%2C+%22allowOthersToUpdate%22%3A+true%2C+%22allowOthersToDelete%22%3A+false%2C+%22allowAnonymousToQuery%22%3A+true%2C+%22allowAnonymousToUpdate%22%3A+true%2C+%22allowAnonymousToDelete%22%3A+true%7D%2C+%22xssPreventionInfo%22%3A+%7B%22xssPreventionEnabled%22%3A+true%2C+%22xssPreventionRule%22%3A+%22InputOnly%22%2C+%22xssInputRule%22%3A+%22rejectInvalid%22%7D%2C+%22name%22%3A+%22ac5b86%22%2C+%22preserveLayerIds%22%3A+true%7D&outputType=featureService&isView=false'

DEBUG:urllib3.connectionpool:https://pythonapi.playground.esri.com:443 "POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1" 504 132
DEBUG:urllib3.util.retry:Incremented Retry for (url='/portal/sharing/rest/content/users/ent_user/createService'): Retry(total=4, connect=5, read=5, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Retry: /portal/sharing/rest/content/users/ent_user/createService

reply: 'HTTP/1.1 504 Gateway Time-out\r\n'
header: Server: awselb/2.0
header: Date: Thu, 13 Jul 2023 22:08:27 GMT
header: Content-Type: text/html
header: Content-Length: 132
header: Connection: keep-alive
send: b'POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1\r\nHost: pythonapi.playground.esri.com\r\nUser-Agent: Geosaurus/2.2.0\r\nAccept-Encoding: gzip, deflate, br\r\nAccept: */*\r\nConnection: keep-alive\r\nReferer: http\r\nContent-Length: 2577\r\nContent-Type: application/x-www-form-urlencoded\r\nX-Esri-Authorization: Bearer LuMO37z0gF4ymNoRvtHjs9q_ywmbAbWXZA8O9tEl9u0c7eiPCEThJgCA6B5xdhQIvYThmsi51Z9_g9LGDAsMz0sY4op6PAD9Np0g7r4MH72jiplo7WjYwRZlIeuJdh7p3QXyP2t9RDYFBw_JnXSDZW-yCD_4u18mv8amMWccjNNPfiTb8U9WPodzsVqcQcqeRqGDho_hPX1r7hz_Bq934PeEw84DgUYCDxotAr4RzPY.\r\n\r\n'
send: b'f=json&createParameters=%7B%22currentVersion%22%3A+11.1%2C+%22serviceItemId%22%3A+%22d788e23ff6ac43d9832dd28fe4125882%22%2C+%22serviceDescription%22%3A+%22%22%2C+%22hasVersionedData%22%3A+false%2C+%22supportsDisconnectedEditing%22%3A+false%2C+%22hasStaticData%22%3A+true%2C+%22hasSharedDomains%22%3A+false%2C+%22maxRecordCount%22%3A+2000%2C+%22supportedQueryFormats%22%3A+%22JSON%22%2C+%22supportsVCSProjection%22%3A+false%2C+%22supportedExportFormats%22%3A+%22csv%2Cshapefile%2Csqlite%2CgeoPackage%2Cfilegdb%2CfeatureCollection%2Cgeojson%2Cexcel%22%2C+%22capabilities%22%3A+%22Query%22%2C+%22description%22%3A+%22%22%2C+%22copyrightText%22%3A+%22%22%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%2C+%22initialExtent%22%3A+%7B%22xmin%22%3A+-20023815.92308155%2C+%22ymin%22%3A+-8904526.928435927%2C+%22xmax%22%3A+20034803.279162966%2C+%22ymax%22%3A+17395888.112442996%2C+%22spatialReference%22%3A+%7B%22wkid%22%3A+102100%2C+%22latestWkid%22%3A+3857%7D%7D%2C+%22allowGeometryUpdates%22%3A+true%2C+%22units%22%3A+%22esriMeters%22%2C+%22supportsAppend%22%3A+true%2C+%22supportsSharedDomains%22%3A+true%2C+%22supportsWebHooks%22%3A+true%2C+%22supportsTemporalLayers%22%3A+true%2C+%22layerOverridesEnabled%22%3A+true%2C+%22size%22%3A+2793472%2C+%22syncEnabled%22%3A+false%2C+%22supportsApplyEditsWithGlobalIds%22%3A+false%2C+%22supportsReturnDeleteResults%22%3A+true%2C+%22supportsLayerOverrides%22%3A+true%2C+%22supportsTilesAndBasicQueriesMode%22%3A+true%2C+%22supportsQueryContingentValues%22%3A+true%2C+%22supportedContingentValuesFormats%22%3A+%22JSON%2C+PBF%22%2C+%22supportsContingentValuesJson%22%3A+2%2C+%22advancedEditingCapabilities%22%3A+%7B%22supportsSplit%22%3A+false%2C+%22supportsReturnServiceEditsInSourceSR%22%3A+false%2C+%22supportsAsyncApplyEdits%22%3A+true%2C+%22supportsReturnEditResults%22%3A+true%2C+%22supportsApplyEditsbyUploadID%22%3A+true%2C+%22supportedApplyEditsUploadIDFormats%22%3A+%22JSON%22%7D%2C+%22editorTrackingInfo%22%3A+%7B%22enableEditorTracking%22%3A+false%2C+%22enableOwnershipAccessControl%22%3A+false%2C+%22allowOthersToQuery%22%3A+true%2C+%22allowOthersToUpdate%22%3A+true%2C+%22allowOthersToDelete%22%3A+false%2C+%22allowAnonymousToQuery%22%3A+true%2C+%22allowAnonymousToUpdate%22%3A+true%2C+%22allowAnonymousToDelete%22%3A+true%7D%2C+%22xssPreventionInfo%22%3A+%7B%22xssPreventionEnabled%22%3A+true%2C+%22xssPreventionRule%22%3A+%22InputOnly%22%2C+%22xssInputRule%22%3A+%22rejectInvalid%22%7D%2C+%22name%22%3A+%22ac5b86%22%2C+%22preserveLayerIds%22%3A+true%7D&outputType=featureService&isView=false'

DEBUG:urllib3.connectionpool:https://pythonapi.playground.esri.com:443 "POST /portal/sharing/rest/content/users/ent_user/createService HTTP/1.1" 504 132
DEBUG:urllib3.util.retry:Incremented Retry for (url='/portal/sharing/rest/content/users/ent_user/createService'): Retry(total=2, connect=5, read=5, redirect=None, status=None)
DEBUG:urllib3.connectionpool:Retry: /portal/sharing/rest/content/users/ent_user/createService

As you can see, it gets stuck after calling POST- we can interpret that this means there's an issue with our Enterprise deployment, and we would have to investigate that in order to fix our cloning.

Cloning and Troubleshooting Complex Items

Updated Cloning Types

'Complex Items' and search_existing_items

Web Experiences

Local/Developer Edition

StoryMaps, Web Scenes, and Item Remapping

Item Remapping

Dashboards, Editing Item Data, and remap_data()

remap_data()

Debugging using HTTP logging

'Complex Items' and `search_existing_items`

Dashboards, Editing Item Data, and `remap_data()`

`remap_data()`