Enrich Standard Geographies

Standard geographies are jurisdictional areas determined by government agencies. At the highest level these are the countries of the world. Within these countries the hierarchical levels have different names. If you already have a list of jurisdictional area identifiers such as postal (ZIP) codes or US Census Block Group Identifiers, these can be used directly as input to the enrich method to retrieve demographic information about these jurisdictional areas for analysis.

Example Use Case - Variable Variance

Just as before, we are going to retrieve high variance variables, but this time we are going to look up the unique identifiers for all the US Census Block Groups in Seattle.

Create a Country

Our analysis starts with identifying the country we are going to be working with and instantating an arcgis.geoenrichment.Country object referencing an arcgis.gis.GIS source to use for analysis.

from arcgis.geoenrichment import Country
from arcgis.gis import GIS

gis = GIS(profile="your_online_profile")
usa = Country("usa", gis=gis)

usa

<Country - United States (GIS @ https://geosaurus.maps.arcgis.com version:10.3)>

Selecting Data to Start

Next, we are using Pandas DataFrame filtering to identify a subset of variables to focus on from the thousands available.

enrich_vars = (
    usa.enrich_variables[
        (usa.enrich_variables.name.str.lower().str.contains("cy"))
        & (
            (usa.enrich_variables.data_collection == "occupation")
            | (usa.enrich_variables.data_collection == "Wealth")
            | (usa.enrich_variables.data_collection == "financial")
            | (usa.enrich_variables.data_collection == "educationalattainment")
            | (usa.enrich_variables.data_collection == "language")
            | (usa.enrich_variables.data_collection == "healthinsurancecoverage")
            | (usa.enrich_variables.data_collection == "veterans")
            | (usa.enrich_variables.data_collection == "yearmovedin")
            | (usa.enrich_variables.data_collection == "yearbuilt")
            | (usa.enrich_variables.data_collection == "population")
            | (usa.enrich_variables.data_collection == "occupation")
            | (usa.enrich_variables.data_collection == "housingcosts")
        )
    ]
    .drop_duplicates("name")
    .reset_index(drop=True)
)

enrich_vars

Get the Geographic Level

Here, we are retrieving levels and using the level_name colum to discover valid values for the enrich method's standard_geography_level parameter.

usa.levels

Retrive Seattle Block Groups

It is not uncommon to already have the standard geography unique identifiers, if you need to retrieve those within a larger area, you can retrieve these using standard_geography_query. The most versatile parameter in this method is geoquery. You can find more explanation of the options for the geoquery parameter under the geographyQuery parameter documentation. We can start by seeing what is returned when searching for seattle.

from arcgis.geoenrichment import standard_geography_query

standard_geography_query("usa", layers="US.Places", geoquery="seattle")

Since only one location is returned, we can use this to retrieve the block groups by populating the sub_geography parameteters.

bg_df = standard_geography_query(
    "usa",
    layers="US.Places",
    geoquery="seattle",
    sub_geography_layer="US.BlockGroups",
    return_sub_geography=True,
)

bg_df.head()

Enrich

Now, we can use the retrieved block groups as input into the enrich method to acheive the same results.

enrich_df = usa.enrich(
    bg_df,
    enrich_variables=enrich_vars,
    standard_geography_level="block_groups",
    standard_geography_id_column="AreaID",
)

enrich_df.head()

Calculate Variance

Variation can now be calculated for the retrieved variables to identify those with exceedingly high variance. Analysis can be used for feature selection or feature reduction to address covariance between variables and perform modeling.

Just as in the previous notebook, we can evaluate the variance and select the top variables.

# get just the enrich value columns
enrich_cols = [
    c for c in enrich_df if c in usa.enrich_variables.name.str.lower().values
]
enrich_df = enrich_df.set_index("std_geography_id").loc[:, enrich_cols]

# get top 20 highest variance columns
top20 = enrich_df.var(ddof=0).sort_values(ascending=False).iloc[:20]
top20.name = "variance"

# add human readable names
ev = usa.enrich_variables
ev.index = ev.name.str.lower()
top20_df = ev.join(top20, how="right").loc[:, ["name", "alias", "variance"]]

top20_df.head()

Continuing Analysis

From here, a variety of techniques can be used, but with so many income and net worth variables, before subsequent modeling steps, covariance needs to be addressed. Using the GeoEnrichment dramatically streamlines getting to this point. It provides extremely easy access to thousands of demographic variables for modeling and analysis directly in Python, making it easy to integrate with data engineering pipelines.