{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Network Analysis: Track river pollutants\n", "\n", ">* 👟 Ready To Run!\n", "* 🖥️ Requires Hydrology Utility Organization Configuration\n", "* 🔬 Data Science\n", "\n", "\n", "## No Dumping - Drains to Ocean\n", "\n", "Have you ever seen a sign near a storm drain that says \"No Dumping - Drains to Ocean\"? If you live inland, you might have seen one that says Drains to River, Lake, Bay, or Waterway. All of these are messages about the same thing—anything washed or dumped into the drain will be transported downstream and has the potential to pollute local waterways and the larger water bodies they connect to. For example, pollution carried down the Mississippi from Midwestern farms has created a dead zone in the Gulf of Mexico the size of Massachusetts where fish and plants have to fight for survival.\n", "\n", "In this notebook, you'll learn how to find the area that drains to a storm drain, and the route that pollutants will take if they are dumped or washed into the drain. You'll find the upstream drainage area, called a watershed, for a storm drain near Blackman Elementary School in Tennessee. Then you'll find the downstream flow path to where it empties into the Gulf of Mexico. Knowing how to find these, you can experiment to find the watersheds and flow paths from other storm drain locations.\n", "\n", "This notebook uses Spatial Analysis online service tools, **Create Watershed** and **Trace Downstream**, that perform analysis on your web GIS. You can find more information about the Create Watershed analysis tool [here](https://developers.arcgis.com/rest/analysis/api-reference/create-watersheds.htm) and the Trace Downstream analysis tool [here](https://developers.arcgis.com/rest/analysis/api-reference/trace-downstream.htm)." ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting set up\n", "Import the necessary libraries, and connect to your Organization." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "\n", "import arcgis\n", "from arcgis.gis import GIS\n", "from arcgis.features import Feature, FeatureSet, FeatureCollection\n", "from arcgis.geometry import Point, distance\n", "from arcgis.features.analysis import create_watersheds\n", "from arcgis.features.analysis import trace_downstream\n", "\n", "gis = GIS(\"home\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a new map and navigate to the area of interest: Blackman Elemetary School in Tennessee." ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 97, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_map = gis.map(\"Blackman Elementary School, 586 Fortress \"\\\n", " \"Blvd, Murfreesboro, TN, 37128, USA.\")\n", "storm_drain_map.basemap = 'hybrid'\n", "storm_drain_map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us pick a storm drain point in the parking lot of a school for our analysis." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "storm_drain_feature = Feature.from_dict(\n", " {'geometry' : {'x' : -9625614.992191713,\n", " 'y' : 4280321.441688138,\n", " 'type' : 'point',\n", " 'spatialReference' : {'latestWkid' : 3857,\n", " 'wkid' : 102100}}})\n", "\n", "storm_drain_df = FeatureSet([storm_drain_feature]).sdf\n", "storm_drain_fc = storm_drain_df.spatial.to_feature_collection(\n", " 'storm_drain_fc')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Plot the point on the map. In the following section you will build watersheds for this point." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_df.spatial.plot(map_widget = storm_drain_map, \n", " symbol_type = 'simple',\n", " symbol_style = 'd', # d - for diamonds\n", " colors = 'Reds_r',\n", " cstep = 10,\n", " outline_color = 'Blues',\n", " marker_size = 20)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create watersheds for the storm drain\n", "\n", "### Create a default watershed for the storm drain point feature\n", "Use the Create Watershed tool from the `arcgis.feature.analysis` module to find the upstream contributing area to the storm drain point. You'll first create a watershed by keeping all the default parameters in the tool." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'snap_pour_pts_layer': ,\n", " 'watershed_layer': }" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "try:\n", " storm_drain_watershed = create_watersheds(storm_drain_fc)\n", "except Exception as e:\n", " print(\"Error: Organization not configured with Hydrology Utility\")\n", " raise e\n", "\n", "storm_drain_watershed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Add the storm drain watershed to the storm drain map." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "storm_drain_map.add_layer(storm_drain_watershed['snap_pour_pts_layer'], \n", " options = {'opacity' : 0.5})\n", "storm_drain_map.add_layer(storm_drain_watershed['watershed_layer'],\n", " options = {'opacity' : 0.5})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create a Spatially Enabled DataFrame from the watershed feature collection layer you created and read the records in this watershed." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
AnalysisAreaDataResolutionDescriptionOBJECTIDOIDPourPtIDSHAPEShape_AreaShape_Length
00.02241330.0NED 30m processed by Esri111{\"rings\": [[[851865, 1464885], [851895, 146485...NoneNone
\n", "
" ], "text/plain": [ " AnalysisArea DataResolution Description OBJECTID OID \\\n", "0 0.022413 30.0 NED 30m processed by Esri 1 1 \n", "\n", " PourPtID SHAPE Shape_Area \\\n", "0 1 {\"rings\": [[[851865, 1464885], [851895, 146485... None \n", "\n", " Shape_Length \n", "0 None " ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_watershed_fc = storm_drain_watershed['watershed_layer']\n", "storm_drain_watershed_fc.query().sdf" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As you can see from the watershed feature layer collection added to the map and the `DataFrame` queried above, the size of this watershed is quite small (`0.022` square miles). These results are not unusual. If your analysis point is located away from a drainage line, the resulting watershed is likely to be very small and not of much use in determining the upstream source of contamination. In most cases, your analysis point should be positioned on the nearest drainage line to find the watershed that flows to a point on the drainage line.\n", "\n", "### Create a watershed on the nearest drainage line using a search distance\n", "\n", "To find the closest drainage line, you can specify a **search distance**, which is what you’ll do in the next section. The results of this tool should be used to provide a regional understanding of the watershed rather than identify the exact location of the watershed at local scales. You'll do that next." ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'snap_pour_pts_layer': ,\n", " 'watershed_layer': }" ] }, "execution_count": 111, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_large_watershed = create_watersheds(\n", " storm_drain_fc, \n", " search_distance = 0.5, \n", " search_units = \"miles\",\n", " context = {'outSR' : {'latestWkid' : 3857,\n", " 'wkid' : 102100}})\n", "\n", "storm_drain_large_watershed" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Query the large watershed created to determine the new analysis area." ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
AnalysisAreaDataResolutionDescriptionOBJECTIDOIDPourPtIDSHAPEShape_AreaShape_Length
021.42669230.0NED 30m processed by Esri111{\"rings\": [[[-9624695.515525267, 4280553.06655...NoneNone
\n", "
" ], "text/plain": [ " AnalysisArea DataResolution Description OBJECTID OID \\\n", "0 21.426692 30.0 NED 30m processed by Esri 1 1 \n", "\n", " PourPtID SHAPE Shape_Area \\\n", "0 1 {\"rings\": [[[-9624695.515525267, 4280553.06655... None \n", "\n", " Shape_Length \n", "0 None " ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_large_watershed_fc = \\\n", " storm_drain_large_watershed['watershed_layer']\n", "storm_drain_large_watershed_fc.query().sdf" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "storm_drain_map.add_layer(\n", " storm_drain_large_watershed['watershed_layer'],\n", " options = {'opacity' : 0.5})\n", "storm_drain_map.add_layer(\n", " storm_drain_large_watershed['snap_pour_pts_layer'],\n", " options = {'opacity' : 0.5})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Zoom out of storm drain map to see the watershed from the closest drainage line. When the analysis is completed, you’ll see that another watershed has been added to the map that encompasses nearly the entire smaller watershed. Zoom out of the map to see the entire watershed. This watershed is much larger with an area of `21.42` square miles as seen by querying the DataFrame information above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Find the distance from the storm drain point to the snapped point on the nearest drainage line\n", "On the map, we are able to see both the drain point we chose and the snapped point the `create_watershed()` tool determined. Below, we will measure the distance between these two points.\n", "\n", "Read the drain points as `arcgis.geometry.Point` objects." ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [], "source": [ "snapped_drain_point = storm_drain_large_watershed['snap_pour_pts_layer']\\\n", " .query().sdf.SHAPE[0]\n", "storm_drain_point = storm_drain_df.SHAPE[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Calculate distance between these two points using the `distance()` function from the `arcgis.geometry` module" ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'distance': 477.58328236573186}" ] }, "execution_count": 118, "metadata": {}, "output_type": "execute_result" } ], "source": [ "distance({'spatialReference' : {'wkid' : 102100}}, \n", " snapped_drain_point,\n", " storm_drain_point)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the distance function returns the distance in `meters`, the default unit for the specified spatial reference. This (distance of `477.5` meters) is less than the `0.5` miles value you input before you ran the tool." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Find the downstream flow path from the storm drain\n", "\n", "You've identified where the water in the storm drain comes from. In the next section, you'll find the path it will take to the Gulf of Mexico. You'll use the **Trace Downstream** tool to find the downstream flow path." ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [], "source": [ "storm_drain_downstream_trace = trace_downstream(\n", " input_layer=storm_drain_fc)" ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 98, "metadata": {}, "output_type": "execute_result" } ], "source": [ "downstream_map = gis.map(\"Blackman Elementary School, 586 Fortress \"\\\n", " \"Blvd, Murfreesboro, TN, 37128, USA.\")\n", "downstream_map.basemap = 'gray-vector'\n", "downstream_map" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [], "source": [ "downstream_map.add_layer(\n", " storm_drain_large_watershed['watershed_layer'],\n", " options = {'opacity' : 0.5})\n", "downstream_map.add_layer(\n", " storm_drain_large_watershed['snap_pour_pts_layer'],\n", " options = {'opacity' : 0.5})\n", "\n", "downstream_map.add_layer(storm_drain_downstream_trace)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inspect the length of downstream trace from the storm drain to the Gulf of Mexico " ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
AnalysisLengthDataResolutionDescriptionOBJECTIDOIDPourPtIDSHAPEShape_Length
01270.87282330.0NED 30m processed by Esri111{\"paths\": [[[851790, 1464960], [851910, 146484...None
\n", "
" ], "text/plain": [ " AnalysisLength DataResolution Description OBJECTID OID \\\n", "0 1270.872823 30.0 NED 30m processed by Esri 1 1 \n", "\n", " PourPtID SHAPE Shape_Length \n", "0 1 {\"paths\": [[[851790, 1464960], [851910, 146484... None " ] }, "execution_count": 91, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_trace_sdf = storm_drain_downstream_trace.query().sdf\n", "storm_drain_trace_sdf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The trace is more than `1,200` miles long. \n", "\n", "### Divide the downstream trace into multiple segments\n", "\n", "The trace line can be split into multiple lines where each line is of the specified length. For example, if there is a large contamination spill near the storm drain and the river it flows at a rate of `5` miles per hour, or `120` miles per day, you might want to split the river into `120` mile segments. Splitting the trace into `120` mile intervals will show approximately where the spill will travel each day. The line will be symbolized using graduated colors based on distance and will be labeled with the distance from the start of the trace. The resulting trace will have multiple line segments, each with fields `FromDistance` and `ToDistance`." ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [], "source": [ "storm_drain_downstream_trace_split = trace_downstream(\n", " input_layer = storm_drain_fc,\n", " split_distance = 120,\n", " split_units = 'Miles')" ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [], "source": [ "downstream_map.add_layer(storm_drain_downstream_trace_split)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inspect the 'from distance' and 'to distance' of each segment in the downstream trace." ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
AnalysisLengthDataResolutionDescriptionFromDistanceOBJECTIDOIDPourPtIDSHAPEToDistanceTotalDistance
0120.00000030.0NED 30m processed by Esri0111{\"paths\": [[[851790, 1464960], [851910, 146484...120.0000001270.872823
1120.00000030.0NED 30m processed by Esri120221{\"paths\": [[[764355.7567000017, 1530430.664799...240.0000001270.872823
2120.00000030.0NED 30m processed by Esri240331{\"paths\": [[[670098.8302000016, 1596693.637700...360.0000001270.872823
3120.00000030.0NED 30m processed by Esri360441{\"paths\": [[[577682.1620999984, 1513992.159600...480.0000001270.872823
4120.00000030.0NED 30m processed by Esri480551{\"paths\": [[[547272.3764000013, 1406887.005400...600.0000001270.872823
5120.00000030.0NED 30m processed by Esri600661{\"paths\": [[[489320.02529999986, 1271903.6392]...720.0000001270.872823
6120.00000030.0NED 30m processed by Esri720771{\"paths\": [[[448150.36129999906, 1152279.11579...840.0000001270.872823
7120.00000030.0NED 30m processed by Esri840881{\"paths\": [[[459682.03990000114, 1017421.54559...960.0000001270.872823
8120.00000030.0NED 30m processed by Esri960991{\"paths\": [[[421726.77800000086, 881120.401600...1080.0000001270.872823
9120.00000030.0NED 30m processed by Esri108010101{\"paths\": [[[481982.42720000073, 794666.9408],...1200.0000001270.872823
1070.87282330.0NED 30m processed by Esri120011111{\"paths\": [[[590137.3244000003, 746022.0360000...1270.8728231270.872823
\n", "
" ], "text/plain": [ " AnalysisLength DataResolution Description FromDistance \\\n", "0 120.000000 30.0 NED 30m processed by Esri 0 \n", "1 120.000000 30.0 NED 30m processed by Esri 120 \n", "2 120.000000 30.0 NED 30m processed by Esri 240 \n", "3 120.000000 30.0 NED 30m processed by Esri 360 \n", "4 120.000000 30.0 NED 30m processed by Esri 480 \n", "5 120.000000 30.0 NED 30m processed by Esri 600 \n", "6 120.000000 30.0 NED 30m processed by Esri 720 \n", "7 120.000000 30.0 NED 30m processed by Esri 840 \n", "8 120.000000 30.0 NED 30m processed by Esri 960 \n", "9 120.000000 30.0 NED 30m processed by Esri 1080 \n", "10 70.872823 30.0 NED 30m processed by Esri 1200 \n", "\n", " OBJECTID OID PourPtID \\\n", "0 1 1 1 \n", "1 2 2 1 \n", "2 3 3 1 \n", "3 4 4 1 \n", "4 5 5 1 \n", "5 6 6 1 \n", "6 7 7 1 \n", "7 8 8 1 \n", "8 9 9 1 \n", "9 10 10 1 \n", "10 11 11 1 \n", "\n", " SHAPE ToDistance \\\n", "0 {\"paths\": [[[851790, 1464960], [851910, 146484... 120.000000 \n", "1 {\"paths\": [[[764355.7567000017, 1530430.664799... 240.000000 \n", "2 {\"paths\": [[[670098.8302000016, 1596693.637700... 360.000000 \n", "3 {\"paths\": [[[577682.1620999984, 1513992.159600... 480.000000 \n", "4 {\"paths\": [[[547272.3764000013, 1406887.005400... 600.000000 \n", "5 {\"paths\": [[[489320.02529999986, 1271903.6392]... 720.000000 \n", "6 {\"paths\": [[[448150.36129999906, 1152279.11579... 840.000000 \n", "7 {\"paths\": [[[459682.03990000114, 1017421.54559... 960.000000 \n", "8 {\"paths\": [[[421726.77800000086, 881120.401600... 1080.000000 \n", "9 {\"paths\": [[[481982.42720000073, 794666.9408],... 1200.000000 \n", "10 {\"paths\": [[[590137.3244000003, 746022.0360000... 1270.872823 \n", "\n", " TotalDistance \n", "0 1270.872823 \n", "1 1270.872823 \n", "2 1270.872823 \n", "3 1270.872823 \n", "4 1270.872823 \n", "5 1270.872823 \n", "6 1270.872823 \n", "7 1270.872823 \n", "8 1270.872823 \n", "9 1270.872823 \n", "10 1270.872823 " ] }, "execution_count": 95, "metadata": {}, "output_type": "execute_result" } ], "source": [ "storm_drain_trace_split_sdf = \\\n", " storm_drain_downstream_trace_split.query().sdf\n", "storm_drain_trace_split_sdf.head(11)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Conclusion\n", "In this sample we used `create_watershed()` and `trace_downstream()` hydrology tools to derive watershed boundaries and the downstream flow path for a given point. The analysis can be easily repated for any point on the globe.\n", "\n", "Our waterbodies are precious natural resources. Just as we saw from this sample, anything washed or dumped into the drain will be transported downstream and has the potential to pollute local waterways and the larger water bodies they connect to. " ] } ], "metadata": { "esriNotebookRuntime": { "notebookRuntimeName": "ArcGIS Notebook Python 3 Standard", "notebookRuntimeVersion": "10.7.1" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "**Table of Contents**", "title_sidebar": "Contents", "toc_cell": true, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "270.2px" }, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }