{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Crime Analysis: Boston Police" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> * 👟 Ready To Run!\n", "* 🔬 Data Science\n", "* 🚀 Data Exploration and Cleaning\n", "\n", "In this DataScience Notebook we will be exploring crime incidents in Boston, MA utilizing an open data portal. The amount of open data available for anlaysis is increasing as more and more public data is being made accessible through open data initiatives. '[Analyze Boston](https://data.boston.gov/)' is the City of Boston's open data hub and provides access to facts, figures, and maps for the city. The goal of Analyze Boston is to create the \"default technology platform to support the publication of the City's public information, in the form of data, and to make this information easy to find, access, and use by a broad audience\" - [Analyze Boston About](https://data.boston.gov/about). \n", "\n", "Getting data from an API allows you to connect to live data, in contrast to a static dataset such as a spread sheet or shapefile that has been downloaded. By using an API you also have the ability to query and subset data to get the data needed for the analysis. In a world where data keeps getting bigger this lets you focus on relevant data, instead of downloading entire datasets which might contain gigabytes of data not pertinent to your analysis. \n", "\n", "Using the Analyze Boston API we will aquire crime incidents and police precints for this analysis. The first stage of the analysis is to familiarize ourselves with and clean the data. Through an exploratory data analysis we being to build a picture of crime incidents in Boston. We use summary statistics and charting to get an idea of the overall patterns revealed in the data. The next step is to geo-enable the data and look at spatial trends. In the analysis we will use several of the tools from the arcpy Spatial Statistics and Space Time Pattern Mining toolboxes. Tools include: optimized hot spot analysis, create space time cube, emerging hot spot analysis and time series clustering.\n", "\n", "Since the data from the API is live we can also create a script to pull the latest data and use it to generate an automated report. Using the python data science notebook schedule option this report can be automatically generated and used by decision makers. The report is made using a combination of maps, plot and combined into a html document.\n", "\n", "**Outline of Steps**\n", "\n", " 1. Connect to [Analyze Boston](https://data.boston.gov/) Open Data portal and API to get police precincts and crime incidents\n", " 2. Clean the data and convert to a spatial format\n", " 3. Perform an exploratory data analysis of the data\n", " 4. Perform a spatial analysis of the data\n", " * Hotspot Analysis\n", " * Space Time Cube\n", " * Time Series Clustering\n", " 5. Create a small script to generate a weekly report from live data\n", " 6. Generate a html report for weekly scheduling" ] }, { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

**Table of Contents**

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, import all necessary libraries" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "import arcgis\n", "from arcgis import GIS\n", "import sys\n", "import json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Libraries for using Spatial Enabled Data Frame" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import pandas as pd\n", "from arcgis.features import GeoAccessor, GeoSeriesAccessor, SpatialDataFrame" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Import Arcpy and validate its version" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'10.7.0'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import arcpy\n", "arcpy.env.overwriteOutput = True\n", "arcpy.GetInstallInfo()['Version']" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "agol_gis = GIS(set_active=False)\n", "gis = GIS(\"home\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. Get data using Analyze Boston API" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this section we will connect to the Analyze Boston public API. From the API we will get a polygon feature class containing police precincts. The police precincts are hosted as an ArcGIS Feature Service and are well integrated with the python API. We will be using this polygon feature to aggregate the crimes and perform our analysis. By using the police precincts any analysis results we find will be relative to the scale and area of the police admisitrative boundaries. We will also be getting crime data from the [Crime Incident Reports](https://data.boston.gov/dataset/crime-incident-reports-august-2015-to-date-source-new-system) API. The crime incident reports data is provided by Boston Police Department (BPD) and documents the initial details of events which BPD officers respond to. We will aquire the incident type, location, time for incidents in the system. The API allows you to query specific incident types and in this analysis we will look at assaults which in the year 2017. The data is returned from the API as a JSON and can be used to create a pandas dataframe." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connect to the Police Precincts Feature Layer\n", "\n", "In this section we connect to the police precinct feature web layer hosted by Analyze Boston." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDPRECINCTPlan_DistSHAPEShape__AreaShape__LengthWARD_PRECINCT
0113None{\"rings\": [[[787723.729257345, 2965812.6409820...7.562361e+0613647.0353440113
1202None{\"rings\": [[[782820.000068769, 2960295.0999848...2.045208e+065709.0448510102
2305None{\"rings\": [[[783942.499781817, 2960316.2000169...4.427769e+069601.0636230105
3403None{\"rings\": [[[775776.888126209, 2961852.5792183...1.546242e+065352.4332020203
4505None{\"rings\": [[[774156.5998687, 2964357.69992466]...1.603861e+067036.3616670205
\n", "
" ], "text/plain": [ " OBJECTID PRECINCT Plan_Dist \\\n", "0 1 13 None \n", "1 2 02 None \n", "2 3 05 None \n", "3 4 03 None \n", "4 5 05 None \n", "\n", " SHAPE Shape__Area \\\n", "0 {\"rings\": [[[787723.729257345, 2965812.6409820... 7.562361e+06 \n", "1 {\"rings\": [[[782820.000068769, 2960295.0999848... 2.045208e+06 \n", "2 {\"rings\": [[[783942.499781817, 2960316.2000169... 4.427769e+06 \n", "3 {\"rings\": [[[775776.888126209, 2961852.5792183... 1.546242e+06 \n", "4 {\"rings\": [[[774156.5998687, 2964357.69992466]... 1.603861e+06 \n", "\n", " Shape__Length WARD_PRECINCT \n", "0 13647.035344 0113 \n", "1 5709.044851 0102 \n", "2 9601.063623 0105 \n", "3 5352.433202 0203 \n", "4 7036.361667 0205 " ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a spatailly enabled data frame (sedf) by connecting to the \n", "# URL for the precincts feature service\n", "prec_item = agol_gis.content.get('2bc185ec9b0b478d8c2f7e720d116863')\n", "precincts_flayer = prec_item.layers[0]\n", "precincts_sedf = pd.DataFrame.spatial.from_layer(precincts_flayer)\n", "\n", "# plot the first few rows\n", "precincts_sedf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check the spatial reference of the precincts feature server. Below are some tips on how to do this:\n", "\n", " 1. [Introduction to Spatial Reference](https://developers.arcgis.com/documentation/core-concepts/spatial-references/)\n", " 2. To plot a sedf, we need to make it have the same spatial refernece as the map. Otherwise, it won't be shown on the map.\n", " 3. ArcGIS Online uses the WGS_1984_Web_Mercator_Auxiliary_Sphere (WKID 3857) projection. However, the precincts web layer uses 102100 (3857). They are the same thing." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'wkid': 102686}\n", "{'wkid': 102100}\n" ] } ], "source": [ "print(precincts_sedf.spatial.sr)\n", "# WKID 102686 is for the NAD 1983 StatePlane Massachusetts Mainland FIPS 2001 Feet projection\n", "# WKID Lookup: http://resources.esri.com/help/9.3/arcgisserver/apis/rest/pcs.html\n", "\n", "# Reproject to web mercator to match our web map\n", "precincts_sedf.spatial.project(spatial_reference = {'wkid': 3857})\n", "# Check the spatial reference of the precincts feature server again\n", "print(precincts_sedf.spatial.sr)" ] }, { "cell_type": "code", "execution_count": 133, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "856973ca7e8f4a8ca2c47803a60b4521", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(layout=Layout(height='400px', width='100%'), ready=True, zoom=9.0)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Create a map to take a look at the precincts feature service\n", "precincts_map = gis.map(\"Boston\")\n", "precincts_map" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Plot on map\n", "precincts_sedf.spatial.plot(precincts_map, \n", " symbol_type = \"simple\",\n", " cmap = [90,180,172,225],\n", " outline_color = [225,225,225,255],\n", " line_width = 0.5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To know more about Visualizing Spatial Data through sedf, see https://developers.arcgis.com/python/guide/visualizing-data-with-the-spatially-enabled-dataframe/" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDPRECINCTPlan_DistSHAPEShape__AreaShape__LengthWARD_PRECINCT
0113None{\"rings\": [[[-7905136.25975774, 5218873.046087...7.562361e+0613647.0353440113
1202None{\"rings\": [[[-7907169.264020915, 5216602.89280...2.045208e+065709.0448510102
2305None{\"rings\": [[[-7906706.816058001, 5216609.05518...4.427769e+069601.0636230105
3403None{\"rings\": [[[-7910067.262196271, 5217262.41730...1.546242e+065352.4332020203
4505None{\"rings\": [[[-7910729.483535311, 5218301.74986...1.603861e+067036.3616670205
\n", "
" ], "text/plain": [ " OBJECTID PRECINCT Plan_Dist \\\n", "0 1 13 None \n", "1 2 02 None \n", "2 3 05 None \n", "3 4 03 None \n", "4 5 05 None \n", "\n", " SHAPE Shape__Area \\\n", "0 {\"rings\": [[[-7905136.25975774, 5218873.046087... 7.562361e+06 \n", "1 {\"rings\": [[[-7907169.264020915, 5216602.89280... 2.045208e+06 \n", "2 {\"rings\": [[[-7906706.816058001, 5216609.05518... 4.427769e+06 \n", "3 {\"rings\": [[[-7910067.262196271, 5217262.41730... 1.546242e+06 \n", "4 {\"rings\": [[[-7910729.483535311, 5218301.74986... 1.603861e+06 \n", "\n", " Shape__Length WARD_PRECINCT \n", "0 13647.035344 0113 \n", "1 5709.044851 0102 \n", "2 9601.063623 0105 \n", "3 5352.433202 0203 \n", "4 7036.361667 0205 " ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Take a look at the data of the precincts in a tabular format\n", "precincts_sedf.head()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OBJECTID int64\n", "PRECINCT object\n", "Plan_Dist object\n", "SHAPE geometry\n", "Shape__Area float64\n", "Shape__Length float64\n", "WARD_PRECINCT object\n", "dtype: object" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# use the `dtypes` property to find the data type of these columns\n", "precincts_sedf.dtypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will need a unique ID to identify each of the precincts in our analysis. Below we use Pandas `DataFrame`'s query capabilities to understand how many unique precincts exist in our study area." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The length of the data frame is 255.\n", "There are 24 unique precincts.\n", "There are 255 unique ward precincts.\n" ] } ], "source": [ "print(\"The length of the data frame is {}.\".format(len(precincts_sedf)))\n", "print(\"There are {} unique precincts.\".format(len(precincts_sedf['PRECINCT'].unique())))\n", "print(\"There are {} unique ward precincts.\".format(len(precincts_sedf['WARD_PRECINCT'].unique())))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are the same number of unique values in the WARD_PRECINCT column as there are features, and we will use this field for aggregation and reporting.\n", "\n", "The WARD_PRECINCT data came in 0 padded (i.e., 0113) it is an object in our dataframe instead of an integer. We will leave the WARD_PRECINCT column as strings to avoid making changes to the input data. Since some of the tools we will use require a unique integer ID field a new field will be added. This field and the WARD_PRECINCT values will also be used create a lookup table to link the ID Field to the string WARD_PRECINCT field." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDWARD_PRECINCT
010113
120102
230105
340203
450205
560204
670206
780303
890304
9100307
10110402
\n", "
" ], "text/plain": [ " OBJECTID WARD_PRECINCT\n", "0 1 0113\n", "1 2 0102\n", "2 3 0105\n", "3 4 0203\n", "4 5 0205\n", "5 6 0204\n", "6 7 0206\n", "7 8 0303\n", "8 9 0304\n", "9 10 0307\n", "10 11 0402" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "precincts_sedf.loc[:10,(\"OBJECTID\", \"WARD_PRECINCT\")]" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "# Add unique ID column which will be used for analysis\n", "precincts_sedf['PrecinctWardID'] = pd.to_numeric(precincts_sedf['OBJECTID'])" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PRECINCTWARD_INDEXPRECINCTWARD_ID
010113
120102
230105
340203
450205
560204
670206
780303
890304
9100307
\n", "
" ], "text/plain": [ " PRECINCTWARD_INDEX PRECINCTWARD_ID\n", "0 1 0113\n", "1 2 0102\n", "2 3 0105\n", "3 4 0203\n", "4 5 0205\n", "5 6 0204\n", "6 7 0206\n", "7 8 0303\n", "8 9 0304\n", "9 10 0307" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create lookup table for PrecinctWardID to WARD_PRECINCT\n", "\n", "precinct_lookup_table = {}\n", "precinct_lookup_table_index = []\n", "precinct_lookup_table_id = []\n", "\n", "# Loop through each precinct_ward and generate the lookup table\n", "for idx, cur_precinct in enumerate(list(precincts_sedf['WARD_PRECINCT'])):\n", " precinct_lookup_table[idx] = cur_precinct\n", " precinct_lookup_table_index.append(idx)\n", " precinct_lookup_table_id.append(cur_precinct)\n", "\n", "# Use the lookuptable to create a lookup data frame for pandas dataframe merge\n", "precinct_lookup_df = pd.DataFrame({'PRECINCTWARD_INDEX':precinct_lookup_table_index,\n", " 'PRECINCTWARD_ID': precinct_lookup_table_id})\n", "\n", "# Since we will be using OBJECTID as the join and it is 1 index, we increment our 0 index \n", "precinct_lookup_df['PRECINCTWARD_INDEX'] = precinct_lookup_df['PRECINCTWARD_INDEX'] + 1\n", "precinct_lookup_df.head(10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now the `precinct_lookup_df` can link between the string `PRECINCTWARD_ID` and our `PRECINCTWARD_INDEX` fields." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Connect to the Crime Incident Reports Dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now use the [Crime Incident Reports](https://data.boston.gov/dataset/crime-incident-reports-august-2015-to-date-source-new-system) API to access the crime incident data. We start by reviewing the API website. There are links to the Crime Incident Reports as well as detailed files which explain the incident types. Going to the Crime Incident Reports (csv) link gives more details about accessing the API, the data that is available and the columns in the dataset. \n", "\n", "The resource URL provides us with the specific access key for the Crime Incident Reports API. Since Analyze Boston hosts many datasets we need to use this to access the data we are interested in. The API reference URL provided is:\n", "\n", "https://data.boston.gov/dataset/6220d948-eae2-4e4b-8723-2dc8e67722a3/resource/12cb3883-56f5-47de-afa5-3b1cf61b257b/download/crime.csv\n", "\n", "From this URL we can identify the resource ID as the portion after /resoruce/, i.e., **12cb3883-56f5-47de-afa5-3b1cf61b257b**.\n", "\n", "Clicking on the DATA API Button loads several examples of querying the Crime Incident Reports API. The examples can be used as a starting place to build a query for the data you are interested in.\n", "\n", "We need to connect to the data API through a http request and will use the python library [urllib](https://docs.python.org/3/library/urllib.html). If you are new to urllib this video [video](https://www.youtube.com/watch?v=LosIGgon_KM) has a quick overview.\n", "\n", "#### Key Steps:\n", "\n", "1) Build a query to get the data we are interested in\n", "\n", "2) Use urllib to encode the query string. \n", " * One reason we do this is that you cannot pass spaces and other special characters to a url, by encoding it we have a string with url friendly parameters.\n", "\n", "3) Use urllib to send the request to the server and return the response. \n", " * Urllib has all the functionality to confirm if a request was successful or not and handle the passing of the data.\n", " \n", "4) Receive the response from the server and parse the response string as a json to begin processing." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make a request to the API" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "# Import the required libraries from urllib\n", "from urllib import request\n", "from urllib import parse" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "# The url of the Analzye Boston datastore API\n", "url = \"https://data.boston.gov/api/3/action/datastore_search_sql\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now build the query for the API. If you are not familiar with SQL you can move on without worrying about the nitty gritty of the query.\n", "\n", "The Crime Incident Reports can be accessed using the identifier `12cb3883-56f5-47de-afa5-3b1cf61b257b`. To select crimes of the assault type we are using the column `OFFENSE_CODE_GROUP` (you can see the columns from the [API URL](https://data.boston.gov/dataset/crime-incident-reports-august-2015-to-date-source-new-system)). We are going to query this column for the values `Aggravated Assault` or `Simple Assault`. Using a `SQL` `AND` statement we also add in a filter for the year `2017`. You can change the query values to analyze any crime types over any period that you are interested in. This is one of the great features of connecting to a live data API, we can change our analysis without having to have downloaded all of the data at once.\n", "\n", "We create a dictionary with a key `sql`, this key is used by the API to accept SQL query paremeters. The query string is then encoded for sending in a http request." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This is the query string, spaces and special characters have been replaced\n", "\n", "sql=SELECT+%2A+from+%2212cb3883-56f5-47de-afa5-3b1cf61b257b%22+WHERE+%0A++++%28%22OFFENSE_CODE_GROUP%22+%3D+%27Aggravated+Assault%27+OR+%22OFFENSE_CODE_GROUP%22+%3D+%27Simple+Assault%27%29+%0A++++AND+%22YEAR%22%3D%272017%27\n" ] } ], "source": [ "# Build SQL query string\n", "params = {\n", " 'sql': \"\"\"SELECT * from \"12cb3883-56f5-47de-afa5-3b1cf61b257b\" WHERE \n", " (\"OFFENSE_CODE_GROUP\" = 'Aggravated Assault' OR \"OFFENSE_CODE_GROUP\" = 'Simple Assault') \n", " AND \"YEAR\"='2017'\"\"\"\n", "}\n", "query_string = parse.urlencode(params)\n", "\n", "print(\"This is the query string, spaces and special characters have been replaced\\n\\n{}\".format(query_string))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "https://data.boston.gov/api/3/action/datastore_search_sql?sql=SELECT+%2A+from+%2212cb3883-56f5-47de-afa5-3b1cf61b257b%22+WHERE+%0A++++%28%22OFFENSE_CODE_GROUP%22+%3D+%27Aggravated+Assault%27+OR+%22OFFENSE_CODE_GROUP%22+%3D+%27Simple+Assault%27%29+%0A++++AND+%22YEAR%22%3D%272017%27\n" ] } ], "source": [ "# Concatinate the url and the query string with a question mark\n", "full_url = url + \"?\" + query_string\n", "print(full_url)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we send the request to the server. If we get a response code of **200** the request was a success. If it did not work try again as the server sometimes errors when responding. If it does not work after several tries confirm that the server is loaded and that the Crime Incident Reports datasource item id has not changed\n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Response Code 200 -- it was a success\n" ] } ], "source": [ "# Request data from server\n", "try:\n", " resp = request.urlopen(full_url)\n", " if resp.code==200:\n", " print(\"Response Code {} -- it was a success\".format(resp.code))\n", " else:\n", " print(\"Response Code {} -- something went wrong, try the request again\".format(resp.code))\n", "except Exception as e:\n", " print(str(e))" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "b'{\"help\": \"https://data.boston.gov/api/3/action/help_show?name=datastore_search_sql\", \"success\": true, \"result\": {\"records\": [{\"STREET\": \"RIVER ST\", \"OFFENSE_DESCRIPTION\": \"ASSAULT - AGGRAVATED - BATTERY\", \"SHOOTING\": null, \"DISTRICT\": \"B3\", \"OFFENSE_CODE\": \"00413\", \"REPORTING_AREA\": \"426\", \"_full_text\": \"\\'-07\\':12 \\'-09\\':11 \\'-71.09301057\\':25,27 \\'00\\':14,15 \\'00413\\':2 \\'15\\':13,19 \\'2017\\':10,16 \\'42.268024\\':24 \\'42.26802400\\':26 \\'426\\':9 \\'9\\':17 \\'aggravated\\':3,6 \\'assault\\':4,5 \\'b3\\':8 \\'battery\\':7 \\'i182055984\\':'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Use the response.peek() method to see what the returned data looks like\n", "resp.peek()[:500]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Explore the json format data from API" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The response includes the url that was used for the search, a key to record if it was a success and the records from the data" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [], "source": [ "# Use the .read().decode() message to read all of the data from the response into a string\n", "json_string = resp.read().decode()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'{\"help\": \"https://data.boston.gov/api/3/action/help_show?name=datastore_search_sql\", \"success\": true, \"result\": {\"records\": [{\"STREET\": \"RIVER ST\", \"OFFENSE_DESCRIPTION\": \"ASSAULT - AGGRAVATED - BATTERY\", \"SHOOTING\": null, \"DISTRICT\": \"B3\", \"OFFENSE_CODE\": \"00413\", \"REPORTING_AREA\": \"426\", \"_full_text\": \"\\'-07\\':12 \\'-09\\':11 \\'-71.09301057\\':25,27 \\'00\\':14,15 \\'00413\\':2 \\'15\\':13,19 \\'2017\\':10,16 \\'42.268024\\':24 \\'42.26802400\\':26 \\'426\\':9 \\'9\\':17 \\'aggravated\\':3,6 \\'assault\\':4,5 \\'b3\\':8 \\'battery\\':7 \\'i182055984\\':1 \\'one\\':21 \\'part\\':20 \\'river\\':22 \\'st\\':23 \\'thursday\\':18\", \"OCCURRED_ON_DATE\": \"2017-09-07 15:00:00\", \"DAY_OF_WEEK\": \"Thursday\", \"MONTH\": \"9\", \"HOUR\": \"15\", \"Long\": \"-71.09301057\", \"YEAR\": \"2017\", \"Lat\": \"42.268024\", \"INCIDENT_NUMBER\": \"I182055984\", \"_id\": 63361, \"OFFENSE_CODE_GROUP\": \"Aggravated Assault\", \"UCR_PART\": \"Part One\", \"Location\": \"(42.26802400, -71.09301057)\"}, {\"STREET\": \"PRINCE ST\", \"OFFENSE_DESCRIPTION\": \"ASSAULT SIMPLE - BATTERY\", \"SHOOTING\": null, \"DISTRICT\": \"A1\", \"OFFENSE_CODE\": '" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# You can change the length to check more records\n", "json_string[:1000]" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict_keys(['help', 'success', 'result'])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load the json from the response string\n", "json_dict = json.loads(json_string)\n", "\n", "# Look at the keys in the resposne dictionary\n", "json_dict.keys()" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "dict_keys(['records', 'fields', 'sql'])" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at keys for the 'result' dictionary\n", "json_dict['result'].keys()" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'STREET': 'RIVER ST',\n", " 'OFFENSE_DESCRIPTION': 'ASSAULT - AGGRAVATED - BATTERY',\n", " 'SHOOTING': None,\n", " 'DISTRICT': 'B3',\n", " 'OFFENSE_CODE': '00413',\n", " 'REPORTING_AREA': '426',\n", " '_full_text': \"'-07':12 '-09':11 '-71.09301057':25,27 '00':14,15 '00413':2 '15':13,19 '2017':10,16 '42.268024':24 '42.26802400':26 '426':9 '9':17 'aggravated':3,6 'assault':4,5 'b3':8 'battery':7 'i182055984':1 'one':21 'part':20 'river':22 'st':23 'thursday':18\",\n", " 'OCCURRED_ON_DATE': '2017-09-07 15:00:00',\n", " 'DAY_OF_WEEK': 'Thursday',\n", " 'MONTH': '9',\n", " 'HOUR': '15',\n", " 'Long': '-71.09301057',\n", " 'YEAR': '2017',\n", " 'Lat': '42.268024',\n", " 'INCIDENT_NUMBER': 'I182055984',\n", " '_id': 63361,\n", " 'OFFENSE_CODE_GROUP': 'Aggravated Assault',\n", " 'UCR_PART': 'Part One',\n", " 'Location': '(42.26802400, -71.09301057)'}" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at a single result record\n", "json_dict['result']['records'][0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data looks good, we have the columns we expect and can see that our query was successful. Our record is in the offence group code 'ASSAULT - AGGRAVATED - BATTERY' and the year is 2017.\n", "\n", "
Note: Since this is a live dataset that you are querying, your results may differ
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create a SeDF from crime records JSON" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Spatially Enabled DataFrame (SeDF) inserts a custom namespace called spatial into the popular Pandas DataFrame structure to give it spatial abilities. Once you have two fields for latitude and longitude respectively, you can convert a Pandas DataFrame to a SeDF. Besides, you can also create SeDF from many other sources, including GeoJSON, feature classes, shapefiles, feature layers, and feature services, etc.\n", "\n", "SeDF is based on data structures inherently suited to data analysis, with natural operations for the filtering and inspecting of subsets of values which are fundamental to statistical and geographic manipulations. For more information about SeDF, please refer to: https://developers.arcgis.com/python/guide/introduction-to-the-spatially-enabled-dataframe/" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DAY_OF_WEEKDISTRICTHOURINCIDENT_NUMBERLatLocationLongMONTHOCCURRED_ON_DATEOFFENSE_CODEOFFENSE_CODE_GROUPOFFENSE_DESCRIPTIONREPORTING_AREASHOOTINGSTREETUCR_PARTYEAR_full_text_id
0ThursdayB315I18205598442.268024(42.26802400, -71.09301057)-71.0930105792017-09-07 15:00:0000413Aggravated AssaultASSAULT - AGGRAVATED - BATTERY426NoneRIVER STPart One2017'-07':12 '-09':11 '-71.09301057':25,27 '00':14...63361
1SundayA12I18204875442.36475392(42.36475392, -71.05469547)-71.0546954752017-05-21 02:00:0000802Simple AssaultASSAULT SIMPLE - BATTERY84NonePRINCE STPart Two2017'-05':11 '-21':12 '-71.05469547':25,27 '00':14...70308
2FridayC612I18204433642.33326722(42.33326722, -71.05048119)-71.0504811992017-09-01 12:00:0000802Simple AssaultASSAULT SIMPLE - BATTERY216NoneDORCHESTER STPart Two2017'-01':12 '-09':11 '-71.05048119':25,27 '00':14...74505
3FridayNone9I182034981-1(-1.00000000, -1.00000000)-192017-09-01 09:00:0000802Simple AssaultASSAULT SIMPLE - BATTERYNoneGOODALEPart Two2017'-01':10 '-09':9 '-1':21,22 '-1.00000000':23,2...83381
4ThursdayD423I18203153542.34717363(42.34717363, -71.09644390)-71.0964439122017-12-21 23:30:0000413Aggravated AssaultASSAULT - AGGRAVATED - BATTERY624NoneLANSDOWNE STPart One2017'-12':11 '-21':12 '-71.0964439':25 '-71.096443...86636
\n", "
" ], "text/plain": [ " DAY_OF_WEEK DISTRICT HOUR INCIDENT_NUMBER Lat \\\n", "0 Thursday B3 15 I182055984 42.268024 \n", "1 Sunday A1 2 I182048754 42.36475392 \n", "2 Friday C6 12 I182044336 42.33326722 \n", "3 Friday None 9 I182034981 -1 \n", "4 Thursday D4 23 I182031535 42.34717363 \n", "\n", " Location Long MONTH OCCURRED_ON_DATE \\\n", "0 (42.26802400, -71.09301057) -71.09301057 9 2017-09-07 15:00:00 \n", "1 (42.36475392, -71.05469547) -71.05469547 5 2017-05-21 02:00:00 \n", "2 (42.33326722, -71.05048119) -71.05048119 9 2017-09-01 12:00:00 \n", "3 (-1.00000000, -1.00000000) -1 9 2017-09-01 09:00:00 \n", "4 (42.34717363, -71.09644390) -71.0964439 12 2017-12-21 23:30:00 \n", "\n", " OFFENSE_CODE OFFENSE_CODE_GROUP OFFENSE_DESCRIPTION \\\n", "0 00413 Aggravated Assault ASSAULT - AGGRAVATED - BATTERY \n", "1 00802 Simple Assault ASSAULT SIMPLE - BATTERY \n", "2 00802 Simple Assault ASSAULT SIMPLE - BATTERY \n", "3 00802 Simple Assault ASSAULT SIMPLE - BATTERY \n", "4 00413 Aggravated Assault ASSAULT - AGGRAVATED - BATTERY \n", "\n", " REPORTING_AREA SHOOTING STREET UCR_PART YEAR \\\n", "0 426 None RIVER ST Part One 2017 \n", "1 84 None PRINCE ST Part Two 2017 \n", "2 216 None DORCHESTER ST Part Two 2017 \n", "3 None GOODALE Part Two 2017 \n", "4 624 None LANSDOWNE ST Part One 2017 \n", "\n", " _full_text _id \n", "0 '-07':12 '-09':11 '-71.09301057':25,27 '00':14... 63361 \n", "1 '-05':11 '-21':12 '-71.05469547':25,27 '00':14... 70308 \n", "2 '-01':12 '-09':11 '-71.05048119':25,27 '00':14... 74505 \n", "3 '-01':10 '-09':9 '-1':21,22 '-1.00000000':23,2... 83381 \n", "4 '-12':11 '-21':12 '-71.0964439':25 '-71.096443... 86636 " ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Now that we can access the records from the API response we can load them into a Pandas DataFrame\n", "crime_df = pd.DataFrame(json_dict['result']['records'])\n", "crime_df.head()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "There are 7351 records in the dataset originally from the API.\n" ] } ], "source": [ "original_number_of_records = len(crime_df)\n", "print(\"There are {} records in the dataset originally from the API.\".format(original_number_of_records))" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DAY_OF_WEEK object\n", "DISTRICT object\n", "HOUR object\n", "INCIDENT_NUMBER object\n", "Lat object\n", "Location object\n", "Long object\n", "MONTH object\n", "OCCURRED_ON_DATE object\n", "OFFENSE_CODE object\n", "OFFENSE_CODE_GROUP object\n", "OFFENSE_DESCRIPTION object\n", "REPORTING_AREA object\n", "SHOOTING object\n", "STREET object\n", "UCR_PART object\n", "YEAR object\n", "_full_text object\n", "_id int64\n", "dtype: object" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at the column types for each of the fields in our dataframe\n", "crime_df.dtypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The columns and data have been loaded into a dataframe and match the columns we saw on the Crime Incident Reports website.\n", "We can look at the unique values in the OFFENSE_CODE_GROUP and YEAR columns to confirm our query worked as expected." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The unique OFFENSE_CODE_GROUP values: \n", "['Aggravated Assault' 'Simple Assault']\n", "\n", "The unique YEAR values: \n", "['2017']\n" ] } ], "source": [ "print(\"The unique OFFENSE_CODE_GROUP values: \\n{}\\n\".format(crime_df['OFFENSE_CODE_GROUP'].unique()))\n", "print(\"The unique YEAR values: \\n{}\".format(crime_df['YEAR'].unique()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The API returned the data we expected." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Clean the data\n", "\n", "We will now take a look at the data and do some data cleaning: change data format, add timestamp field, and remove data with no or invalid location information.\n", "\n", "We want to use the coordinates from the Lat and Long columns to create points. If there are crime incidents without location coordinates we will drop them. We will assess how many records we lose due to a lack of coordinates. Depending on the analysis requirements, dropping crime incidents without coordinates may not be acceptable." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DAY_OF_WEEK object\n", "DISTRICT object\n", "HOUR int64\n", "INCIDENT_NUMBER object\n", "Lat float64\n", "Location object\n", "Long float64\n", "MONTH int64\n", "OCCURRED_ON_DATE object\n", "OFFENSE_CODE object\n", "OFFENSE_CODE_GROUP object\n", "OFFENSE_DESCRIPTION object\n", "REPORTING_AREA object\n", "SHOOTING object\n", "STREET object\n", "UCR_PART object\n", "YEAR int64\n", "_full_text object\n", "_id int64\n", "dtype: object" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Most fields are objects in the dataframe, we will convert the following fields to numeric values:\n", "# Lat and Long need to be numeric for creating SeDF\n", "# HOUR, MONTH, YEAR are for analysis purpose\n", "numeric_fields = ['Lat', 'Long', 'HOUR', 'MONTH', 'YEAR']\n", "for field in numeric_fields:\n", " crime_df[field] = pd.to_numeric(crime_df[field])\n", "\n", "crime_df.dtypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `OCCURRED_ON_DATE` field is also an object whereas we require Timestamp object to work with temporal events. We can use python libraries to create datetime objects for this column. Lets start by taking a look at the format." ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
HOURMONTHYEAROCCURRED_ON_DATE
015920172017-09-07 15:00:00
12520172017-05-21 02:00:00
212920172017-09-01 12:00:00
39920172017-09-01 09:00:00
4231220172017-12-21 23:30:00
5231220172017-12-21 23:30:00
6141220172017-12-14 14:30:00
721020172017-10-09 02:40:00
8191220172017-12-15 19:30:00
9201020172017-10-21 20:30:00
1041020172017-10-05 04:00:00
\n", "
" ], "text/plain": [ " HOUR MONTH YEAR OCCURRED_ON_DATE\n", "0 15 9 2017 2017-09-07 15:00:00\n", "1 2 5 2017 2017-05-21 02:00:00\n", "2 12 9 2017 2017-09-01 12:00:00\n", "3 9 9 2017 2017-09-01 09:00:00\n", "4 23 12 2017 2017-12-21 23:30:00\n", "5 23 12 2017 2017-12-21 23:30:00\n", "6 14 12 2017 2017-12-14 14:30:00\n", "7 2 10 2017 2017-10-09 02:40:00\n", "8 19 12 2017 2017-12-15 19:30:00\n", "9 20 10 2017 2017-10-21 20:30:00\n", "10 4 10 2017 2017-10-05 04:00:00" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at the temporal information in the dataframe\n", "crime_df.loc[:10, ('HOUR', 'MONTH', 'YEAR', 'OCCURRED_ON_DATE')]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Create timestamp field\n", "The dates from the API are in format `2017-03-08T21:34:00` (YYYY-MM-DDTHH:MM:SS). \n", "\n", "For spatial-temporal analysis with arcpy, we need to process this format of date into a `Timestamp` object as a new field in the DataFrame." ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "import datetime as dt\n", "\n", "def process_date(input_date):\n", " '''Function to process date in YYYY-MM-DDTHH:MM:SS format'''\n", " try:\n", " mydate = dt.datetime.strptime(input_date, '%Y-%m-%dT%H:%M:%S')\n", " except ValueError: \n", " mydate = dt.datetime.strptime(input_date, '%Y-%m-%d %H:%M:%S')\n", " except:\n", " mydate = dt.datetime.strptime('1700', '%Y')\n", " return mydate" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OCCURRED_ON_DATEDateTime
02017-09-07 15:00:002017-09-07 15:00:00
12017-05-21 02:00:002017-05-21 02:00:00
22017-09-01 12:00:002017-09-01 12:00:00
32017-09-01 09:00:002017-09-01 09:00:00
42017-12-21 23:30:002017-12-21 23:30:00
52017-12-21 23:30:002017-12-21 23:30:00
62017-12-14 14:30:002017-12-14 14:30:00
72017-10-09 02:40:002017-10-09 02:40:00
82017-12-15 19:30:002017-12-15 19:30:00
92017-10-21 20:30:002017-10-21 20:30:00
102017-10-05 04:00:002017-10-05 04:00:00
\n", "
" ], "text/plain": [ " OCCURRED_ON_DATE DateTime\n", "0 2017-09-07 15:00:00 2017-09-07 15:00:00\n", "1 2017-05-21 02:00:00 2017-05-21 02:00:00\n", "2 2017-09-01 12:00:00 2017-09-01 12:00:00\n", "3 2017-09-01 09:00:00 2017-09-01 09:00:00\n", "4 2017-12-21 23:30:00 2017-12-21 23:30:00\n", "5 2017-12-21 23:30:00 2017-12-21 23:30:00\n", "6 2017-12-14 14:30:00 2017-12-14 14:30:00\n", "7 2017-10-09 02:40:00 2017-10-09 02:40:00\n", "8 2017-12-15 19:30:00 2017-12-15 19:30:00\n", "9 2017-10-21 20:30:00 2017-10-21 20:30:00\n", "10 2017-10-05 04:00:00 2017-10-05 04:00:00" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Calculate the new time column\n", "crime_df['DateTime'] = crime_df.apply((lambda x: process_date(x['OCCURRED_ON_DATE'])), axis=1)\n", "crime_df.loc[:10, ('OCCURRED_ON_DATE', 'DateTime')]" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n" ] }, { "data": { "text/plain": [ "DAY_OF_WEEK object\n", "DISTRICT object\n", "HOUR int64\n", "INCIDENT_NUMBER object\n", "Lat float64\n", "Location object\n", "Long float64\n", "MONTH int64\n", "OCCURRED_ON_DATE object\n", "OFFENSE_CODE object\n", "OFFENSE_CODE_GROUP object\n", "OFFENSE_DESCRIPTION object\n", "REPORTING_AREA object\n", "SHOOTING object\n", "STREET object\n", "UCR_PART object\n", "YEAR int64\n", "_full_text object\n", "_id int64\n", "DateTime datetime64[ns]\n", "dtype: object" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Confirm that the new field is the correct Timestamp object we expect\n", "print(type(crime_df.iloc[0]['DateTime']))\n", "crime_df.dtypes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Add an OBJECTID field" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DAY_OF_WEEKDISTRICTHOURINCIDENT_NUMBERLatLocationLongMONTHOCCURRED_ON_DATEOFFENSE_CODE...OFFENSE_DESCRIPTIONREPORTING_AREASHOOTINGSTREETUCR_PARTYEAR_full_text_idDateTimeOBJECTID
7348WednesdayB221I152019928-0042.342528(42.34252789, -71.07678918)-71.07678932017-03-08 21:34:000423...ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER153NonePEMBROKE STPart One2017'-00':2 '-03':16 '-08':17 '-71.07678918':30,32...3694402017-03-08 21:34:007349
7349WednesdayB221I152019928-0042.342528(42.34252789, -71.07678918)-71.07678932017-03-08 21:34:000803...A&B ON POLICE OFFICER153NonePEMBROKE STPart Two2017'-00':2 '-03':14 '-08':15 '-71.07678918':28,30...3694412017-03-08 21:34:007350
7350WednesdayB221I152019928-0042.342528(42.34252789, -71.07678918)-71.07678932017-03-08 21:34:000803...A&B ON POLICE OFFICER153NonePEMBROKE STPart Two2017'-00':2 '-03':14 '-08':15 '-71.07678918':28,30...3694422017-03-08 21:34:007351
\n", "

3 rows × 21 columns

\n", "
" ], "text/plain": [ " DAY_OF_WEEK DISTRICT HOUR INCIDENT_NUMBER Lat \\\n", "7348 Wednesday B2 21 I152019928-00 42.342528 \n", "7349 Wednesday B2 21 I152019928-00 42.342528 \n", "7350 Wednesday B2 21 I152019928-00 42.342528 \n", "\n", " Location Long MONTH OCCURRED_ON_DATE \\\n", "7348 (42.34252789, -71.07678918) -71.076789 3 2017-03-08 21:34:00 \n", "7349 (42.34252789, -71.07678918) -71.076789 3 2017-03-08 21:34:00 \n", "7350 (42.34252789, -71.07678918) -71.076789 3 2017-03-08 21:34:00 \n", "\n", " OFFENSE_CODE ... OFFENSE_DESCRIPTION \\\n", "7348 0423 ... ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER \n", "7349 0803 ... A&B ON POLICE OFFICER \n", "7350 0803 ... A&B ON POLICE OFFICER \n", "\n", " REPORTING_AREA SHOOTING STREET UCR_PART YEAR \\\n", "7348 153 None PEMBROKE ST Part One 2017 \n", "7349 153 None PEMBROKE ST Part Two 2017 \n", "7350 153 None PEMBROKE ST Part Two 2017 \n", "\n", " _full_text _id \\\n", "7348 '-00':2 '-03':16 '-08':17 '-71.07678918':30,32... 369440 \n", "7349 '-00':2 '-03':14 '-08':15 '-71.07678918':28,30... 369441 \n", "7350 '-00':2 '-03':14 '-08':15 '-71.07678918':28,30... 369442 \n", "\n", " DateTime OBJECTID \n", "7348 2017-03-08 21:34:00 7349 \n", "7349 2017-03-08 21:34:00 7350 \n", "7350 2017-03-08 21:34:00 7351 \n", "\n", "[3 rows x 21 columns]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "crime_df['OBJECTID'] = list(range(1, len(crime_df) + 1))\n", "crime_df.tail(3)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Examine the coordinates\n", "\n", "If any of the coordinates are missing we will not be able to create a point for the incident. We will use the dataframe to interrogate the coordinates column and look for missing or odd values." ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDLatLong
0142.268024-71.093011
1242.364754-71.054695
2342.333267-71.050481
34-1.000000-1.000000
4542.347174-71.096444
5642.347174-71.096444
6742.330398-71.051270
7842.243807-71.143743
8942.338027-71.054929
91042.366435-71.061354
1011NaNNaN
111242.278298-71.138591
121342.321675-71.091954
1314NaNNaN
1415NaNNaN
\n", "
" ], "text/plain": [ " OBJECTID Lat Long\n", "0 1 42.268024 -71.093011\n", "1 2 42.364754 -71.054695\n", "2 3 42.333267 -71.050481\n", "3 4 -1.000000 -1.000000\n", "4 5 42.347174 -71.096444\n", "5 6 42.347174 -71.096444\n", "6 7 42.330398 -71.051270\n", "7 8 42.243807 -71.143743\n", "8 9 42.338027 -71.054929\n", "9 10 42.366435 -71.061354\n", "10 11 NaN NaN\n", "11 12 42.278298 -71.138591\n", "12 13 42.321675 -71.091954\n", "13 14 NaN NaN\n", "14 15 NaN NaN" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Check if there are any invalid Lat and Long that can't be plot on to map\n", "crime_df.loc[:, ('OBJECTID', 'Lat', 'Long')].head(15)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "There are NaNs, and for Boston a (long,lat) of (-1.0, -1.0) is an invalid coordinate. We will need to filter this out of the data set we use to create points." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now use a query on the data frame to remove null and -1 values. A property of NaN values is that they do not return true when compared to themselves, so we can use that in our query. We also filter out -1 values. " ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDLatLong
0142.268024-71.093011
1242.364754-71.054695
2342.333267-71.050481
4542.347174-71.096444
5642.347174-71.096444
6742.330398-71.051270
7842.243807-71.143743
8942.338027-71.054929
91042.366435-71.061354
111242.278298-71.138591
121342.321675-71.091954
161742.333112-71.072764
171842.336063-71.107828
181942.366054-71.033262
192042.332547-71.072124
\n", "
" ], "text/plain": [ " OBJECTID Lat Long\n", "0 1 42.268024 -71.093011\n", "1 2 42.364754 -71.054695\n", "2 3 42.333267 -71.050481\n", "4 5 42.347174 -71.096444\n", "5 6 42.347174 -71.096444\n", "6 7 42.330398 -71.051270\n", "7 8 42.243807 -71.143743\n", "8 9 42.338027 -71.054929\n", "9 10 42.366435 -71.061354\n", "11 12 42.278298 -71.138591\n", "12 13 42.321675 -71.091954\n", "16 17 42.333112 -71.072764\n", "17 18 42.336063 -71.107828\n", "18 19 42.366054 -71.033262\n", "19 20 42.332547 -71.072124" ] }, "execution_count": 40, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Remove locations with no coordinates\n", "crime_clean_df = crime_df.query('(Long == Long) and (Long != -1)').copy()\n", "# Print out the first 15 records to check if invalid entries have been removed correctly\n", "crime_clean_df.loc[:, ('OBJECTID', 'Lat', 'Long')].head(15)" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [], "source": [ "cleaned_number_of_records = len(crime_clean_df)" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "From the API we recieved 7351 records. After cleaning 6849 records remained. This works out to 3.54% of data dropped during cleaning.\n" ] } ], "source": [ "percent_records_dropped = (original_number_of_records - cleaned_number_of_records)/(cleaned_number_of_records+original_number_of_records) * 100\n", "print(\"From the API we recieved {} records. After cleaning {} \"\\\n", " \"records remained. This works out to {:.2f}% of data dropped during cleaning.\".format(original_number_of_records, cleaned_number_of_records,\n", " percent_records_dropped))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create spatial version of crime data\n", "\n", "The longitude and latitude (x, y) coordinates in the data can be used to create points for the location of each crime incident. The spatially enabled dataframe has a method [from_xy](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.features.toc.html?highlight=from_xy#arcgis.features.SpatialDataFrame.from_xy) to create points from a dataframe." ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DAY_OF_WEEKDISTRICTHOURINCIDENT_NUMBERLatLocationLongMONTHOCCURRED_ON_DATEOFFENSE_CODE...REPORTING_AREASHOOTINGSTREETUCR_PARTYEAR_full_text_idDateTimeOBJECTIDSHAPE
0ThursdayB315I18205598442.268024(42.26802400, -71.09301057)-71.09301192017-09-07 15:00:0000413...426NoneRIVER STPart One2017'-07':12 '-09':11 '-71.09301057':25,27 '00':14...633612017-09-07 15:00:001{\"x\": -71.09301057, \"y\": 42.268024, \"spatialRe...
1SundayA12I18204875442.364754(42.36475392, -71.05469547)-71.05469552017-05-21 02:00:0000802...84NonePRINCE STPart Two2017'-05':11 '-21':12 '-71.05469547':25,27 '00':14...703082017-05-21 02:00:002{\"x\": -71.05469547, \"y\": 42.36475392, \"spatial...
2FridayC612I18204433642.333267(42.33326722, -71.05048119)-71.05048192017-09-01 12:00:0000802...216NoneDORCHESTER STPart Two2017'-01':12 '-09':11 '-71.05048119':25,27 '00':14...745052017-09-01 12:00:003{\"x\": -71.05048119, \"y\": 42.33326722, \"spatial...
4ThursdayD423I18203153542.347174(42.34717363, -71.09644390)-71.096444122017-12-21 23:30:0000413...624NoneLANSDOWNE STPart One2017'-12':11 '-21':12 '-71.0964439':25 '-71.096443...866362017-12-21 23:30:005{\"x\": -71.0964439, \"y\": 42.34717363, \"spatialR...
5ThursdayD423I18203153542.347174(42.34717363, -71.09644390)-71.096444122017-12-21 23:30:0000802...624NoneLANSDOWNE STPart Two2017'-12':11 '-21':12 '-71.0964439':25 '-71.096443...866372017-12-21 23:30:006{\"x\": -71.0964439, \"y\": 42.34717363, \"spatialR...
\n", "

5 rows × 22 columns

\n", "
" ], "text/plain": [ " DAY_OF_WEEK DISTRICT HOUR INCIDENT_NUMBER Lat \\\n", "0 Thursday B3 15 I182055984 42.268024 \n", "1 Sunday A1 2 I182048754 42.364754 \n", "2 Friday C6 12 I182044336 42.333267 \n", "4 Thursday D4 23 I182031535 42.347174 \n", "5 Thursday D4 23 I182031535 42.347174 \n", "\n", " Location Long MONTH OCCURRED_ON_DATE \\\n", "0 (42.26802400, -71.09301057) -71.093011 9 2017-09-07 15:00:00 \n", "1 (42.36475392, -71.05469547) -71.054695 5 2017-05-21 02:00:00 \n", "2 (42.33326722, -71.05048119) -71.050481 9 2017-09-01 12:00:00 \n", "4 (42.34717363, -71.09644390) -71.096444 12 2017-12-21 23:30:00 \n", "5 (42.34717363, -71.09644390) -71.096444 12 2017-12-21 23:30:00 \n", "\n", " OFFENSE_CODE ... \\\n", "0 00413 ... \n", "1 00802 ... \n", "2 00802 ... \n", "4 00413 ... \n", "5 00802 ... \n", "\n", " REPORTING_AREA SHOOTING STREET UCR_PART YEAR \\\n", "0 426 None RIVER ST Part One 2017 \n", "1 84 None PRINCE ST Part Two 2017 \n", "2 216 None DORCHESTER ST Part Two 2017 \n", "4 624 None LANSDOWNE ST Part One 2017 \n", "5 624 None LANSDOWNE ST Part Two 2017 \n", "\n", " _full_text _id \\\n", "0 '-07':12 '-09':11 '-71.09301057':25,27 '00':14... 63361 \n", "1 '-05':11 '-21':12 '-71.05469547':25,27 '00':14... 70308 \n", "2 '-01':12 '-09':11 '-71.05048119':25,27 '00':14... 74505 \n", "4 '-12':11 '-21':12 '-71.0964439':25 '-71.096443... 86636 \n", "5 '-12':11 '-21':12 '-71.0964439':25 '-71.096443... 86637 \n", "\n", " DateTime OBJECTID \\\n", "0 2017-09-07 15:00:00 1 \n", "1 2017-05-21 02:00:00 2 \n", "2 2017-09-01 12:00:00 3 \n", "4 2017-12-21 23:30:00 5 \n", "5 2017-12-21 23:30:00 6 \n", "\n", " SHAPE \n", "0 {\"x\": -71.09301057, \"y\": 42.268024, \"spatialRe... \n", "1 {\"x\": -71.05469547, \"y\": 42.36475392, \"spatial... \n", "2 {\"x\": -71.05048119, \"y\": 42.33326722, \"spatial... \n", "4 {\"x\": -71.0964439, \"y\": 42.34717363, \"spatialR... \n", "5 {\"x\": -71.0964439, \"y\": 42.34717363, \"spatialR... \n", "\n", "[5 rows x 22 columns]" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create spatially enabled data frame with x, y coordinates\n", "crime_sedf = pd.DataFrame.spatial.from_xy(crime_clean_df, 'Long', 'Lat')\n", "# Check the fields in SeDF, you can notice one new field: SHAPE\n", "crime_sedf.head()" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DAY_OF_WEEK object\n", "DISTRICT object\n", "HOUR int64\n", "INCIDENT_NUMBER object\n", "Lat float64\n", "Location object\n", "Long float64\n", "MONTH int64\n", "OCCURRED_ON_DATE object\n", "OFFENSE_CODE object\n", "OFFENSE_CODE_GROUP object\n", "OFFENSE_DESCRIPTION object\n", "REPORTING_AREA object\n", "SHOOTING object\n", "STREET object\n", "UCR_PART object\n", "YEAR int64\n", "_full_text object\n", "_id int64\n", "DateTime datetime64[ns]\n", "OBJECTID int64\n", "SHAPE geometry\n", "dtype: object" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# We can see a new field SHAPE is added as the last column in the dataframe, and it's type is geometry\n", "crime_sedf.dtypes" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'wkid': 4326}" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "crime_sedf.spatial.sr" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`4326` is the most common spatial reference for storing a referencing data across the entire world. It serves as the default for both the PostGIS spatial database and the GeoJSON standard. It is also used by default in most web mapping libraries. Because of its use in GPS, 4326 is generally assumed to be the spatial reference when talking about \"latitude\" or \"longitude\".\n", "\n", "When the dataframe has WKID = 4326, we don't need to convert it to WKID = 3857 before plotting to the map." ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [], "source": [ "# set it to the same as the map we are going to plot, if you have trouble plotting it.\n", "# crime_sedf.spatial.sr = {'latestWkid': 3857, 'wkid': 102100}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can look at the police precincts and crime incident data in a map" ] }, { "cell_type": "code", "execution_count": 135, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "6c4f250ed55946a6b2b026756e504c60", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(jupyter_target='notebook', layout=Layout(height='400px', width='100%'), ready=True, zoom=10.0)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "my_map = gis.map(\"Boston\")\n", "my_map" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Add precint polygons\n", "precincts_sedf.spatial.plot(my_map, \n", " symbol_type = \"simple\",\n", " cmap = [90,180,172,225],\n", " outline_color = [225,225,225,255],\n", " line_width = 0.5)" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Add crime points\n", "# limit number of points to 1000 to speed it up\n", "crime_sedf.copy().loc[:1000].spatial.plot(my_map, \n", " marker_size=3, \n", " cmap = [45,0,75, 225],\n", " line_width = 0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Publish the cleaned dataset into a web layer\n", "Now that you have downloaded and cleaned the crime data, you can publish that into a web layer into your GIS for analysis later." ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [], "source": [ "crime_points_layer = crime_sedf.spatial.to_featurelayer(title='Boston crime cleaned 2017', \n", " tags=['boston','crime','police','2017'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2. Perform an exploratory data analysis of the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this exploratory data analysis we will look into the data and learn more about the story it is telling us. Since crimes occur in time we are particularily interested in temporal patterns. Temporal patterns could be used to adjust patrols and anticipate times with the most incidents. In order to aggregate our data by time easily we create a copy of the crime dataframe and set its index to use the DateTime field." ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "6849" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(crime_sedf)" ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [], "source": [ "# Craete a copy of the cleaned crime sedf\n", "crime_df_time_index = crime_sedf.copy()\n", "\n", "# Switch the index to use the DateTime column so we can aggregate by time periods\n", "crime_df_time_index.index=crime_df_time_index['DateTime']" ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [], "source": [ "# Imports for plotting in Bokeh\n", "\n", "import numpy as np\n", "from bokeh.io import output_notebook, output_file, show\n", "from bokeh.plotting import figure\n", "from bokeh.models import Legend, Range1d\n", "from bokeh.embed import file_html\n", "from bokeh.resources import CDN" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Loading BokehJS ...\n", "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function(root) {\n", " function now() {\n", " return new Date();\n", " }\n", "\n", " var force = true;\n", "\n", " if (typeof (root._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n", " root._bokeh_onload_callbacks = [];\n", " root._bokeh_is_loading = undefined;\n", " }\n", "\n", " var JS_MIME_TYPE = 'application/javascript';\n", " var HTML_MIME_TYPE = 'text/html';\n", " var EXEC_MIME_TYPE = 'application/vnd.bokehjs_exec.v0+json';\n", " var CLASS_NAME = 'output_bokeh rendered_html';\n", "\n", " /**\n", " * Render data to the DOM node\n", " */\n", " function render(props, node) {\n", " var script = document.createElement(\"script\");\n", " node.appendChild(script);\n", " }\n", "\n", " /**\n", " * Handle when an output is cleared or removed\n", " */\n", " function handleClearOutput(event, handle) {\n", " var cell = handle.cell;\n", "\n", " var id = cell.output_area._bokeh_element_id;\n", " var server_id = cell.output_area._bokeh_server_id;\n", " // Clean up Bokeh references\n", " if (id != null && id in Bokeh.index) {\n", " Bokeh.index[id].model.document.clear();\n", " delete Bokeh.index[id];\n", " }\n", "\n", " if (server_id !== undefined) {\n", " // Clean up Bokeh references\n", " var cmd = \"from bokeh.io.state import curstate; print(curstate().uuid_to_server['\" + server_id + \"'].get_sessions()[0].document.roots[0]._id)\";\n", " cell.notebook.kernel.execute(cmd, {\n", " iopub: {\n", " output: function(msg) {\n", " var id = msg.content.text.trim();\n", " if (id in Bokeh.index) {\n", " Bokeh.index[id].model.document.clear();\n", " delete Bokeh.index[id];\n", " }\n", " }\n", " }\n", " });\n", " // Destroy server and session\n", " var cmd = \"import bokeh.io.notebook as ion; ion.destroy_server('\" + server_id + \"')\";\n", " cell.notebook.kernel.execute(cmd);\n", " }\n", " }\n", "\n", " /**\n", " * Handle when a new output is added\n", " */\n", " function handleAddOutput(event, handle) {\n", " var output_area = handle.output_area;\n", " var output = handle.output;\n", "\n", " // limit handleAddOutput to display_data with EXEC_MIME_TYPE content only\n", " if ((output.output_type != \"display_data\") || (!output.data.hasOwnProperty(EXEC_MIME_TYPE))) {\n", " return\n", " }\n", "\n", " var toinsert = output_area.element.find(\".\" + CLASS_NAME.split(' ')[0]);\n", "\n", " if (output.metadata[EXEC_MIME_TYPE][\"id\"] !== undefined) {\n", " toinsert[toinsert.length - 1].firstChild.textContent = output.data[JS_MIME_TYPE];\n", " // store reference to embed id on output_area\n", " output_area._bokeh_element_id = output.metadata[EXEC_MIME_TYPE][\"id\"];\n", " }\n", " if (output.metadata[EXEC_MIME_TYPE][\"server_id\"] !== undefined) {\n", " var bk_div = document.createElement(\"div\");\n", " bk_div.innerHTML = output.data[HTML_MIME_TYPE];\n", " var script_attrs = bk_div.children[0].attributes;\n", " for (var i = 0; i < script_attrs.length; i++) {\n", " toinsert[toinsert.length - 1].firstChild.setAttribute(script_attrs[i].name, script_attrs[i].value);\n", " }\n", " // store reference to server id on output_area\n", " output_area._bokeh_server_id = output.metadata[EXEC_MIME_TYPE][\"server_id\"];\n", " }\n", " }\n", "\n", " function register_renderer(events, OutputArea) {\n", "\n", " function append_mime(data, metadata, element) {\n", " // create a DOM node to render to\n", " var toinsert = this.create_output_subarea(\n", " metadata,\n", " CLASS_NAME,\n", " EXEC_MIME_TYPE\n", " );\n", " this.keyboard_manager.register_events(toinsert);\n", " // Render to node\n", " var props = {data: data, metadata: metadata[EXEC_MIME_TYPE]};\n", " render(props, toinsert[toinsert.length - 1]);\n", " element.append(toinsert);\n", " return toinsert\n", " }\n", "\n", " /* Handle when an output is cleared or removed */\n", " events.on('clear_output.CodeCell', handleClearOutput);\n", " events.on('delete.Cell', handleClearOutput);\n", "\n", " /* Handle when a new output is added */\n", " events.on('output_added.OutputArea', handleAddOutput);\n", "\n", " /**\n", " * Register the mime type and append_mime function with output_area\n", " */\n", " OutputArea.prototype.register_mime_type(EXEC_MIME_TYPE, append_mime, {\n", " /* Is output safe? */\n", " safe: true,\n", " /* Index of renderer in `output_area.display_order` */\n", " index: 0\n", " });\n", " }\n", "\n", " // register the mime type if in Jupyter Notebook environment and previously unregistered\n", " if (root.Jupyter !== undefined) {\n", " var events = require('base/js/events');\n", " var OutputArea = require('notebook/js/outputarea').OutputArea;\n", "\n", " if (OutputArea.prototype.mime_types().indexOf(EXEC_MIME_TYPE) == -1) {\n", " register_renderer(events, OutputArea);\n", " }\n", " }\n", "\n", " \n", " if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n", " root._bokeh_timeout = Date.now() + 5000;\n", " root._bokeh_failed_load = false;\n", " }\n", "\n", " var NB_LOAD_WARNING = {'data': {'text/html':\n", " \"
\\n\"+\n", " \"

\\n\"+\n", " \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n", " \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n", " \"

\\n\"+\n", " \"
    \\n\"+\n", " \"
  • re-rerun `output_notebook()` to attempt to load from CDN again, or
    • \\n\"+\n", " \"
  • use INLINE resources instead, as so:
    • \\n\"+\n", " \"
\\n\"+\n", " \"\\n\"+\n", " \"from bokeh.resources import INLINE\\n\"+\n", " \"output_notebook(resources=INLINE)\\n\"+\n", " \"\\n\"+\n", " \"
\"}};\n", "\n", " function display_loaded() {\n", " var el = document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\");\n", " if (el != null) {\n", " el.textContent = \"BokehJS is loading...\";\n", " }\n", " if (root.Bokeh !== undefined) {\n", " if (el != null) {\n", " el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n", " }\n", " } else if (Date.now() < root._bokeh_timeout) {\n", " setTimeout(display_loaded, 100)\n", " }\n", " }\n", "\n", "\n", " function run_callbacks() {\n", " try {\n", " root._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n", " }\n", " finally {\n", " delete root._bokeh_onload_callbacks\n", " }\n", " console.info(\"Bokeh: all callbacks have finished\");\n", " }\n", "\n", " function load_libs(js_urls, callback) {\n", " root._bokeh_onload_callbacks.push(callback);\n", " if (root._bokeh_is_loading > 0) {\n", " console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n", " return null;\n", " }\n", " if (js_urls == null || js_urls.length === 0) {\n", " run_callbacks();\n", " return null;\n", " }\n", " console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n", " root._bokeh_is_loading = js_urls.length;\n", " for (var i = 0; i < js_urls.length; i++) {\n", " var url = js_urls[i];\n", " var s = document.createElement('script');\n", " s.src = url;\n", " s.async = false;\n", " s.onreadystatechange = s.onload = function() {\n", " root._bokeh_is_loading--;\n", " if (root._bokeh_is_loading === 0) {\n", " console.log(\"Bokeh: all BokehJS libraries loaded\");\n", " run_callbacks()\n", " }\n", " };\n", " s.onerror = function() {\n", " console.warn(\"failed to load library \" + url);\n", " };\n", " console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n", " document.getElementsByTagName(\"head\")[0].appendChild(s);\n", " }\n", " };var element = document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\");\n", " if (element == null) {\n", " console.log(\"Bokeh: ERROR: autoload.js configured with elementid '3fa988c7-a464-4d1c-b335-d4d083e325f8' but no matching script tag was found. \")\n", " return false;\n", " }\n", "\n", " var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-0.13.0.min.js\"];\n", "\n", " var inline_js = [\n", " function(Bokeh) {\n", " Bokeh.set_log_level(\"info\");\n", " },\n", " \n", " function(Bokeh) {\n", " \n", " },\n", " function(Bokeh) {\n", " console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.css\");\n", " Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.css\");\n", " console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.css\");\n", " Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.css\");\n", " console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.css\");\n", " Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.css\");\n", " }\n", " ];\n", "\n", " function run_inline_js() {\n", " \n", " if ((root.Bokeh !== undefined) || (force === true)) {\n", " for (var i = 0; i < inline_js.length; i++) {\n", " inline_js[i].call(root, root.Bokeh);\n", " }if (force === true) {\n", " display_loaded();\n", " }} else if (Date.now() < root._bokeh_timeout) {\n", " setTimeout(run_inline_js, 100);\n", " } else if (!root._bokeh_failed_load) {\n", " console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n", " root._bokeh_failed_load = true;\n", " } else if (force !== true) {\n", " var cell = $(document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\")).parents('.cell').data().cell;\n", " cell.output_area.append_execute_result(NB_LOAD_WARNING)\n", " }\n", "\n", " }\n", "\n", " if (root._bokeh_is_loading === 0) {\n", " console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n", " run_inline_js();\n", " } else {\n", " load_libs(js_urls, function() {\n", " console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n", " run_inline_js();\n", " });\n", " }\n", "}(window));" ], "application/vnd.bokehjs_load.v0+json": "\n(function(root) {\n function now() {\n return new Date();\n }\n\n var force = true;\n\n if (typeof (root._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n root._bokeh_onload_callbacks = [];\n root._bokeh_is_loading = undefined;\n }\n\n \n\n \n if (typeof (root._bokeh_timeout) === \"undefined\" || force === true) {\n root._bokeh_timeout = Date.now() + 5000;\n root._bokeh_failed_load = false;\n }\n\n var NB_LOAD_WARNING = {'data': {'text/html':\n \"
\\n\"+\n \"

\\n\"+\n \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n \"

\\n\"+\n \"
    \\n\"+\n \"
  • re-rerun `output_notebook()` to attempt to load from CDN again, or
    • \\n\"+\n \"
  • use INLINE resources instead, as so:
    • \\n\"+\n \"
\\n\"+\n \"\\n\"+\n \"from bokeh.resources import INLINE\\n\"+\n \"output_notebook(resources=INLINE)\\n\"+\n \"\\n\"+\n \"
\"}};\n\n function display_loaded() {\n var el = document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\");\n if (el != null) {\n el.textContent = \"BokehJS is loading...\";\n }\n if (root.Bokeh !== undefined) {\n if (el != null) {\n el.textContent = \"BokehJS \" + root.Bokeh.version + \" successfully loaded.\";\n }\n } else if (Date.now() < root._bokeh_timeout) {\n setTimeout(display_loaded, 100)\n }\n }\n\n\n function run_callbacks() {\n try {\n root._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n }\n finally {\n delete root._bokeh_onload_callbacks\n }\n console.info(\"Bokeh: all callbacks have finished\");\n }\n\n function load_libs(js_urls, callback) {\n root._bokeh_onload_callbacks.push(callback);\n if (root._bokeh_is_loading > 0) {\n console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n return null;\n }\n if (js_urls == null || js_urls.length === 0) {\n run_callbacks();\n return null;\n }\n console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n root._bokeh_is_loading = js_urls.length;\n for (var i = 0; i < js_urls.length; i++) {\n var url = js_urls[i];\n var s = document.createElement('script');\n s.src = url;\n s.async = false;\n s.onreadystatechange = s.onload = function() {\n root._bokeh_is_loading--;\n if (root._bokeh_is_loading === 0) {\n console.log(\"Bokeh: all BokehJS libraries loaded\");\n run_callbacks()\n }\n };\n s.onerror = function() {\n console.warn(\"failed to load library \" + url);\n };\n console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n document.getElementsByTagName(\"head\")[0].appendChild(s);\n }\n };var element = document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\");\n if (element == null) {\n console.log(\"Bokeh: ERROR: autoload.js configured with elementid '3fa988c7-a464-4d1c-b335-d4d083e325f8' but no matching script tag was found. \")\n return false;\n }\n\n var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-gl-0.13.0.min.js\"];\n\n var inline_js = [\n function(Bokeh) {\n Bokeh.set_log_level(\"info\");\n },\n \n function(Bokeh) {\n \n },\n function(Bokeh) {\n console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.css\");\n Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.13.0.min.css\");\n console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.css\");\n Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.13.0.min.css\");\n console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.css\");\n Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-tables-0.13.0.min.css\");\n }\n ];\n\n function run_inline_js() {\n \n if ((root.Bokeh !== undefined) || (force === true)) {\n for (var i = 0; i < inline_js.length; i++) {\n inline_js[i].call(root, root.Bokeh);\n }if (force === true) {\n display_loaded();\n }} else if (Date.now() < root._bokeh_timeout) {\n setTimeout(run_inline_js, 100);\n } else if (!root._bokeh_failed_load) {\n console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n root._bokeh_failed_load = true;\n } else if (force !== true) {\n var cell = $(document.getElementById(\"3fa988c7-a464-4d1c-b335-d4d083e325f8\")).parents('.cell').data().cell;\n cell.output_area.append_execute_result(NB_LOAD_WARNING)\n }\n\n }\n\n if (root._bokeh_is_loading === 0) {\n console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n run_inline_js();\n } else {\n load_libs(js_urls, function() {\n console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n run_inline_js();\n });\n }\n}(window));" }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Set bokeh to output plots in the notebook\n", "output_notebook()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What Categories of Assault are in the Data?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The OFFENSE_DESCRIPTION field has more details about the type of crime incident. Grouping the data by this column and counting the number of incidents will let us examine which offense descriptions are the most common." ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['ASSAULT - AGGRAVATED - BATTERY', 'ASSAULT SIMPLE - BATTERY', 'STALKING', 'ASSAULT - AGGRAVATED',\n", " 'ASSAULT - SIMPLE', 'ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER', 'A&B ON POLICE OFFICER'], dtype=object)" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "crime_df_time_index['OFFENSE_DESCRIPTION'].unique()" ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OFFENSE_DESCRIPTION\n", "ASSAULT SIMPLE - BATTERY 4321\n", "ASSAULT - AGGRAVATED - BATTERY 1464\n", "ASSAULT - AGGRAVATED 814\n", "ASSAULT - SIMPLE 227\n", "STALKING 20\n", "A&B ON POLICE OFFICER 2\n", "ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER 1\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Group by offense categories\n", "from math import pi\n", "group = crime_df_time_index.groupby('OFFENSE_DESCRIPTION').count()['OBJECTID'].sort_values(ascending=False)\n", "group" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
count
ASSAULT SIMPLE - BATTERY4321
ASSAULT - AGGRAVATED - BATTERY1464
ASSAULT - AGGRAVATED814
ASSAULT - SIMPLE227
STALKING20
A&B ON POLICE OFFICER2
ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER1
\n", "
" ], "text/plain": [ " count\n", "ASSAULT SIMPLE - BATTERY 4321\n", "ASSAULT - AGGRAVATED - BATTERY 1464\n", "ASSAULT - AGGRAVATED 814\n", "ASSAULT - SIMPLE 227\n", "STALKING 20\n", "A&B ON POLICE OFFICER 2\n", "ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER 1" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a pandas dataframe for offense counts\n", "offense_categories = pd.Series(group.index.values)\n", "count = list(group.values)\n", "offense_count_df = pd.DataFrame({\"count\": count}, index = offense_categories)\n", "offense_count_df" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAIECAYAAAAQDvnsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAAIABJREFUeJzt3Wm4JVV59vH/zSQIyNjI0CigqCgCbRCIoARIEIyIMsgkoKIkSoKKeRONiQOGqEEJSpSEAIImCkRFQQ3BMDggDo0MioggCrZobMABRWTo+/2w6tC7D2fs7r1XVdf9u65zdQ17b56jZ9dTtYZnyTYREdE/K9UOICIi6kgCiIjoqSSAiIieSgKIiOipJICIiJ5KAoiI6KkkgIiInkoCiIjoqSSAiIieWqV2AFPZcMMNvcUWW9QOIyKiU6655pq7bM+Z7nWtTgBbbLEF8+fPrx1GRESnSLp9Jq9LE1BERE8lAURE9FQSQERET7W6DyAiYjoPPvggCxYs4P77768dysitvvrqzJ07l1VXXXWp3p8EEBGdtmDBAtZee2222GILJNUOZ2Rsc/fdd7NgwQK23HLLpfqMNAFFRKfdf//9bLDBBr26+ANIYoMNNlimJ58kgIjovL5d/Mcs6++dBBAR0VMrVB/AFm/63FA//0fv/tOhfn5ELLvlfR1ow/f+1FNP5dhjj+Wxj33scv3cPAFERLTcqaeeyn333bfcPzcJICJiOfjIRz7Cdtttx/bbb8+RRx7J7bffzl577cV2223HXnvtxR133AHAy1/+cj7xiU888r611loLgCuvvJI/+qM/4qCDDuJpT3saRxxxBLb5wAc+wJ133skee+zBHnvssVxjXqGagCIiarjxxhs56aSTuOqqq9hwww255557OProoznqqKM4+uijOfvsszn++OP59Kc/PeXnXHvttdx4441suumm7Lrrrlx11VUcf/zxnHLKKVxxxRVsuOGGyzXuPAFERCyjyy+/nIMOOuiRC/T666/P1VdfzeGHHw7AkUceyVe+8pVpP2ennXZi7ty5rLTSSuywww786Ec/GmbYSQAREcvK9rRDMsfOr7LKKixatOiR9z3wwAOPvOYxj3nMI9srr7wyDz300BCiXSwJICJiGe21115ccMEF3H333QDcc889POc5z+G8884D4D//8z/ZbbfdgFLm/pprrgHgM5/5DA8++OC0n7/22mtz7733Lve40wcQESuUGsM2n/GMZ/CWt7yF3XffnZVXXpl58+bxgQ98gFe+8pWcfPLJzJkzhw9/+MMAvPrVr2b//fdnp512Yq+99mLNNdec9vOPPfZY9t13XzbZZBOuuOKK5Ra3bC+3D1vedtxxR89mQZjMA4jon5tuuoltttmmdhjVTPT7S7rG9o7TvTdNQBERPZUEEBHRU0kAEdF5bW7KHqZl/b1nnAAkrSzpWkmfbfa3lPR1SbdIOl/Sas3xxzT7tzbntxj4jDc3x2+W9PxlijwigrIoyt133927JDC2HsDqq6++1J8xm1FArwNuAh7X7L8H+Gfb50n6V+AY4PTm31/YfrKkQ5vXHSLp6cChwDOATYH/lfQU2w8vdfQR0Xtz585lwYIFLFy4sHYoIze2ItjSmlECkDQX+FPgJOAElRkNewKHNy85F3g7JQHs32wDfAL4l+b1+wPn2f498ENJtwI7AVcvdfQR0XurrrrqUq+I1XczbQI6FfhrYFGzvwHwS9tj09QWAJs125sBPwZozv+qef0jxyd4T0REjNi0CUDSC4Gf275m8PAEL/U056Z6z+B/71hJ8yXN7+MjXUTEqMzkCWBX4EWSfgScR2n6ORVYV9JYE9Jc4M5mewGwOUBzfh3gnsHjE7znEbbPsL2j7R3nzJkz618oIiJmZtoEYPvNtufa3oLSiXu57SOAK4CDmpcdDXym2b6o2ac5f7lL9/xFwKHNKKEtga2Bbyy33yQiImZlWWoB/Q1wnqR/AK4FzmqOnwV8tOnkvYeSNLB9o6QLgO8CDwHHZQRQREQ9s0oAtq8Ermy2b6OM4hn/mvuBgyd5/0mUkUQREVFZZgJHRPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9FQSQERETyUBRET0VBJARERPJQFERPRUEkBERE8lAURE9NS0CUDS6pK+Iel6STdKekdzfEtJX5d0i6TzJa3WHH9Ms39rc36Lgc96c3P8ZknPH9YvFRER05vJE8DvgT1tbw/sAOwjaRfgPcA/294a+AVwTPP6Y4Bf2H4y8M/N65D0dOBQ4BnAPsCHJK28PH+ZiIiYuWkTgIvfNLurNj8G9gQ+0Rw/F3hxs71/s09zfi9Jao6fZ/v3tn8I3ArstFx+i4iImLUZ9QFIWlnSdcDPgS8APwB+afuh5iULgM2a7c2AHwM0538FbDB4fIL3RETEiM0oAdh+2PYOwFzKXfs2E72s+VeTnJvs+BIkHStpvqT5CxcunEl4ERGxFGY1Csj2L4ErgV2AdSWt0pyaC9zZbC8ANgdozq8D3DN4fIL3DP43zrC9o+0d58yZM5vwIiJiFmYyCmiOpHWb7TWAPwZuAq4ADmpedjTwmWb7omaf5vzltt0cP7QZJbQlsDXwjeX1i0RExOysMv1L2AQ4txmxsxJwge3PSvoucJ6kfwCuBc5qXn8W8FFJt1Lu/A8FsH2jpAuA7wIPAcfZfnj5/joRETFT0yYA2zcA8yY4fhsTjOKxfT9w8CSfdRJw0uzDjIiI5S0zgSMieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6KgkgIqKnkgAiInoqCSAioqeSACIieioJICKip5IAIiJ6atoEIGlzSVdIuknSjZJe1xxfX9IXJN3S/Ltec1ySPiDpVkk3SHrWwGcd3bz+FklHD+/XioiI6czkCeAh4I22twF2AY6T9HTgTcBltrcGLmv2AfYFtm5+jgVOh5IwgLcBOwM7AW8bSxoRETF60yYA2z+1/a1m+17gJmAzYH/g3OZl5wIvbrb3Bz7i4mvAupI2AZ4PfMH2PbZ/AXwB2Ge5/jYRETFjs+oDkLQFMA/4OvB42z+FkiSAjZqXbQb8eOBtC5pjkx0f/984VtJ8SfMXLlw4m/AiImIWZpwAJK0FfBJ4ve1fT/XSCY55iuNLHrDPsL2j7R3nzJkz0/AiImKWZpQAJK1Kufj/p+1PNYf/r2naofn3583xBcDmA2+fC9w5xfGIiKhgJqOABJwF3GT7lIFTFwFjI3mOBj4zcPyoZjTQLsCvmiai/wH2lrRe0/m7d3MsIiIqWGUGr9kVOBL4tqTrmmN/C7wbuEDSMcAdwMHNuc8DLwBuBe4DXgFg+x5J7wS+2bzuRNv3LJffIiIiZm3aBGD7K0zcfg+w1wSvN3DcJJ91NnD2bAKMiIjhyEzgiIieSgKIiOipJICIiJ5KAoiI6KkkgIiInkoCiIjoqSSAiIieSgKIiOipJICIiJ5KAoiI6KkkgIiInkoCiIjoqSSAiIieSgKIiOipJICIiJ5KAoiI6KkkgIiInkoCiIjoqSSAiIieSgKIiOipaReFjxF6+zpD/vxfDffzI6JT8gQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9NW0CkHS2pJ9L+s7AsfUlfUHSLc2/6zXHJekDkm6VdIOkZw285+jm9bdIOno4v05ERMzUTJ4AzgH2GXfsTcBltrcGLmv2AfYFtm5+jgVOh5IwgLcBOwM7AW8bSxoREVHHtAnA9peAe8Yd3h84t9k+F3jxwPGPuPgasK6kTYDnA1+wfY/tXwBf4NFJJSIiRmhp+wAeb/unAM2/GzXHNwN+PPC6Bc2xyY4/iqRjJc2XNH/hwoVLGV5ERExneXcCa4JjnuL4ow/aZ9je0faOc+bMWa7BRUTEYkubAP6vadqh+ffnzfEFwOYDr5sL3DnF8YiIqGRpE8BFwNhInqOBzwwcP6oZDbQL8Kumieh/gL0lrdd0/u7dHIuIiEpWme4Fkj4O/BGwoaQFlNE87wYukHQMcAdwcPPyzwMvAG4F7gNeAWD7HknvBL7ZvO5E2+M7liMiYoSmTQC2D5vk1F4TvNbAcZN8ztnA2bOKLiIihiYzgSMieioJICKip5IAIiJ6ato+gIiZeua5zxzq53/76G8P9fMj+iZPABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPZUEEBHRU0kAERE9tUrtACLa4qanbTPUz9/mezcN9fMjZitPABERPZUEEBHRU0kAERE9lQQQEdFTSQARET2VBBAR0VNJABERPTXyBCBpH0k3S7pV0ptG/d+PiIhipAlA0srAB4F9gacDh0l6+ihjiIiIYtQzgXcCbrV9G4Ck84D9ge+OOI6IFc4H//zyoX7+cf+651A/P0ZPtkf3H5MOAvax/apm/0hgZ9t/MfCaY4Fjm92nAjcPMaQNgbuG+PnDlvjrSvz1dDl2GH78T7Q9Z7oXjfoJQBMcWyID2T4DOGMkwUjzbe84iv/WMCT+uhJ/PV2OHdoT/6g7gRcAmw/szwXuHHEMERHB6BPAN4GtJW0paTXgUOCiEccQERGMuAnI9kOS/gL4H2Bl4GzbN44yhnFG0tQ0RIm/rsRfT5djh5bEP9JO4IiIaI/MBI6I6KkkgIiInkoCiIjoqV4kAEnPrh3DspB0QO0YlgdJa0ua1/ysVTue6DZJT6wdw7KQtEbtGHqRAIAzJX1Q0uNqB7KUjpX0ua7+wUtaTdKZwE+Ac4GPAgsknSFp1brRTU/Sxwe2/3Hcuf8efUSzI+lxkrYe2D9Y0lHNz+NrxjYTkv5Q0kGSNmr2t5P0MeArlUObEUmPl7SDpFWa/Q0lnQjcWjm03iSAZwE/BOZLOqx2MLNlex/gw8D/SHqzpHWbL/XjOpLU3gKsBcy1vZ3tbYEnAmsCf1c1spl52sD2PuPObTzKQJbSe4FdB/bfBTwbeB7wjioRzZCkk4GzgQOBz0l6G/AF4OvA1lO9tw0k/SWl1tm/A1+XdASlvM16wM41Y4OeDQNtKo9eTSk/sYhSmsK2168a2AxJeiblrudeloz/CVUDm4akbwN/aPs3446vDXzV9jPrRDYzkr5l+1njtyfabyNJ1wLPcvNll3St7XnN9lds71Y1wClI+i4l9vslrUepHLCd7VsqhzYjTfzPs32XpC2A7wN72L6qamCNUdcCqkbS0ZS7zbdRSlIvqhvRzDWzpt8MHAYcbfvTlUOatfEX/+bYvZK6cAfy2Cb5rgSs0Wyr+anejjsDq3jJO70jB7bXHXUws/Q72/cD2P6FpJu7cvFv3G/7LgDbP5L0/bZc/KEnCUDSl4CfAbvb7mLtoeuBiyl3QvfVDmYpLGru9qctBthSC4EPNdt3DWyPnWu7RZI2tv0zANvfAZC0Ge2/EXqSpMFyMVsM7tt+UYWYZmOupFMG9jca3Ld9QoWYHtGLJiBJ+9i+ZJJza9j+3ahjmg1Jz7T97UnOzbW9YNQxzYakBSxushpjOtKENRVJf2D7mtpxTEXSy4DXAW8Erm0OP4vSN/AB2x+tFdt0JO0+1XnbXxxVLEtD0jFTnbd91qhimUgvEgCUnnhgE+A7TU2iDYHjgWNsb1Y3uuk1Q1k3A77StCc+A/gbYE/bc+tG11+S7uhCApO0D/C3wDOaQ98B3m271aOYJD3O9q8nOfcE23eMOqblRZJc+QLci1FAbe+Jn46kdwH/CRwBXCLpLcAVlKahp9SMbSYGR15J2mXcudeMPqLlaqJmrdaxfYnt59neoPnZve0X/8aVYxuSLht3rvV9YZK+OLB9zrjT1Z8ce9EHALwGeGpbe+JnYH9ge9u/k7Q+ZSTE9raHuVra8vT/gLGx9B+iND+MeTVw+sgjWn5a/wgt6a1TnLbtd44smNkbTLDjR+t1IfkODtPebty56vH3JQG0uid+Bu4f66ewfY+k73Xo4g9L/qGP/6Ov/iWYjqQLmfhCL2CDEYezNH47wbE1gWMo8bc5AXiS7Yn222iqGKvH35cE0Oqe+BnYStKnmm1RRkKM7WO77aUiuv4l/pelPNcKtt83tt2Mxnod8ArgPOB9k72vJTaSdALl735sm2Z/2jVvW2BdSftRmtvXkTQ2aknAOvXCaoLoQydw23vipyNpr6nO2x7fNtoqku4Dvkf5o39qs02z/xTba9aKbaaasf9PAm7s2Dh0AJqmwxMo/UjnAu+3/Yu6UU2vmfk7Kdttn8k85Qgr20dOdX7YepEAptKGnvjpSFrT9kSP8UjazPZPRh3TbEh60lTnbf9gVLEsDUl/S2ku+RalhMKJts+uG9XMNeUUDqCsQvXBiSblRT/1IgFI+qLt3Zvtc2y/fOBcF6byD5YiuNT23hOdaytJ/21739pxLC1JNwI72f6tpDnA5213psKspEXA74GHWLLJbWweRmvrSUn6J+A22/867vgbgI1t/02dyGZG0uuAe8ffMKgsjSvbp9WJrOhLH0Cre+JnYDDG8e2eXYi/CwXTpvL7sScw2wsldWr4tO1OxTvOC4FtJzj+fuAGylyYNnsVsOMEx88EvgEkAYxAq3viZ6DrnaiDnV+PYvuiyc61xPhO+Cd1qRO+af+flO17RhXLUrDtR5WrsL1IUhdufrD9+wmO3d+G+PuSAFrdEz8DG0k6nsUjIY5vjndlJMQ6wEFMXguo7QngwHH7rR/5M841LC69MZ6BrUYbzqzcJ2nr8R3vKusbtLqEyxhJc2wvHHdsI1rw9N6XPoBW98RPR9KU47Rt//2oYlkaXein6CtJ67f5CUDSvpRmkn9g8czZHSnVcV9v+/O1YpsJSS8HXgu8gTKIAOAPKHWY/s32hyuFBvTnCeATtj9TO4hlcKftLs+WrX6nsywkfWuq821PbpLOtP2qCY7PBS5h4jb2VrD935JeTJlN/pfN4RuBAycrkNgmts+RdBfwTyyuw3QjcJLti+tFVvTlCaDTd6ArQPzb275+guM7A4fbfl2FsGZM0neAB4GPAZ+jjKh5RAeGsZ4LrAwcNdaerrI40ueAd9g+p2J4UVFfngCiosGLv6RtgcOBQyg1jT5ZK66Zsr1tE/dhlKJ811OSwf9O1EHZQi8H/g04X9KhlAKI5wN/bvtzNQObjqSLmWKgQ9vXA5D0z0wdf9YDGLaBmaiPOkUZZdDqu2tJDwETlcTtxJKWkrYCDqVc+H9Dufi8oQtllCci6RDKqnLvsX1y7XhmStL7KYX4ngi81PbXKoc0rawHMFx9SQA3AlMNQ2z7I/y1TDyWGADbD48wnFlrJiJ9GXi17e83x26z3ebRJ0uQtDHlqeUgShK7APjkZLXq20TSaSweBXQ4pTPyprHzto+f5K3Vpeb/cPWlCej3bb/IT6ftF/lpHEJ5AriseaQ/nw51DDd16NcF/ovSnLJw4NykC5a0yPxJtqH980g+TVM+XNInbY8fktt217A4/lNtv75yPEvoSwLowsLdU/nU9C9pL9v/BfxXU4nyAMoQvo2bO9MLbV9eNcDpPZVyoTyOMqRvjJrjrW7Ksn3uZOckvXeUsSyFwRuFzjwxDhiM/3nVophEl6eIz8b9tQNYRi+pHcDyYPte2+fa3gfYnNIv8/a6UU3P9lzbmzc/Txj42byr/RgDXlo7gGlMNQu+C1odc1/6ALo+jLLr8X+asiLYRWML23SJpPH1o5Zg+4ZRxbK8Sfqx7c1rxzEZSQ9TFrQR5Un+vrFTtLyQHUxbCr36AJS+JICMAqpI0oGUPoDdgS9QksElth+qGtgMSfryFKdtu3WP9oOmqAUk4Hrbc0cZT5+0vRR6XxJARgG1gKQ1gRdTksGOwMXAx21fUTWwFZykHzJ5LSBsbznaiGZO0gG2P9Vsr+cOLGIzSNKzbX+zdhyT6UsCuNb2vNpxLK2uxz+RZoWtjwDb2V65djxTkfQHwALb/9fsH0HpzL6dsjjML2vGtyIbtxZG55pCx8X/Fdu71Y5pUF86gSed8NIs8NF2k44CktSZxCBpQ0mvkfRF4LPA5ZRZqW13BmUxFSTtRinkdQFlcMEZFeNaapKeJOktTZmLNtMk210xGHPrlj7txTBQ268Z3G+GI76EMilme2CTGnHNlO0lqoFKegqLZ9beD+xQI66ZkvQKShmFZ1LGdb8V+FKbJ8iMs4rtu5vtQ4EzbJ9PKa3wqBpHbSVpE8qcjMMpCyO9i/L/S5ut0dzkrASs3mw/clG1PWWhvhZYqbnerDSwPRh/1TkkvWgCApD0GMrqQodT7jrHxqRf2YU29KZy46GUL+zKlGGUO9u+tWpgM9CU4/44cGlXOn4HNXfJ29t+WNJNlBo6Xxw7Z7u11TQBJL2a8nczl/LkcgHwmTa3/Y+RNFX/kG3vObJgloKkBcAiJlmLofYw4l4kgKYa4p6UJofzgP8Fvt+FLwCApC8BG1Fm0J5n+yZJP+xK/BOR9ETKRekw29vXjmcqkt4K/DFlBvCTgXkuK1I9Cfio7edUDXAakh4ArgbeaHt+c6xTpThiOHrRBERZgOEu4FrgOtsPSupS5ruXUsBrHcqTC7R8gslEmlWQXkp5CnsWcDKltEKr2T5R0uWUpsJLBiqArgq0to7OgE2Bg4FTJD2e8gSwat2QZq75uzmOUk/fwHeBD9r+edXAZkjSqpSn98H4z7f9QNXA6MkTADwy6uQwygXox5RFMLaxfVfVwGaoGct9EOV3eAKwPrBXB9pAB/sAtgI+weJCap19gumqcU2Jj6WU4vjbulFNTtKulNLb51Dq6ohy83A0cITtq+pFNz1JT6MMd/4GS8a/E/Ai2zdN8fah600CGCRpF8pd6AHAbW2fyDOepE0pX+JDgcfbfmLlkKYk6UHgq5Ql/K5tjqUJYkQk7TJR6WdJTwUOtf2OCmHNiKSvAa8Z+7sZOL4DZUnFVo8ik/QF4H22Lxl3/PnAX9veq05kTRx9TABjJK0E7GH7stqxLC1JW9m+rXYcU2ke4Q+h3HWuS+nLeFWbSxCsSLo4fn6MpO/afvpsz7WFpO/Zftpsz41KL+YBSHqMpCMkvaDZP6GpT3MypV+g1SRt0IzZfq2kx0o6TdJ1klq/mhaA7Z/bPq3pLN2XsqTiPZK+LenEyuEtFUn71I6hJyRpvQkOrk83rl8rS1pt/MFmVGL1CZC9eAKQdB6l7e2xzc8tlIlIuwFPd/uXlbsEuAFYC3gupU304mb7pbb3qBjeUpO0DWUU0FtrxzJbXbqrlvRL4EuTnW/z37+kY4FXA39FWcgGyqCO9wBn2/63WrHNhKS3UeJ9re0FzbG5wGmUOkxvrxheb0YBbeuyrusqwE8G2t0+K6kLlRw3sb2PJAF32H5Xc/w7kv6sZmDLaGPg2bWDWEpdmpW6EHhf7SCWhu0zJN0JvJMlR9H8g+2LqwY3A7bfIen1wNeb64+BhymtD++vGhz9SQAPANh+SNJPxp3rwsSkh6HMGpG0cNy51i9KrrKu6+mU4YifpsxAPZdS3vcfKoa2LF47/Uta4zdu+dq5U7H9WcoTeyfZPhU4tWnKku17asc0pi8JYK6kUyh3bWPbNPub1QtrxraS9ClKvGPbNPtdGEp5KmW8/NWUPoBvAO+wfcqU72ox21fXjmEWfiFpY9s/A5B0FHAgpZjd29t0QVqRtbGSaV/6AI6Z6rzts0YVy9KQNOVQsbaPYhpfzVTSbcCTOlQLqNMkfQv4Y9v3SHoeZTb8X1JqSG1j+6CqAUY1vXgCaPsFfjptv8DPwDqSxnc07le6NMD2RaMPaeYkbWZ7fNNhl6w0cJd/CKWY3SeBT0q6rmJcUVkXhlFF911FKUUw9jO434W7z9Z3Nk5jlaYDEmAvSk2sR85ViGfGJJ06sP26cefOGXlAsyTpjQPbB4w7985Hv2O0etEEFLEsxjdhdY2ktwAvoNTDegLwrGZAwZOBc23vWjXAKWiKBWG6MBS37fG3OvvH5CRt2KE6RpsCTxzrOJV0PGVOA5Tqpq2eyQxsNjBw4FFsnzDKYGbL9kmSLqMUs7t0oO9lJUpfQJutSAvCjI+/+u/T2wQg6Ru2d6odxzK4lFJUqgtOppR/GPMXwFmUSXknAi+rEdQs/A64sXYQy2KiWkC2v18jlllaqRk+udLA9tiFs/pM2hnwJNsT7Y9cbxMAHSqHO4nqdw+zsM24jt7f2n4PgKQvV4ppNu7u+kCCDluHxVU0YfFsYGjBBXQGtpd0DyX+tZttmv21Jn/baPQ5AVwy/Uta7ezaAczC6uP29x7Y3nCUgSylSVeMk7Sp7TtHGUyf2N6idgzL6FF1gNqkt6OAbL+5dgzLwvZptWOYhd80HY4A2F4Ij6xt/NtqUc2Q7anKVTyqaSWWH0nPl/SokWKSDpf0JzVimqUdKHMwHh78AfahrMtcVW8TQIzU2yl1l46QtE3z8zLK8Mq31Q1tmXWpKa6L3gFMVMbickr/UdudTCk+Od4twHtHHMuj9LkJKEbE9uebGkx/A/x1c/hG4BDbXZ80zisiAAAgAElEQVSI1IV26C577NgT4yDbP5O0Zo2AZmnORKPcbH9f0pwaAQ3qRQKQtKbtCZsaujTLU9LalEXJAW6x/Zua8cyG7espq7B1jqTTmPhCL8oCNzE8q0taxfYSRRubdXbXqBTTbEwV42NHFsUk+tIE9MhIE0mXjjvX+lmeklaTdCbwE0oVzY8CCySd0XwRWk3Sc5omn7H98yRd2vz8UcXQZmo+ZSTK+J/5tH8cfdd9Cvj3wbv9Zvtfm3Ntd7mkRy25KemtwJWjD2dJvXgCYMl22vGPXV1ow30LZcjYXNu/BpC0DvAh4O9ofzv6icDrB/a3BY4B1qQ0C11ZIaYZs31u7Rh67O8oJcNvl3Q75fu6OWUeyd/XDGyG3gicLen7LF59cAfg28ArqkXV6EUpiLZPx56OpG8Dfzi+yadpEvqq7WfWiWxmJH1zcCSNpM/Y3r/ZvqrNpQgAJF3Moyf03AVcYfs/6kTVL5LWYHHz5622f1czntlqRrw9o9m9sS2T8PryBLBRU35AA9s0+9U7YmZiovZ+2/dK6kIGX6KdfOzi33j8iGNZGhON1lgfeJmkbW2/adQB9cX4AmqNrQcqyba6GUjS4FDPHzT/rj523HbVFQn7kgA+zOIL/eA2wDkjj2b2FjV3+xM1V3UhAdwsaR/bS0y+k7Qv0Io7oalMtpqWpIsofQFJAMOz3xTnTPv7AT44xTkDzxtVIBPpRQKwPWlboaS/GGUsS2kDyrDJwQTgZr8LCeAEyjyAL7Lkwt67M/UXvNVsPzx2JxrDYXvSdnJJB44ylqVh+7mTnZO04yhjmTCGPvQBTEXSHbafUDuOFV3ThnskA+2gwEe70JYraf0JDq8HHAU82fYRIw4p6P53tw3x9+IJYBqtv4WTdJjtjzfbuwxWdpT0Gtun14tuZpoL/RmDxyTtLOlw26+b5G1tcQ2Ln7hotu+mjF56TaWYogPf3WlUjz8JoBtNKP8P+Hiz/SGWLAP9aqD1CWCMpG2Bw4BDgTuBT9aNaHq2t5zsXBfmYazAuvDdnUr1+HuRACT9gkffwdHsr10lqNlp9aIS05G0FeWCfzjwG8raAKtO1T7aZioN/3tQfp/96MZIpk5qhkBPNgu79f+7S7qQyePfYMThPEovEgDdKDk8lVYvKjEDt1JmYx8wNv5ZUudm0EramXLRfwllGOhxlKezGJ4X1g5gGf3LUp4biV50Akt6DKWp5MnADZR1UCet8d42ku4Dvke5a3hqs02z/xTbrS6KJelgyhPATpTSG+cD50zVtNImkk4CXgrcQWmKuxCY35X4VwSStqQMIDBwUweWEV2CpNWArSjx/9D2A5VDAvqTAD5OuVh+mVKH+1bbb6gb1cxJetJU523/YKrzbdHMZTiA0gewO3AmcKHty6sGNg1JC4GbgVOBz9q+X9JttreqHNoKT9LjKH8nOwLXUb7H21M65o8ZK43SVpJWBt4JHEup5bUSZW3mM4C3ji9yN2p9SQDfHiuX0HTafb3t5R8GSfpv2/vWjmN5krQhcAilJHTVyTDTab7Ee1MS157AFcAfA5vX/gKv6CSdA/wIONH2ouaYKHWAnmz7qHrRTU/SeylN0K+z/avm2LrA+4Bf174R7UsC6Fz9n0GSrrU9r3YcAZJWp7RLHw7sClxmu5NlrrtA0i22t57tubaQdAvw1LHkNXB8FUpTVtX4+9IJPLYwMyy5OLMA255ook+brCPpRZOdHLfgegyR7fuBTwCfGGjSiuFp/Si3aXj8xb85+FAb6nj1JQG0emHmGVgHOIjJawElAQyRpP2AG2zf3uy/FTgQuB1o+yS2rruq+d/7nR5orpD093RjPeabmsmOHxs8KOkwSr9SVX1pAnrcVOc70JHUqSarmZioOFxbSboB2MX2fZJeCJxC6Q+YBxxs+/lVA1yBNd/dsyiTH6+j3PDMo9TWP2asXb2tJM0FPgP8gsUzyp9NqZD7Yts/rhhebxLAIuCnwNjQzyWKqtWuxzGdFbEPoEtJTdL1trdvts8Gbrb9nma/M79HlzUj4Z5O+e7e2JWRb2Mk7U0ZxipKHaxL3YKLb1+agE4HdgO+CHzc9tWV45mtl090cGxiUgdq6UykS227krQWcB+wF6Ucx5jV64TUD5I2Av6WMofn28C72v7EPmjcHKRvU+a/tGYOUi/WBLZ9HOWx8WLg1ZKulfSPkp5YObQZaRZUB0otnSb2H1AWKvlhvciWyWtrBzALp1KaH+ZTRm7MB5A0j/JkGcPzEeC3wGmUZVE/UDecWTuHcvN5C/Bi4OSq0YzTiyagQU2b4qHAPwJvsf1vlUOa1iS1dN7Q9qarFYmkzYCNgOsHxqNvDKxm+46qwa3AJF1ne4eB/U41ubV9DlIvmoCaWvT7USYebQZ8Gni27a7cPa8QtXS6qnlS/KXtnzT7e1Du5m6nBfVcVnCStB6LmwxXHty3fc+k72yHB8c2bD/YtgWEevEEIOleynqcH6fU0Vnil277OPoVoJbOZmMXzy6S9HXgJbbvlLQD8L/Au4DtgAdtv6pqgCswST8CFjHJEOi2l+OQ9DAwNlJprPrwr2nJHKS+JID/YPKqmW77dPIxHa6l06nH9vEk3WB7u2b7vcAi238taSXgurFzEeM1ZUQmVbtDuBdNQLZfVjuG5cH2vcC5wLkDtXTeDrQ6AdCtET8TGYx/T+DNALYXte2RPtql9gV+On15Ajh+qvO2Wz2yQNKnKc1XF3VhDd3xJP0c+I/Jzts+YYThzJqk91MqOP4UeBGlBPeDkjYBLrZdfXHviKXRiycAYE7tAJbRRyl9AKdJ+gIlGVzSoUqUv6NMfumq11OetjYBdrM91rG3MWWMekQn9eIJYEUhaU3K6JNDKfXRL6ZMbLuiamDT6HofwGQk7QYc1swzieicXiQASa8EvmT71qaW+L+xuJjXKwYnWnWFpGdSJslsZ3vKjqbaJH3T9rMnObep7TtHHdPSakYBHU5ZIeyHwCdtZyjokDQj+AbX8KbZX4UyB6PVrRhNOejBi6wG9m37qaOParFW/4+3HJ1AaUaB8ij/bEpdkXmUGYatXpBkTNPxOzYkdAtKWeJX14xpJia7+De+BrR6Qpukp1D+Nz8MuJsyDFe296gaWA/YXntwvxkJ91rgzyhLc7bdbuP2V6KM5Pt/QPUbz74kgIcG2m33o6wJ/H/AJZL+sWJcMyLpFZSLzzMpk9jeSnmiWREe37owjOZ7lIl4+9m+FUBSZ5YUXRE0q2i9HjgK+BhlIufddaOaXnOdGVvF7HDgTZT+sBfZvqFmbNCfBGBJjwd+SSnm9a6Bc2vUCWlW9qTUo7m0Qx2/M9WFJHYg5QngCkmXAOfRjcTVec1T7xspT+5nA/PaXgJ6ULPy19HAXwFfBw6yXX0dgDF96QPYn8UVHC+1/Yrm+HOBN9t+QbXgllJTnuAwSifk9rXjmYqk05j4Qi/gaNtTrtfQFgOd8GNrA59LmYh3adXAVmCSfgssBD4M3Dv+vO1TRh7ULEi6gzKT+Z+ZoHBj7SoEvUgAAJJWA9axvXDg2NqU/w06UV62KY37Usqj5LMolQU/ZfvaqoFNQ9LRU523fe6oYlleJK1P6Y85xPaeteNZUUl6O1M8Jdp+x+iimb22VyHoTQLosoE+gK0oHb8XUEafdKIWUES0U1/6ALruDOCrlOUHrwVow4LSMyXpYpa8CzJwF3CF7UlnCEcASNqXUn7j6ZS/ne8C77H9+aqBzZCkbSh9AM9gcfzvtX1T1cDoSQLoejVKSgnrQ4APNqMhzgdWrRvSrLx3gmPrAy+TtK3tN406oOgGSa+mDPn8a8qCPFAmQb5b0lzbZ1QLbgaaNaTfD7wH+CCl3+sPgIsknWD74qrx9aEJaEWaiTrY+UsZU3yh7bfWjWrpNJUSrxlc8CNikKTvUspv3DPu+AbAV2xvUyeymZF0HWUdj9vGHd+K8t2tOoCjF0tCsgIN2bN9u+13N384L60dz7Joe6XEaAVNtOhLF+YANFYdf/EHaI5Vf4rvRRMQsJmkSYeLtb0a5RQ2psxqbrVmxMx461Em9XS5SFwM368lbT++XIuk7ZlgWGgLPdw0VS0YPChpc6D6DVBfEkCnq1FK2h04HdiUMhP4XZQx6GsA/1AxtJm6htL5NVjL5W7gSuA1lWKKbngjpb38wyz+O3o2ZXJVF9b5eDtwmaR3smT8b2l+qkofQAdIupZSO+RqYF/gLOAdbZ8EMxOSVh0o0xHxKM0s/uMoo2hEuZn7oO2fVQ1shiQ9i8WjgAR8B3if7WuqBkZ/EkCnq1FKutb2vIH924AndbUWUFMXZQ/KhLb9bD++ckgRvdSLJqCuV6ME1pH0onHH9htbjrD2dPKZkrQz5aL/Esow0OMoTzYRUUEvngCmIunHtjevHcdUJH10itPVp5NPR9JJlBFLd1BWM7sQmJ+ZzBF1JQFId9hu+xNAp0laCNxMqWj6Wdv3S7rN9laVQ4uoRtIatdf47kUT0DTVKNcdcTizJmlT4Im2r272jwfWak6fN9E445bZGNibMnntVElXAGtIWmUFLG8dy1FTDvo44BeUctAnA88FfgC8cWx9hjZrOrE3Ab5j+6HmdzoeOIYyy7+avkwEm08ZgjX+Zz7wlxXjmqmTWXJh+7+gjCF+DHBilYhmwfbDtv+7aap6MvAZyoimn0j6WN3oouU+Rvk73xr4BnAbcBDwWeDMinHNiKS/pNT++Xfg65KOoDwNrwfsXDM2SBNQJ4wfxjo4KkjSl20/t150S68px31AF8tBx2hIut729s3IsdsHm2slXdf2MiJNKYvn2b5L0hbA94E9bF9VNbBGX5qAul6NcvVx+3sPbG84ykCWhqT9gBts397sv5WyytbtwOtqxhat9zCUkQ6S7hp3blGFeGbrftt3Adj+kaTvt+XiDz1JAHS/GuVvJD15rL1zbFGbZrHy31aNbGZOAnaBR6ojvozSHzAP+Ffg+fVCi5bbStJFlP66sW2a/S6MIps7rgzNRoP7tcvQ9LoJqCvVKCW9ADgFeCfwrebwHwB/D5xg+3O1YpuJscf4Zvts4Gbb72n2Oz1LO4arKYMyKdtfHFUsS0PSMVOdt33WqGKZSF+eACZk++GxyVRtZvvzkn4C/A2lLjqU6fCH2L6uXmQzJklrAfcBe7F4fWZ4dPNWxCOmusA3JaFbbaoLvFpw8elFAlgRqlE21RAPrx3HUjoVuA74NXCT7fkAkuYBP60ZWHSLpB8AnwP+AziHskpYa0n6ou3dm+1zbL984PQ1lLW9q+lFAqDj1SglPQfYaqzDWtJ5lD4MgH+0fWWt2GbC9tmS/gfYCBgs6/tT4BV1ooousv0kSW+gDCPuwt/O4wa2txt3Lk8AozBVyQFJ1RdlmIETgdcP7G9LmUSyJqVZ6MoKMc1Ys4rZL8eW5ZS0B/Biyiigf6kZW7SbpEuBVw+MINsF+HPKMpEvBD5SMbyZmKqTtXoHbF8mgi1BxZ6SzgQWTPuG+tax/Z2B/R/Y/rrty1nyDqOtLqAkKyTtAPwXpS7Q9izZHxAx3kYDF/8/pcwG3s/2mXRjFNC6kvaTtD9NUcfmZ39gndrB9eIJYEyHq1EuUa7C9v4Du10opbzGQMntlwFn236fpJUofQMRk/m9pKOBzSnlE+bZ/omkx9HcVLTcVSxeuvWrwMED5746+nCW1IsEMEE1yhMp1Si7MgP1Zkn72L5k8KCkfSkzC9tusK1zT+DNALYXtWAgRLTbEcCbgAeA9wDnSvoSsD8dKAVh+8jaMUylF/MAul6Nspnw9Vngiyw5D2B3yuPw92rFNhOS3k8phvVT4EXAU2w/KGkT4GLbO1YNMDqjGTn2x8C1tv+3djzTaQo3Tsr2B0YVy0T60gewMWU26ouAW5v6+mtI6sQTkO3vU9rLvwk8rfn5BrBd2y/+jdcDnwJ+BOw2sATkxsDf1goqusf2tbZPBn4n6YO145mBOQM/bxq3P2eK941EL54ABklanTJ64HBgV+Ay250cXz/Wp2G7k/V0JO0GHGb7uNqxRPs1AwgOpzTn/hD4lO3T6kY1c+OXdm2DTtwBL0+27wc+AXxirBpl5ZBmRdK2lDo6hwJ3Ap+sG9HsTPAl7lT8MVpN8+ehlL/5u4HzKTeue1QNbOm07m67Fwmg69UoJW1F+RIcDvyG8iVYtStloFewL3GM1veAL1P6um4FaCaCxXLQiyYgSTcAu9i+r6lGeQqLq1EebLvV1SglLaJ8CV7d9AfQsU7ssfiPGfgSdyb+qEfSSyg3D88BLgHOA87synrSkq5l8Z3/0ygJDcrIONcuhNiLJwDK/9D3NdsHAGfZvga4RtJrK8Y1U4dQvgSXNWsbnE8LppHPwoGU+K+QNPYl7lL8UYntC4ELJa1JmT3+BuDxkk4HLrR9adUAp3dQ7QCm0qcngOdQqlH+EDhwoCDZd223uqDUmIE+i8MoQ0DPpHwJLq8a2AwNfIkPo8wHOJdufImjRZrijgdTquHuWTueqUj6b9v71o5jMn1JAK+kDDf8NfBz2/s0x+cB77W9V834lkazsPQhlC/B82rHM1td+hJHLK02jvwZ1IsEACBpM5pqlLYXNcc2BlazfUfV4CJihSTpNpYs5LgE2xdNdm4UetEHkGqUEVHJOpR+gIn6vAxUTQC9eAKQ9HXgJbbvbMah/y/wLkp97gdtv6pqgBGxQmr7kqe9eAJgBaxGOVFxuIhonVaPdutLLaDx1Sgvg1KNsk44y8U/1g4gIqb1cgBJT5C0j6TnS3pC5Zge0ZcngMslXUCpRrkecDlAU43ygZqBLYNW31lEBAC3SfoYpe7Y9ZTv7TMlXQ0ca/vemsH1pQ9AlCGTmwAXDHQGzwPmdHEcuqQ/tH117TgiYnKSzgZ+Arzd9sPNsZWAtwNPGLdI/Mj1IgFMJtUoI2KYJN1ie+vZnhuVvjQBPSLVKCOiJao34/aiE1jSUyS9VdJNlHH/P6apRmm79fMAmklsEdE9X5P0qEWPJL0J+HqFeJaMow9NQF2vRtn2scQRMTFJ6wIfBp4BjFUGfRbwXeDltn9ZMbzeNAF1vRpll2KNiEZzgX9JsybGMyjf5bfZvrluZEUvngDGdLUapaSfA/8x2XnbJ4wwnIhYQfQqAQzqUjVKSbcDJ0523vZZIwwnIlYQvU0AXZI+gIgYhl6MAloBPDzZCUmbjjKQiFg2zfokrZAngI6TdIft1tQWiYiptemJPk8A3ZcRQhHd0prvbBJA9+URLqJbXlw7gDF9mQfQaZJOY+ILvYB1RxxORCwD27fXjmFMEkA3zF/KcxERk0oncERET+UJoAMkXcySTUAG7gKusD3pDOGIqEvSNcBXgK8CV9leUDmkJeQJoAMk7T7B4fUp6xvfYvtNIw4pImagKT//nIGfVYCrKAnhq7avqRheEkCXSVoZuMb2DrVjiYjpSXo8pQTN64Etba9cM540AXWY7YfLapcR0UbNcrTbU+7+dwWeAvwfcA5QfUnXJIAOaArXjbcecBRw44jDiYiZ+zVwM3A6ZV3gWyrHs4Q0AXWApB9SOn7HbvcN3A1cCbzT9q8rhRYRU5B0JPCHwDzgfuAblDv/q23/X83YIAmg8yStavvB2nFExNQkrQXsQmkOOhJYyfaTasaUJqAOatoV96Asbr8f8Pi6EUXEZCStDuzE4n6AnSn9AFfVjAvyBNApknamXPRfQhkGehxwke1fVA0sIiYk6ZvAkyjrAX+VctG/2vavqgbWSALoAEknAS8F7gA+DlwIzLe9ZdXAImJKknYCvmX7odqxTCQJoAMkLaSMJDgV+Kzt+yXdZnuryqFFxBQknQn8AWW03iXA/9heWDeqxZIAOqCZ8LU3ixezvwL4Y2Dztt5ZRMRikrYF9gWeD6wOXE5JCF+zvahaXEkA3dJ0KL2Q0hewK3CZ7cPrRhURMyVpTWAvSkLYuebqYEkAHSZpbeAA2+fWjiUiJidpG+Bpze5Ntr9XM54xGQbaAZL2A24YW0hC0luBA4HbgdfVjC0iJifpccCngCcD36ZM5txW0veBA23fWzW+PAG0n6QbgF1s3yfphcAplP6AecDBtp9fNcCImJCkUykz9//K9sPNsVWA9wCr2K56A5cE0AGSrre9fbN9NnCz7fc0+9+q2YYYEZOT9F1gu/GDNSStBlxve5s6kRVZFL4bJGktSStROo8uGzi3eqWYImJ6D0w0Us/2A8DvK8SzhPQBdMOpwHWUyoI32Z4PIGke8NOagUXElFaX9EwWF3IcI+AxFeJZMog0AXWDpM2AjSiPjYuaYxsDq9m+o2pwETEhSV+e6rzt544qlokkAXSApCcCvxyrHyJpD+DFlFFA/9I8TkZEzEr6ALrhAmBNeGSN0f+i1AXaHvhQxbgiosPSB9ANa9i+s9l+GXC27fc1ncLXVYwrIjosTwDdMNiBtCfNKKCaNUQiYnrNTVprtTq4eMTlki6Q9H7KWsCXA0jaBEj7f0R7fbMpCd1K6QTugGYFsEOATYALbP+kOT4PmGP70prxRcTEJD0HOI2yFvCbbf+yckhLSALoMEm7AYfZPq52LBExsaYZ6DhK3a7PAo803do+oVZckE7gzmlGAR1OWSHsh8An60YUEdNYB3gmcA9lYZjW9N0lAXSApKcAh1IKwN0NnE95etujamARMSVJrwLeDPwz8OdtG7iRBNAN3wO+DOxn+1YASW+oG1JEzMCfALvZbmXJlowC6oYDgZ8BV0j6d0l78ejaIhHRMrYPmejiL+nJkk6vEdOgJIAOsH2h7UMoKwpdCbwBeLyk0yXtXTW4iJiUpG0lfV7SdZLeLmmOpPOBLwG3VY8vo4C6SdL6wMHAIbb3rB1PRDyapK8BZwJXA/sAr6eUcnmL7d/VjA2SACIihkbSdbZ3GNj/MbDlRGsE1JBO4IiI4Rm/HsBvgG2ayZ3YvqFaZOQJICJiaCR9hbIm8ERs+3mjjGe8JICIiJ7KKKCIiCGR9MaB7QPGnXvn6CNaUhJARMTwHDGw/Xfjzv3pKAOZSBJARMTwaJLtifZHLgkgImJ4PMn2RPsjl07giIghkfQw8CvK3f7awK/HTgFr2V61VmyQeQAREcO0BvBw7SAmkyagiIjh+Zrthyf7qR1cEkBExPBU7+idSpqAIiKGZ46k4yc7afsDowxmvCSAiIjhWRnYkJY+CWQUUETEkEi61va82nFMJn0AERHD0+prbKuDi4jouJVrBzCVJICIiOF5oHYAU0kncETE8GyUUUAREf2UUUAREX0k6Vu2n1U7jsmkDyAiYnhaeec/JgkgImJ49h5/QNIakg6V9JkaAQ1KAoiIGBLbCwEkrSLphZI+BvyUshrYOTVjg/QBREQMjaQ9gMOAFwBfAc4HTrX9xKqBNZIAIiKGRNIi4MvA0bZ/1By7zfZWVQNrZBhoRMTw7AwcClwp6SbgPFo0OzhPABERQyZJwPMozUEvAb4BXGj77KpxJQFERIyOpFWA5wOH2j6yaixJABERwyNpJduLmu1NgZ2AH9j+dt3IMgw0ImJoJL0SuEvSHc32F4GXAZ+S9Fd1o8sTQETE0Ej6DrA7sDZwI7CF7YWS1gS+YfsZNePLKKCIiOF50PbdwN2Sbh2bGGb7t5Kql4pOAoiIGJ41JD2T0ty+WrOt5mf1qpGRJqCIiKGR9OWpztt+7qhimUgSQERET2UUUERETyUBRET0VBJARERPJQFERIyIpD+R9Jhme//q8aQTOCJiNCT9G7A9pRjc82zvUDOePAFERAyJpB0lbTi2b/vPgIuBI4F3VwuskQQQETE8ZwG/GduR9E+UYnBPA/6iVlBjMhM4ImJ4VrF9v6SVgbOBh4EDbD/c1AOqKgkgImJ4rpZ0KbAxsD6l3f9hSbsBv6sbWjqBIyKGStIfAQ8APwU+QakM+hjgQNvzK4aWBBARMUqSNgF+5hZcfJMAIiJ6KqOAIiJ6KgkgImJIJG1WO4apJAFERAzPxbUDmEoSQETE8Kh2AFNJJ3BExJBI+jnwH5Odt33CCMN5lEwEi4gYnt8BN9YOYjJ5AoiIGBJJ37L9rNpxTCZ9ABERw/PwZCckbTrKQCaMIU8AERGjJ+kO20+oGUOeACIi6qg+QigJICKijurNLxkFFBExJJJOY+ILvYB1RxzOoyQBREQMz1TlnquWgoZ0AkdE9FaeACIihkTSxSzZBGTgLuAK25POEB6VPAFERAyJpN0nOLw+8DLgFttvGnFIS0gCiIgYsWaR+Gts71AzjgwDjYgYMduTzhAepfQBREQMiaT1Jzi8HnAULSgSlwQQETE811A6fsdm/Rq4G7gSeE2lmB6RPoCIiAokrWr7wZoxpA8gImJEVOwp6UxgQe14kgAiIoZM0s6S3g/cDlwEfBl4Wt2o0gQUETE0kk4CXgrcAXwcuBCYb3vLqoE10gkcETE8xwI3A6cDn7V9v6TW3HWnCSgiYng2Bk4CXgTcKumjwBqSWnHznSagiIgRkLQ68ELgcGBX4DLbh1eNKQkgImK0JK0NHGD73JpxpAkoImJIJO0n6YkD+2+VdD3wn8CX6kVWJAFERAzPScBCAEkvpFQBfSVlKOi/VowLSAKIiBgm276v2T4AOMv2NbbPBOZUjAtIAoiIGCZJWkvSSsBewGUD51avFNMjWjEUKSJiBXUqcB3wa+Am2/MBJM0DflozMMgooIiIoZK0GbARcL3tRc2xjYHVbN9RNbYkgIiI4WhGAP3S9q+a/T2AF1NqAv2L7Qdqxpc+gIiI4bkAWBNA0g7Af1HqAm0PfKhiXED6ACIihmkN23c22y8Dzrb9vqZT+LqKcQF5AoiIGCYNbO9JMwporC+gtjwBREQMz+WSLqCM+FkPuBxA0iZA1fZ/SCdwRMTQSBJwCLAJcIHtnzTH5wFzbF9aM748AUREDInLHfZ5E5xaE9gfSAKIiFjRNaOADqesEPZD4JN1I0oCiIgYGrmzkDsAAACHSURBVElPAQ4FDgPuBs6nNL3vUTWwRvoAIiKGRNIiygLwx9i+tTl2m+2t6kZWZBhoRMTwHAj8DLhC0r9L2oslh4ZWlSeAiIghk7QmpQTEYZT5AOcCF9YeBZQEEBExQpLWBw4GDrG9Z9VYkgAiIvopfQARET2VBBAR0VNJABERPZUEEBHRU/8fTPTDXjqrGoMAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Create a simple bar chart for offense categories\n", "p = offense_count_df.plot.bar(y = \"count\")" ] }, { "cell_type": "code", "execution_count": 59, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"7834e29e-27ab-4752-beb1-e91b38d1fc0e\":{\"roots\":{\"references\":[{\"attributes\":{\"callback\":null,\"end\":4753},\"id\":\"b1ac6567-e5c7-492e-a84c-9569a88ff3c9\",\"type\":\"Range1d\"},{\"attributes\":{},\"id\":\"701085e9-9690-4abf-aa82-2df3a4cd7f3f\",\"type\":\"HelpTool\"},{\"attributes\":{},\"id\":\"f7a325e8-31b3-4f2c-8b98-cb4cdd491d63\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"b21294fb-0710-4c74-8fe3-0970226ae804\",\"type\":\"CategoricalScale\"},{\"attributes\":{},\"id\":\"a680085b-0b3f-4ae3-95e9-715364bdda89\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"plot\":{\"id\":\"62683021-9c5d-4b20-abd2-08649c16e7a1\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9b9e0c69-1b48-421d-8ad9-c765ad0a273d\",\"type\":\"CategoricalTicker\"}},\"id\":\"71ec2925-286c-4296-b08d-55cdb9fceb4a\",\"type\":\"Grid\"},{\"attributes\":{\"callback\":null,\"data\":{\"top\":[4321,1464,814,227,20,2,1],\"x\":[\"ASSAULT SIMPLE - BATTERY\",\"ASSAULT - AGGRAVATED - BATTERY\",\"ASSAULT - AGGRAVATED\",\"ASSAULT - SIMPLE\",\"STALKING\",\"A&B ON POLICE OFFICER\",\"ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER\"]},\"selected\":{\"id\":\"0b27214c-d4dc-4df8-8f39-5569860174dc\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"a680085b-0b3f-4ae3-95e9-715364bdda89\",\"type\":\"UnionRenderers\"}},\"id\":\"c11d51b4-a696-4fc6-ac0a-580eba449a15\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"formatter\":{\"id\":\"1def2689-a882-423a-8550-9728dbd2a859\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"62683021-9c5d-4b20-abd2-08649c16e7a1\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"7a0948ca-2dc7-4857-8e66-1ad61780262a\",\"type\":\"BasicTicker\"}},\"id\":\"0d41cca1-de72-4e7a-9476-8304a37b9ce7\",\"type\":\"LinearAxis\"},{\"attributes\":{\"formatter\":{\"id\":\"9596ee9c-8c74-4493-96c1-4a06fdf383a1\",\"type\":\"CategoricalTickFormatter\"},\"major_label_orientation\":0.5235987755982988,\"plot\":{\"id\":\"62683021-9c5d-4b20-abd2-08649c16e7a1\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9b9e0c69-1b48-421d-8ad9-c765ad0a273d\",\"type\":\"CategoricalTicker\"}},\"id\":\"9a2b1ebf-25f0-4841-a71c-ce62d48c1035\",\"type\":\"CategoricalAxis\"},{\"attributes\":{},\"id\":\"c7920b84-09b5-4bca-9ba0-87f80fff65db\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1def2689-a882-423a-8550-9728dbd2a859\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"4981e556-f4ef-452d-b437-e206ec48ad31\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"aabca3b1-f046-49fb-a377-dde0a440a446\",\"type\":\"LinearScale\"},{\"attributes\":{\"overlay\":{\"id\":\"6397f337-9729-4f8d-a829-dbc38fcce05c\",\"type\":\"BoxAnnotation\"}},\"id\":\"8176753b-cb41-433a-a859-b316c27bece7\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"92ca1110-6467-4745-a96a-1e45eb81a28e\",\"type\":\"SaveTool\"},{\"attributes\":{\"data_source\":{\"id\":\"c11d51b4-a696-4fc6-ac0a-580eba449a15\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"295463d1-35f6-494d-b0f0-4b4d1e71f0b3\",\"type\":\"VBar\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"f686b0b8-e754-4dc9-9969-67ffc678b454\",\"type\":\"VBar\"},\"selection_glyph\":null,\"view\":{\"id\":\"491c05ee-87f2-4900-9a6a-a4070c40e6d3\",\"type\":\"CDSView\"}},\"id\":\"0f97b2f1-7acd-4dfd-8ba1-0e8dd1175c81\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"plot\":null,\"text\":\"Assaults by Categor\"},\"id\":\"98b6d8e6-3c56-4a36-8c09-a108f08e23df\",\"type\":\"Title\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"c7920b84-09b5-4bca-9ba0-87f80fff65db\",\"type\":\"PanTool\"},{\"id\":\"4981e556-f4ef-452d-b437-e206ec48ad31\",\"type\":\"WheelZoomTool\"},{\"id\":\"8176753b-cb41-433a-a859-b316c27bece7\",\"type\":\"BoxZoomTool\"},{\"id\":\"92ca1110-6467-4745-a96a-1e45eb81a28e\",\"type\":\"SaveTool\"},{\"id\":\"f7a325e8-31b3-4f2c-8b98-cb4cdd491d63\",\"type\":\"ResetTool\"},{\"id\":\"701085e9-9690-4abf-aa82-2df3a4cd7f3f\",\"type\":\"HelpTool\"}]},\"id\":\"ae7ab4ce-267f-420d-b17f-19263801b1ca\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"7a0948ca-2dc7-4857-8e66-1ad61780262a\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"9b9e0c69-1b48-421d-8ad9-c765ad0a273d\",\"type\":\"CategoricalTicker\"},{\"attributes\":{},\"id\":\"0b27214c-d4dc-4df8-8f39-5569860174dc\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.9},\"x\":{\"field\":\"x\"}},\"id\":\"f686b0b8-e754-4dc9-9969-67ffc678b454\",\"type\":\"VBar\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"62683021-9c5d-4b20-abd2-08649c16e7a1\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"7a0948ca-2dc7-4857-8e66-1ad61780262a\",\"type\":\"BasicTicker\"}},\"id\":\"7975ff28-6694-4ed9-b354-c0c45eba8a42\",\"type\":\"Grid\"},{\"attributes\":{\"below\":[{\"id\":\"9a2b1ebf-25f0-4841-a71c-ce62d48c1035\",\"type\":\"CategoricalAxis\"}],\"left\":[{\"id\":\"0d41cca1-de72-4e7a-9476-8304a37b9ce7\",\"type\":\"LinearAxis\"}],\"min_border_left\":180,\"plot_height\":400,\"renderers\":[{\"id\":\"9a2b1ebf-25f0-4841-a71c-ce62d48c1035\",\"type\":\"CategoricalAxis\"},{\"id\":\"71ec2925-286c-4296-b08d-55cdb9fceb4a\",\"type\":\"Grid\"},{\"id\":\"0d41cca1-de72-4e7a-9476-8304a37b9ce7\",\"type\":\"LinearAxis\"},{\"id\":\"7975ff28-6694-4ed9-b354-c0c45eba8a42\",\"type\":\"Grid\"},{\"id\":\"6397f337-9729-4f8d-a829-dbc38fcce05c\",\"type\":\"BoxAnnotation\"},{\"id\":\"0f97b2f1-7acd-4dfd-8ba1-0e8dd1175c81\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"98b6d8e6-3c56-4a36-8c09-a108f08e23df\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"ae7ab4ce-267f-420d-b17f-19263801b1ca\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"a6731ea0-e20f-4664-a373-43f200fafda9\",\"type\":\"FactorRange\"},\"x_scale\":{\"id\":\"b21294fb-0710-4c74-8fe3-0970226ae804\",\"type\":\"CategoricalScale\"},\"y_range\":{\"id\":\"b1ac6567-e5c7-492e-a84c-9569a88ff3c9\",\"type\":\"Range1d\"},\"y_scale\":{\"id\":\"aabca3b1-f046-49fb-a377-dde0a440a446\",\"type\":\"LinearScale\"}},\"id\":\"62683021-9c5d-4b20-abd2-08649c16e7a1\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_color\":{\"value\":\"#1f77b4\"},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.9},\"x\":{\"field\":\"x\"}},\"id\":\"295463d1-35f6-494d-b0f0-4b4d1e71f0b3\",\"type\":\"VBar\"},{\"attributes\":{},\"id\":\"9596ee9c-8c74-4493-96c1-4a06fdf383a1\",\"type\":\"CategoricalTickFormatter\"},{\"attributes\":{\"callback\":null,\"factors\":[\"ASSAULT SIMPLE - BATTERY\",\"ASSAULT - AGGRAVATED - BATTERY\",\"ASSAULT - AGGRAVATED\",\"ASSAULT - SIMPLE\",\"STALKING\",\"A&B ON POLICE OFFICER\",\"ASSAULT & BATTERY D/W - OTHER ON POLICE OFFICER\"]},\"id\":\"a6731ea0-e20f-4664-a373-43f200fafda9\",\"type\":\"FactorRange\"},{\"attributes\":{\"source\":{\"id\":\"c11d51b4-a696-4fc6-ac0a-580eba449a15\",\"type\":\"ColumnDataSource\"}},\"id\":\"491c05ee-87f2-4900-9a6a-a4070c40e6d3\",\"type\":\"CDSView\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"6397f337-9729-4f8d-a829-dbc38fcce05c\",\"type\":\"BoxAnnotation\"}],\"root_ids\":[\"62683021-9c5d-4b20-abd2-08649c16e7a1\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"7834e29e-27ab-4752-beb1-e91b38d1fc0e\",\"roots\":{\"62683021-9c5d-4b20-abd2-08649c16e7a1\":\"514211b2-52bf-4191-8656-e376b62b7f9a\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "62683021-9c5d-4b20-abd2-08649c16e7a1" } }, "output_type": "display_data" } ], "source": [ "# Create an interactive bar chart with bokeh library\n", "p = figure(plot_width=600, plot_height=400, title=\"Assaults by Categor\", \n", " x_range=group.index.values, y_range=(0, int(max(group.values) * 1.1)))\n", "p.vbar(x = group.index, top=group.values, width=0.9)\n", "# pad the plot\n", "p.min_border_left = 180\n", "\n", "p.xaxis.major_label_orientation = pi/6\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From this chart we can see the different offense descriptions and the number of incidents in the year 2017." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How do the assaults change by month over the year?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The crime incidents have a temporal component. They may show trends that change month by month over the year. We can use a pandas grouper with the month frequency to count the number of crimes per month. This is then used to create a plot showing the yearly trend of crime incidents." ] }, { "cell_type": "code", "execution_count": 60, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "DateTime\n", "2017-05-31 620\n", "2017-08-31 616\n", "2017-06-30 610\n", "2017-09-30 607\n", "2017-10-31 604\n", "2017-07-31 604\n", "2017-11-30 562\n", "2017-12-31 561\n", "2017-04-30 556\n", "2017-01-31 531\n", "2017-03-31 510\n", "2017-02-28 468\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Group by month and count the number of incidents\n", "group = crime_df_time_index.groupby(pd.Grouper(freq='M')).count()['OBJECTID']\n", "group.sort_values(ascending=False)" ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Feb had the lowest number of crime incidents and they peak in May.\n" ] } ], "source": [ "# look at the trend over the course of the year.\n", "import calendar\n", "month_frequency = pd.DatetimeIndex(group.sort_values(ascending=False).index.values)\n", "print(\"{} had the lowest number of crime incidents and they peak in {}.\".format(calendar.month_abbr[month_frequency.month[-1]], calendar.month_abbr[month_frequency.month[0]]))" ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"bc569d44-d0c0-4e11-9b5f-2c88fcbc6283\":{\"roots\":{\"references\":[{\"attributes\":{\"days\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]},\"id\":\"da87423f-2522-4ba7-914d-2255795b738e\",\"type\":\"DaysTicker\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"5c446844-1865-4fce-b72a-a2f5b0507e48\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9a90b59b-fb86-4e0e-89c1-3f92300eb69a\",\"type\":\"BasicTicker\"}},\"id\":\"3bf2988f-6424-44ea-a1d6-fc49d365fb85\",\"type\":\"Grid\"},{\"attributes\":{\"days\":[1,4,7,10,13,16,19,22,25,28]},\"id\":\"25fe31a6-f251-45d4-b9b8-518e7a1eacad\",\"type\":\"DaysTicker\"},{\"attributes\":{\"days\":[1,8,15,22]},\"id\":\"edc2abe6-d0a2-4592-9e2b-d90fe974d4e2\",\"type\":\"DaysTicker\"},{\"attributes\":{},\"id\":\"a87a6d91-7ebe-46d9-acc8-249f0fef61c2\",\"type\":\"DatetimeTickFormatter\"},{\"attributes\":{\"days\":[1,15]},\"id\":\"ee905b86-9e51-431b-9fb7-c84861aeddfa\",\"type\":\"DaysTicker\"},{\"attributes\":{},\"id\":\"6f4aa61b-cb51-4c37-9285-3586a3abe95f\",\"type\":\"LinearScale\"},{\"attributes\":{\"months\":[0,1,2,3,4,5,6,7,8,9,10,11]},\"id\":\"0cb931d9-7a82-4ad2-9908-90ec1e193c7d\",\"type\":\"MonthsTicker\"},{\"attributes\":{},\"id\":\"5242bc17-a323-4e5d-b385-4a2bcf73a7ab\",\"type\":\"LinearScale\"},{\"attributes\":{\"months\":[0,2,4,6,8,10]},\"id\":\"1d1c414b-14ec-456c-af70-d54313a9621a\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"plot\":{\"id\":\"5c446844-1865-4fce-b72a-a2f5b0507e48\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"42ce30e6-7ff9-413e-8f38-ec52936b9d77\",\"type\":\"DatetimeTicker\"}},\"id\":\"938d6c73-a9e3-4472-81d6-2e632d68849f\",\"type\":\"Grid\"},{\"attributes\":{\"months\":[0,4,8]},\"id\":\"b31658f5-b242-4cc9-8802-1a8120747722\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"formatter\":{\"id\":\"a87a6d91-7ebe-46d9-acc8-249f0fef61c2\",\"type\":\"DatetimeTickFormatter\"},\"plot\":{\"id\":\"5c446844-1865-4fce-b72a-a2f5b0507e48\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"42ce30e6-7ff9-413e-8f38-ec52936b9d77\",\"type\":\"DatetimeTicker\"}},\"id\":\"1554d009-d270-4b1a-88bc-a5c0f3444e21\",\"type\":\"DatetimeAxis\"},{\"attributes\":{\"months\":[0,6]},\"id\":\"3ef2b85b-4dfb-4e81-8cb1-2907861b5477\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"num_minor_ticks\":5,\"tickers\":[{\"id\":\"40be76fa-5e3e-4805-a8dc-2b6b424adaf1\",\"type\":\"AdaptiveTicker\"},{\"id\":\"c6ff38e8-b4a6-4454-b157-4b99a0232bd0\",\"type\":\"AdaptiveTicker\"},{\"id\":\"836f6056-8e32-4d0b-85ab-f4e253caaeb1\",\"type\":\"AdaptiveTicker\"},{\"id\":\"da87423f-2522-4ba7-914d-2255795b738e\",\"type\":\"DaysTicker\"},{\"id\":\"25fe31a6-f251-45d4-b9b8-518e7a1eacad\",\"type\":\"DaysTicker\"},{\"id\":\"edc2abe6-d0a2-4592-9e2b-d90fe974d4e2\",\"type\":\"DaysTicker\"},{\"id\":\"ee905b86-9e51-431b-9fb7-c84861aeddfa\",\"type\":\"DaysTicker\"},{\"id\":\"0cb931d9-7a82-4ad2-9908-90ec1e193c7d\",\"type\":\"MonthsTicker\"},{\"id\":\"1d1c414b-14ec-456c-af70-d54313a9621a\",\"type\":\"MonthsTicker\"},{\"id\":\"b31658f5-b242-4cc9-8802-1a8120747722\",\"type\":\"MonthsTicker\"},{\"id\":\"3ef2b85b-4dfb-4e81-8cb1-2907861b5477\",\"type\":\"MonthsTicker\"},{\"id\":\"fdec31ca-eee2-4c18-a34d-1c8a69e4cf67\",\"type\":\"YearsTicker\"}]},\"id\":\"42ce30e6-7ff9-413e-8f38-ec52936b9d77\",\"type\":\"DatetimeTicker\"},{\"attributes\":{},\"id\":\"fdec31ca-eee2-4c18-a34d-1c8a69e4cf67\",\"type\":\"YearsTicker\"},{\"attributes\":{\"formatter\":{\"id\":\"cd472478-aeef-4b13-9c32-0b340a4cf3c1\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"5c446844-1865-4fce-b72a-a2f5b0507e48\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9a90b59b-fb86-4e0e-89c1-3f92300eb69a\",\"type\":\"BasicTicker\"}},\"id\":\"8cc51e69-a174-46fc-8699-52461477a101\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"1204830e-7c8b-45bf-8403-d0d8f12b86c9\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"be4d9f51-1308-4502-bbeb-0fa714c9dcae\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"9e259519-e28b-4413-929a-c472644166f3\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"ecc2e5c4-99ed-4ca4-a92a-f8042731c65d\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"b5dc74dd-c057-45d8-8655-287ad40e479f\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"043348c8-ce9e-4f11-8efa-4d1974762653\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"6eeb553b-c73e-418d-826f-3e466fb56702\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"55419c66-ce5a-4a69-94a0-8e9b64b3ab98\",\"type\":\"PanTool\"},{\"attributes\":{},\"id\":\"1c636ddc-8dea-4782-87d1-8b4272b16a7d\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"overlay\":{\"id\":\"6eeb553b-c73e-418d-826f-3e466fb56702\",\"type\":\"BoxAnnotation\"}},\"id\":\"ea6b18b5-0765-4096-a108-76e784102d0c\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"3674c781-95cb-4b1c-ada8-19d69caceef9\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"f1d705bf-0ee1-49cf-9a4d-64a720eca5e2\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"ba5244c1-1e5f-4269-865c-2c92f644f427\",\"type\":\"HelpTool\"},{\"attributes\":{\"data_source\":{\"id\":\"3c025885-799e-44f3-be08-3a39b31ca93a\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"be4d9f51-1308-4502-bbeb-0fa714c9dcae\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"1a55b212-ee84-4654-9bf1-6c3a6b4bc03f\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"2e5f6a5e-a573-4abe-b2ad-54fc66892d6e\",\"type\":\"CDSView\"}},\"id\":\"d548f63b-caa5-49cc-9514-0383989144e1\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1a55b212-ee84-4654-9bf1-6c3a6b4bc03f\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"3c025885-799e-44f3-be08-3a39b31ca93a\",\"type\":\"ColumnDataSource\"}},\"id\":\"2e5f6a5e-a573-4abe-b2ad-54fc66892d6e\",\"type\":\"CDSView\"},{\"attributes\":{\"data_source\":{\"id\":\"0e92ddaa-a09b-4d90-ab40-3269de857fcc\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"4eafc383-6923-42e8-87ed-d2345f98097c\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"b5dc74dd-c057-45d8-8655-287ad40e479f\",\"type\":\"Circle\"},\"selection_glyph\":null,\"view\":{\"id\":\"e99fe9ca-bd65-4977-8853-252ce52b4c62\",\"type\":\"CDSView\"}},\"id\":\"ec2d363e-eb46-4668-8fd8-869399124497\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"below\":[{\"id\":\"1554d009-d270-4b1a-88bc-a5c0f3444e21\",\"type\":\"DatetimeAxis\"}],\"left\":[{\"id\":\"8cc51e69-a174-46fc-8699-52461477a101\",\"type\":\"LinearAxis\"}],\"plot_height\":400,\"renderers\":[{\"id\":\"1554d009-d270-4b1a-88bc-a5c0f3444e21\",\"type\":\"DatetimeAxis\"},{\"id\":\"938d6c73-a9e3-4472-81d6-2e632d68849f\",\"type\":\"Grid\"},{\"id\":\"8cc51e69-a174-46fc-8699-52461477a101\",\"type\":\"LinearAxis\"},{\"id\":\"3bf2988f-6424-44ea-a1d6-fc49d365fb85\",\"type\":\"Grid\"},{\"id\":\"6eeb553b-c73e-418d-826f-3e466fb56702\",\"type\":\"BoxAnnotation\"},{\"id\":\"d548f63b-caa5-49cc-9514-0383989144e1\",\"type\":\"GlyphRenderer\"},{\"id\":\"ec2d363e-eb46-4668-8fd8-869399124497\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"2aa759ad-e7d4-4f80-8ab8-4082b10bccc2\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"1068af8b-469c-41da-8557-ec38f04aeced\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"abc30c22-cc35-424d-8d53-517a39ff7632\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"6f4aa61b-cb51-4c37-9285-3586a3abe95f\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"d55a6df4-fdf3-46b4-84d5-784eeb3e7ca3\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"5242bc17-a323-4e5d-b385-4a2bcf73a7ab\",\"type\":\"LinearScale\"}},\"id\":\"5c446844-1865-4fce-b72a-a2f5b0507e48\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":{\"__ndarray__\":\"AADAIh2fdUIAAMBDIKh1QgAAAJYasnVCAACAgsK7dUIAAMDUvMV1QgAAQMFkz3VCAACAE1/ZdUIAAMBlWeN1QgAAQFIB7XVCAACApPv2dUIAAACRowB2QgAAQOOdCnZC\",\"dtype\":\"float64\",\"shape\":[12]},\"y\":[531,468,510,556,620,610,604,616,607,604,562,561]},\"selected\":{\"id\":\"1204830e-7c8b-45bf-8403-d0d8f12b86c9\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"9e259519-e28b-4413-929a-c472644166f3\",\"type\":\"UnionRenderers\"}},\"id\":\"3c025885-799e-44f3-be08-3a39b31ca93a\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"0e92ddaa-a09b-4d90-ab40-3269de857fcc\",\"type\":\"ColumnDataSource\"}},\"id\":\"e99fe9ca-bd65-4977-8853-252ce52b4c62\",\"type\":\"CDSView\"},{\"attributes\":{\"callback\":null},\"id\":\"abc30c22-cc35-424d-8d53-517a39ff7632\",\"type\":\"DataRange1d\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":{\"__ndarray__\":\"AADAIh2fdUIAAMBDIKh1QgAAAJYasnVCAACAgsK7dUIAAMDUvMV1QgAAQMFkz3VCAACAE1/ZdUIAAMBlWeN1QgAAQFIB7XVCAACApPv2dUIAAACRowB2QgAAQOOdCnZC\",\"dtype\":\"float64\",\"shape\":[12]},\"y\":[531,468,510,556,620,610,604,616,607,604,562,561]},\"selected\":{\"id\":\"ecc2e5c4-99ed-4ca4-a92a-f8042731c65d\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"043348c8-ce9e-4f11-8efa-4d1974762653\",\"type\":\"UnionRenderers\"}},\"id\":\"0e92ddaa-a09b-4d90-ab40-3269de857fcc\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"plot\":null,\"text\":\"Assaults by Month\"},\"id\":\"2aa759ad-e7d4-4f80-8ab8-4082b10bccc2\",\"type\":\"Title\"},{\"attributes\":{\"mantissas\":[1,2,5],\"max_interval\":500.0,\"num_minor_ticks\":0},\"id\":\"40be76fa-5e3e-4805-a8dc-2b6b424adaf1\",\"type\":\"AdaptiveTicker\"},{\"attributes\":{},\"id\":\"cd472478-aeef-4b13-9c32-0b340a4cf3c1\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"base\":60,\"mantissas\":[1,2,5,10,15,20,30],\"max_interval\":1800000.0,\"min_interval\":1000.0,\"num_minor_ticks\":0},\"id\":\"c6ff38e8-b4a6-4454-b157-4b99a0232bd0\",\"type\":\"AdaptiveTicker\"},{\"attributes\":{},\"id\":\"9a90b59b-fb86-4e0e-89c1-3f92300eb69a\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"#1f77b4\"},\"size\":{\"units\":\"screen\",\"value\":8},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4eafc383-6923-42e8-87ed-d2345f98097c\",\"type\":\"Circle\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"55419c66-ce5a-4a69-94a0-8e9b64b3ab98\",\"type\":\"PanTool\"},{\"id\":\"1c636ddc-8dea-4782-87d1-8b4272b16a7d\",\"type\":\"WheelZoomTool\"},{\"id\":\"ea6b18b5-0765-4096-a108-76e784102d0c\",\"type\":\"BoxZoomTool\"},{\"id\":\"3674c781-95cb-4b1c-ada8-19d69caceef9\",\"type\":\"SaveTool\"},{\"id\":\"f1d705bf-0ee1-49cf-9a4d-64a720eca5e2\",\"type\":\"ResetTool\"},{\"id\":\"ba5244c1-1e5f-4269-865c-2c92f644f427\",\"type\":\"HelpTool\"}]},\"id\":\"1068af8b-469c-41da-8557-ec38f04aeced\",\"type\":\"Toolbar\"},{\"attributes\":{\"callback\":null},\"id\":\"d55a6df4-fdf3-46b4-84d5-784eeb3e7ca3\",\"type\":\"DataRange1d\"},{\"attributes\":{\"base\":24,\"mantissas\":[1,2,4,6,8,12],\"max_interval\":43200000.0,\"min_interval\":3600000.0,\"num_minor_ticks\":0},\"id\":\"836f6056-8e32-4d0b-85ab-f4e253caaeb1\",\"type\":\"AdaptiveTicker\"}],\"root_ids\":[\"5c446844-1865-4fce-b72a-a2f5b0507e48\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"bc569d44-d0c0-4e11-9b5f-2c88fcbc6283\",\"roots\":{\"5c446844-1865-4fce-b72a-a2f5b0507e48\":\"4ae8ecb8-e429-43bd-ab19-2eac27b28c45\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "5c446844-1865-4fce-b72a-a2f5b0507e48" } }, "output_type": "display_data" } ], "source": [ "# Create the plot\n", "p = figure(plot_width=600, plot_height=400, title=\"Assaults by Month\", x_axis_type='datetime')\n", "p.line(group.index, group, line_width=2)\n", "p.circle(group.index, group, alpha=0.5, size=8)\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How do incidents vary by day of week ?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Another way to look at the crime incidents is by day of week. Are there any trends between weekends or weekdays?" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "DAY_OF_WEEK\n", "Sunday 1126\n", "Saturday 1073\n", "Friday 976\n", "Thursday 942\n", "Wednesday 939\n", "Tuesday 921\n", "Monday 872\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Group by the day of the week\n", "group = crime_df_time_index.groupby('DAY_OF_WEEK').count()['OBJECTID']\n", "group.sort_values(ascending=False)" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Sunday and Saturday have more crime incidents than other weekdays during this time period.\n" ] } ], "source": [ "weekdays_frequency = group.sort_values(ascending=False).index.values\n", "print(\"{} and {} have more crime incidents than other weekdays during this time period.\".format(weekdays_frequency[0], weekdays_frequency[1]))" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"d9dbe675-dfc1-40e5-9ecc-23e7635cc494\":{\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"c880d9c3-95db-4734-bedc-d3235ea9daae\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"formatter\":{\"id\":\"1c03f874-9a63-4981-ac64-21b7dae6bc91\",\"type\":\"CategoricalTickFormatter\"},\"plot\":{\"id\":\"8863a464-064f-4482-b3c7-e4c832180f8d\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"75bd2ee1-144a-4bca-8175-c2c586d80d4a\",\"type\":\"CategoricalTicker\"}},\"id\":\"e6ba16b1-b849-4836-8fe1-3a61f9a05116\",\"type\":\"CategoricalAxis\"},{\"attributes\":{},\"id\":\"c0dfce3d-eedd-491f-96a6-9f9a5fb69973\",\"type\":\"LinearScale\"},{\"attributes\":{\"below\":[{\"id\":\"e6ba16b1-b849-4836-8fe1-3a61f9a05116\",\"type\":\"CategoricalAxis\"}],\"left\":[{\"id\":\"d465acbc-5f09-44fc-b453-695dc95d9df0\",\"type\":\"LinearAxis\"}],\"plot_height\":250,\"renderers\":[{\"id\":\"e6ba16b1-b849-4836-8fe1-3a61f9a05116\",\"type\":\"CategoricalAxis\"},{\"id\":\"0cb7015c-3c0d-4363-a78e-7bd9a52899f8\",\"type\":\"Grid\"},{\"id\":\"d465acbc-5f09-44fc-b453-695dc95d9df0\",\"type\":\"LinearAxis\"},{\"id\":\"afbb1837-3e84-47f4-8d5e-4872b807b60c\",\"type\":\"Grid\"},{\"id\":\"3af708a2-f808-46fb-aeeb-b3bfde874967\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"1d1a99e0-b135-4571-8f2a-145df418694a\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"aada079c-d303-4db9-b08d-755ad9d61f8e\",\"type\":\"Toolbar\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"3448d3ae-250f-42d1-93f8-c80a76337ecd\",\"type\":\"FactorRange\"},\"x_scale\":{\"id\":\"e8dc41b1-19ee-47ee-b7e4-f9cc4e97e184\",\"type\":\"CategoricalScale\"},\"y_range\":{\"id\":\"5cca3a39-1bbd-4636-af8c-87eca50723f2\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"c0dfce3d-eedd-491f-96a6-9f9a5fb69973\",\"type\":\"LinearScale\"}},\"id\":\"8863a464-064f-4482-b3c7-e4c832180f8d\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{},\"id\":\"75bd2ee1-144a-4bca-8175-c2c586d80d4a\",\"type\":\"CategoricalTicker\"},{\"attributes\":{\"callback\":null,\"data\":{\"top\":[976,872,1073,1126,942,921,939],\"x\":[\"Friday\",\"Monday\",\"Saturday\",\"Sunday\",\"Thursday\",\"Tuesday\",\"Wednesday\"]},\"selected\":{\"id\":\"d30fde13-2b1e-4a15-ae83-1733b8b2973c\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"c880d9c3-95db-4734-bedc-d3235ea9daae\",\"type\":\"UnionRenderers\"}},\"id\":\"35a70a48-2c99-4008-8972-baccf21d5a7a\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"plot\":{\"id\":\"8863a464-064f-4482-b3c7-e4c832180f8d\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"75bd2ee1-144a-4bca-8175-c2c586d80d4a\",\"type\":\"CategoricalTicker\"}},\"id\":\"0cb7015c-3c0d-4363-a78e-7bd9a52899f8\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.7},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.7},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.6},\"x\":{\"field\":\"x\"}},\"id\":\"a1043ec9-8bfe-4769-9eb3-b98a0857256f\",\"type\":\"VBar\"},{\"attributes\":{\"plot\":null,\"text\":\"Assaults by Day of Week\"},\"id\":\"1d1a99e0-b135-4571-8f2a-145df418694a\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"5ca6d438-c0f6-48c8-a4ae-480b7691c365\",\"type\":\"BasicTicker\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"8863a464-064f-4482-b3c7-e4c832180f8d\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"5ca6d438-c0f6-48c8-a4ae-480b7691c365\",\"type\":\"BasicTicker\"}},\"id\":\"afbb1837-3e84-47f4-8d5e-4872b807b60c\",\"type\":\"Grid\"},{\"attributes\":{\"data_source\":{\"id\":\"35a70a48-2c99-4008-8972-baccf21d5a7a\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"a1043ec9-8bfe-4769-9eb3-b98a0857256f\",\"type\":\"VBar\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"832a3194-bece-48fc-a42a-6c0499dae051\",\"type\":\"VBar\"},\"selection_glyph\":null,\"view\":{\"id\":\"b48a9951-5b38-423c-a8f9-4c0c08bf3730\",\"type\":\"CDSView\"}},\"id\":\"3af708a2-f808-46fb-aeeb-b3bfde874967\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"factors\":[\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\",\"Sunday\"]},\"id\":\"3448d3ae-250f-42d1-93f8-c80a76337ecd\",\"type\":\"FactorRange\"},{\"attributes\":{\"source\":{\"id\":\"35a70a48-2c99-4008-8972-baccf21d5a7a\",\"type\":\"ColumnDataSource\"}},\"id\":\"b48a9951-5b38-423c-a8f9-4c0c08bf3730\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"e8dc41b1-19ee-47ee-b7e4-f9cc4e97e184\",\"type\":\"CategoricalScale\"},{\"attributes\":{},\"id\":\"1c03f874-9a63-4981-ac64-21b7dae6bc91\",\"type\":\"CategoricalTickFormatter\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\"},\"id\":\"aada079c-d303-4db9-b08d-755ad9d61f8e\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"d70f75fd-389a-4491-9312-bdb6e7230455\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"callback\":null},\"id\":\"5cca3a39-1bbd-4636-af8c-87eca50723f2\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"d30fde13-2b1e-4a15-ae83-1733b8b2973c\",\"type\":\"Selection\"},{\"attributes\":{\"formatter\":{\"id\":\"d70f75fd-389a-4491-9312-bdb6e7230455\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"8863a464-064f-4482-b3c7-e4c832180f8d\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"5ca6d438-c0f6-48c8-a4ae-480b7691c365\",\"type\":\"BasicTicker\"}},\"id\":\"d465acbc-5f09-44fc-b453-695dc95d9df0\",\"type\":\"LinearAxis\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.6},\"x\":{\"field\":\"x\"}},\"id\":\"832a3194-bece-48fc-a42a-6c0499dae051\",\"type\":\"VBar\"}],\"root_ids\":[\"8863a464-064f-4482-b3c7-e4c832180f8d\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"d9dbe675-dfc1-40e5-9ecc-23e7635cc494\",\"roots\":{\"8863a464-064f-4482-b3c7-e4c832180f8d\":\"1ab1d982-a8e4-446a-a3a4-1e936468bbcb\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "8863a464-064f-4482-b3c7-e4c832180f8d" } }, "output_type": "display_data" } ], "source": [ "# Set up labels for the days of the week\n", "weekdays = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']\n", "\n", "p = figure(x_range = weekdays, plot_height = 250, title = \"Assaults by Day of Week\",\n", " toolbar_location = None, tools = \"\")\n", "p.vbar(x = group.index, top = group.values.tolist(), width = 0.6, alpha = 0.7)\n", "\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How do incidents vary by hour of day?\n", "\n", "Are there trends by time of day? Is there a time of day that is most dangerous? Ignoring day of week we will aggregate all the incidents by the hour which they occured. This will let us see the number of incidents over a 24 hour period." ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"dde61526-8275-4b2f-8766-ea177afbb03b\":{\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"cc1dbe52-a9ec-43f3-b053-bd5f94ec6c0b\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"e5fd56f2-994c-419b-a3eb-edb27283005d\",\"type\":\"LinearScale\"},{\"attributes\":{\"formatter\":{\"id\":\"159b02d8-386e-4bb3-b20c-7d120c65d263\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"0d4fce12-09ed-4cb8-85a8-576b951e2551\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1e57da40-3e3f-4dbc-a1f8-ddb68c9f6a65\",\"type\":\"BasicTicker\"}},\"id\":\"96e409fb-117e-4f80-8df3-283037ccb8e5\",\"type\":\"LinearAxis\"},{\"attributes\":{\"plot\":{\"id\":\"0d4fce12-09ed-4cb8-85a8-576b951e2551\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1e57da40-3e3f-4dbc-a1f8-ddb68c9f6a65\",\"type\":\"BasicTicker\"}},\"id\":\"394a213a-763b-4fa9-a313-155c2f47359c\",\"type\":\"Grid\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"0d4fce12-09ed-4cb8-85a8-576b951e2551\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9dc81618-5adf-483d-98f2-a60e896facfc\",\"type\":\"BasicTicker\"}},\"id\":\"ff50afbf-4a89-43d2-b70a-72bd8083e0f0\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"9dc81618-5adf-483d-98f2-a60e896facfc\",\"type\":\"BasicTicker\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5c26c920-1182-4703-9c6d-36988c96264d\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"1e57da40-3e3f-4dbc-a1f8-ddb68c9f6a65\",\"type\":\"BasicTicker\"},{\"attributes\":{\"formatter\":{\"id\":\"455ca440-2bd3-4804-a0bb-7340a56cda5f\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"0d4fce12-09ed-4cb8-85a8-576b951e2551\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"9dc81618-5adf-483d-98f2-a60e896facfc\",\"type\":\"BasicTicker\"}},\"id\":\"968284f1-0ef8-4ad6-a47a-859b99e9b2ab\",\"type\":\"LinearAxis\"},{\"attributes\":{\"data_source\":{\"id\":\"630e0f2b-cd5a-4795-9572-511cb3c7c108\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"9e8497f8-91b9-4b3b-9014-81e478915a76\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"5c26c920-1182-4703-9c6d-36988c96264d\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"3a78edf0-4fc1-4e59-b657-cc0f9049ad90\",\"type\":\"CDSView\"}},\"id\":\"495e61f6-fd03-446a-9c4a-a74afca53d32\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"#1f77b4\"},\"size\":{\"units\":\"screen\",\"value\":6},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"c246c873-40b3-452f-b5d5-115a245e2a0f\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23],\"y\":[378,323,304,121,107,60,71,121,193,213,232,284,308,339,340,367,411,430,400,414,383,402,354,294]},\"selected\":{\"id\":\"813131a0-f10e-4a38-b915-803f7cc46420\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"0e264363-a62a-4c66-8362-af30df518ff1\",\"type\":\"UnionRenderers\"}},\"id\":\"787440a6-3e40-4dbe-bd64-041fb4bbe42d\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"8d0a69b0-8eee-49b9-9705-2715f20fed29\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"849a051f-6708-4de1-9d2a-080c2e0d32d7\",\"type\":\"PanTool\"},{\"attributes\":{\"below\":[{\"id\":\"96e409fb-117e-4f80-8df3-283037ccb8e5\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"968284f1-0ef8-4ad6-a47a-859b99e9b2ab\",\"type\":\"LinearAxis\"}],\"plot_height\":250,\"plot_width\":800,\"renderers\":[{\"id\":\"96e409fb-117e-4f80-8df3-283037ccb8e5\",\"type\":\"LinearAxis\"},{\"id\":\"394a213a-763b-4fa9-a313-155c2f47359c\",\"type\":\"Grid\"},{\"id\":\"968284f1-0ef8-4ad6-a47a-859b99e9b2ab\",\"type\":\"LinearAxis\"},{\"id\":\"ff50afbf-4a89-43d2-b70a-72bd8083e0f0\",\"type\":\"Grid\"},{\"id\":\"8d0a69b0-8eee-49b9-9705-2715f20fed29\",\"type\":\"BoxAnnotation\"},{\"id\":\"495e61f6-fd03-446a-9c4a-a74afca53d32\",\"type\":\"GlyphRenderer\"},{\"id\":\"eb4f79b1-95aa-4dec-b90f-68023785037e\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"c5894a56-56cc-43e3-bef1-594c5cfd0a83\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"7041fa53-734a-4812-b9bb-f515c816412f\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"9e4ec690-f781-4cf9-8873-d0de5b2b1986\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"e5fd56f2-994c-419b-a3eb-edb27283005d\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"8fef88d4-78b1-40a9-b3aa-b47559bd7901\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"cc1dbe52-a9ec-43f3-b053-bd5f94ec6c0b\",\"type\":\"LinearScale\"}},\"id\":\"0d4fce12-09ed-4cb8-85a8-576b951e2551\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"source\":{\"id\":\"630e0f2b-cd5a-4795-9572-511cb3c7c108\",\"type\":\"ColumnDataSource\"}},\"id\":\"3a78edf0-4fc1-4e59-b657-cc0f9049ad90\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"116ed488-f1d9-4de6-aaec-7172138c469d\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"callback\":null},\"id\":\"8fef88d4-78b1-40a9-b3aa-b47559bd7901\",\"type\":\"DataRange1d\"},{\"attributes\":{\"overlay\":{\"id\":\"8d0a69b0-8eee-49b9-9705-2715f20fed29\",\"type\":\"BoxAnnotation\"}},\"id\":\"670ba4ce-b3cc-457b-9528-b60584094e36\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"b1cf7551-c96a-44ab-992b-9be932258a4e\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"74c49863-fa94-48da-bfa5-47b38c76823b\",\"type\":\"ResetTool\"},{\"attributes\":{\"plot\":null,\"text\":\"Assaults Aggregated by Hour\"},\"id\":\"c5894a56-56cc-43e3-bef1-594c5cfd0a83\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"14d52af7-3f21-42ef-af7f-5d5fa793ef5f\",\"type\":\"HelpTool\"},{\"attributes\":{\"data_source\":{\"id\":\"787440a6-3e40-4dbe-bd64-041fb4bbe42d\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"c246c873-40b3-452f-b5d5-115a245e2a0f\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"016ca60a-eda5-4133-b6ca-c9065bf697ad\",\"type\":\"Circle\"},\"selection_glyph\":null,\"view\":{\"id\":\"20480e3c-846f-40b4-b1ac-5f545a65115e\",\"type\":\"CDSView\"}},\"id\":\"eb4f79b1-95aa-4dec-b90f-68023785037e\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null},\"id\":\"9e4ec690-f781-4cf9-8873-d0de5b2b1986\",\"type\":\"DataRange1d\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"size\":{\"units\":\"screen\",\"value\":6},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"016ca60a-eda5-4133-b6ca-c9065bf697ad\",\"type\":\"Circle\"},{\"attributes\":{\"source\":{\"id\":\"787440a6-3e40-4dbe-bd64-041fb4bbe42d\",\"type\":\"ColumnDataSource\"}},\"id\":\"20480e3c-846f-40b4-b1ac-5f545a65115e\",\"type\":\"CDSView\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23],\"y\":[378,323,304,121,107,60,71,121,193,213,232,284,308,339,340,367,411,430,400,414,383,402,354,294]},\"selected\":{\"id\":\"f1e48311-1a40-438d-a9c4-e1ed521773b8\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"16bf324b-0a24-44df-a6c2-b13d7bf07c25\",\"type\":\"UnionRenderers\"}},\"id\":\"630e0f2b-cd5a-4795-9572-511cb3c7c108\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"159b02d8-386e-4bb3-b20c-7d120c65d263\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"455ca440-2bd3-4804-a0bb-7340a56cda5f\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"f1e48311-1a40-438d-a9c4-e1ed521773b8\",\"type\":\"Selection\"},{\"attributes\":{\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"9e8497f8-91b9-4b3b-9014-81e478915a76\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"16bf324b-0a24-44df-a6c2-b13d7bf07c25\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"813131a0-f10e-4a38-b915-803f7cc46420\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"0e264363-a62a-4c66-8362-af30df518ff1\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"849a051f-6708-4de1-9d2a-080c2e0d32d7\",\"type\":\"PanTool\"},{\"id\":\"116ed488-f1d9-4de6-aaec-7172138c469d\",\"type\":\"WheelZoomTool\"},{\"id\":\"670ba4ce-b3cc-457b-9528-b60584094e36\",\"type\":\"BoxZoomTool\"},{\"id\":\"b1cf7551-c96a-44ab-992b-9be932258a4e\",\"type\":\"SaveTool\"},{\"id\":\"74c49863-fa94-48da-bfa5-47b38c76823b\",\"type\":\"ResetTool\"},{\"id\":\"14d52af7-3f21-42ef-af7f-5d5fa793ef5f\",\"type\":\"HelpTool\"}]},\"id\":\"7041fa53-734a-4812-b9bb-f515c816412f\",\"type\":\"Toolbar\"}],\"root_ids\":[\"0d4fce12-09ed-4cb8-85a8-576b951e2551\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"dde61526-8275-4b2f-8766-ea177afbb03b\",\"roots\":{\"0d4fce12-09ed-4cb8-85a8-576b951e2551\":\"241593c0-55de-4e58-a963-994331345ce7\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "0d4fce12-09ed-4cb8-85a8-576b951e2551" } }, "output_type": "display_data" } ], "source": [ "# Group by the hour of day and plot\n", "group = crime_df_time_index.groupby('HOUR').count()['OBJECTID']\n", "\n", "# Create the plot\n", "p = figure(plot_width=800, plot_height=250, title=\"Assaults Aggregated by Hour\")\n", "p.line(group.index, group, line_width=2)\n", "p.circle(group.index, group, alpha=0.5, size = 6)\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this exploratory data analysis we explored the different categories and number of crime incidents. We used the temporal data to look at the trend over the course of the year, February had the lowest number of incidents and they peak in May. We also looked at the number of incidents by day of week and saw that weekends generally have more incidents than weekdays. Aggregating the assaults by the hour of day showed that the early morning from 3am - 7am have the lowest number of incidents while the late afternoon and evenings have the most incidents. All of this information can be used to allocate police resources to times when they are most needed." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 3. Perform a Spatial Analysis of the Data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Up to now the focus of our analysis has been looking at temporal trends in the crime incident data. In addition to a temporal component each incident also has a geographic location. This is where arcpy library jumped in. We will now be taking a look at crime incident trends by location. We will be using the police precints as our aggregation layer as these would be the divisions that the police operate in. Some of the questions we will investigate include:\n", "\n", "1) Where do the most crimes occur?\n", "\n", "2) Are there any spatio-temporal patterns in the crime incidents? And,\n", "\n", "3) Are there areas which share similar temporal trends in incidents?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hot Spot Analysis\n", "\n", "The [Optimized Hot Spot Analysis ](http://pro.arcgis.com/en/pro-app/tool-reference/spatial-statistics/optimized-hot-spot-analysis.htm) tool will aggregate the crime incidents points into the police precinct polygons and create a map of statistically significant hot and cold spots using the Getis-Ord Gi* statistic. It evaluates the characteristics of the input feature class to produce optimal results.\n", "\n" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'/arcgis/home/BostonCrimeAnalysis.gdb'" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create an output geodatabase, this is where we will save our outputs\n", "outputSpace = r\"/arcgis/home\"\n", "outputGDB = \"BostonCrimeAnalysis.gdb\"\n", "arcpy.management.CreateFileGDB(outputSpace, outputGDB, \"CURRENT\")\n", "\n", "# reference path to output gdb\n", "analysis_gdb = os.path.join(outputSpace, outputGDB)\n", "\n", "# Set workspace for default output location\n", "arcpy.env.workspace = analysis_gdb\n", "arcpy.env.workspace" ] }, { "cell_type": "code", "execution_count": 68, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'/arcgis/home/BostonCrimeAnalysis.gdb/precincts_feature'" ] }, "execution_count": 68, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set up paths for output feature classes\n", "crime_points = os.path.join(analysis_gdb, 'assault_points')\n", "precincts_feature = os.path.join(analysis_gdb, 'precincts_feature')\n", "\n", "# Write the spatially enabled data frames to feature classes for geoprocessing\n", "crime_sedf.spatial.to_featureclass(crime_points)\n", "precincts_sedf.spatial.to_featureclass(precincts_feature)" ] }, { "cell_type": "code", "execution_count": 69, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/arcgis/home/BostonCrimeAnalysis.gdb/assault_points\n", "/arcgis/home/BostonCrimeAnalysis.gdb/precincts_feature\n" ] } ], "source": [ "# Files in gdb is not readable directly\n", "# But we can list the full path of files contained in the gdb in the following way\n", "datasets = arcpy.ListDatasets(feature_type='feature')\n", "datasets = [''] + datasets if datasets is not None else []\n", "\n", "for ds in datasets:\n", " for fc in arcpy.ListFeatureClasses(feature_dataset=ds):\n", " path = os.path.join(arcpy.env.workspace, ds, fc)\n", " print(path)" ] }, { "cell_type": "code", "execution_count": 70, "metadata": {}, "outputs": [], "source": [ "# Run optimized hotspot anlaysis aggregating the crime incidents into the police precinct polygons\n", "hotspot_result = \"hotspot_result\"\n", "aggregation_method = \"COUNT_INCIDENTS_WITHIN_AGGREGATION_POLYGONS\"\n", "hotspot_result = arcpy.stats.OptimizedHotSpotAnalysis(Input_Features=crime_points, \n", " Output_Features=hotspot_result, \n", " Incident_Data_Aggregation_Method=aggregation_method, \n", " Polygons_For_Aggregating_Incidents_Into_Counts=precincts_feature, \n", " )" ] }, { "cell_type": "code", "execution_count": 71, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:10 PM\n", "Running script OptimizedHotSpotAnalysis...\n", "WARNING 001605: Distances for Geographic Coordinates (degrees, minutes, seconds) are analyzed using Chordal Distances in meters.\n", "\n", "************************** Initial Data Assessment ***************************\n", "Making sure there are enough incidents for analysis....\n", " - There are 6840 valid input features.\n", "\n", "Evaluating the aggregation polygons....\n", " - There are 255 valid input aggregation polygons.\n", "\n", "Looking for locational outliers....\n", " - There were 2 outlier locations; these will not be used to compute the optimal fixed distance band.\n", "\n", "**************************** Incident Aggregation ****************************\n", "Counting the number of incidents in each polygon....\n", " - Analysis is performed on all aggregation polygons.\n", "\n", "Evaluating incident counts and number of polygons....\n", " - The aggregation process resulted in 255 weighted polygons.\n", " - Incident Count Properties:\n", " Min: 0.0000\n", " Max: 372.0000\n", " Mean: 26.8275\n", " Std. Dev.: 34.4724\n", "***************************** Scale of Analysis ******************************\n", "Looking for an optimal scale of analysis by assessing the intensity of clustering at increasing distances....\n", " - The optimal fixed distance band is based on peak clustering found at 2282.7579 Meters\n", "\n", "***************************** Hot Spot Analysis ******************************\n", "Finding statistically significant clusters of high and low incident counts....\n", " - There are 99 output features statistically significant based on an FDR correction for multiple testing and spatial dependence.\n", "\n", " - 2.4% of features had less than 8 neighbors based on the distance band of 2282.7579 Meters\n", "*********************************** Output ***********************************\n", "Creating output feature class: /arcgis/home/BostonCrimeAnalysis.gdb\\hotspot_result\n", " - Red output features represent hot spots where high incident counts cluster.\n", " - Blue output features represent cold spots where low incident counts cluster.\n", "\n", "\n", "\n", "Completed script Optimized Hot Spot Analysis...\n", "Succeeded at Friday, March 15, 2019 5:46:16 PM (Elapsed Time: 6.28 seconds)\n" ] } ], "source": [ "# Print out message by running the tool\n", "print(hotspot_result.getMessages())" ] }, { "cell_type": "code", "execution_count": 72, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDSOURCE_IDJoin_CountGiZScoreGiPValueNNeighborsGi_BinSHAPE
01123-0.6813590.49564560{\"rings\": [[[-71.01304725199998, 42.3853109380...
1228-0.7195130.471825160{\"rings\": [[[-71.03131003999994, 42.3702461870...
23320-0.5964870.550850150{\"rings\": [[[-71.02715579999995, 42.3702870860...
34411.4451350.148420220{\"rings\": [[[-71.05734319999993, 42.3746231680...
4551-0.8241810.409837160{\"rings\": [[[-71.06329203599995, 42.3815201530...
566241.6796400.093027190{\"rings\": [[[-71.05818234199995, 42.3793662030...
67712-1.1109880.266574140{\"rings\": [[[-71.06636894899998, 42.3797311290...
788243.2323710.001228262{\"rings\": [[[-71.05724029099997, 42.3634811770...
89983.0564760.002240252{\"rings\": [[[-71.05664811599996, 42.3638840410...
91010624.8620070.000001413{\"rings\": [[[-71.06704979399996, 42.3443456650...
\n", "
" ], "text/plain": [ " OBJECTID SOURCE_ID Join_Count GiZScore GiPValue NNeighbors Gi_Bin \\\n", "0 1 1 23 -0.681359 0.495645 6 0 \n", "1 2 2 8 -0.719513 0.471825 16 0 \n", "2 3 3 20 -0.596487 0.550850 15 0 \n", "3 4 4 1 1.445135 0.148420 22 0 \n", "4 5 5 1 -0.824181 0.409837 16 0 \n", "5 6 6 24 1.679640 0.093027 19 0 \n", "6 7 7 12 -1.110988 0.266574 14 0 \n", "7 8 8 24 3.232371 0.001228 26 2 \n", "8 9 9 8 3.056476 0.002240 25 2 \n", "9 10 10 62 4.862007 0.000001 41 3 \n", "\n", " SHAPE \n", "0 {\"rings\": [[[-71.01304725199998, 42.3853109380... \n", "1 {\"rings\": [[[-71.03131003999994, 42.3702461870... \n", "2 {\"rings\": [[[-71.02715579999995, 42.3702870860... \n", "3 {\"rings\": [[[-71.05734319999993, 42.3746231680... \n", "4 {\"rings\": [[[-71.06329203599995, 42.3815201530... \n", "5 {\"rings\": [[[-71.05818234199995, 42.3793662030... \n", "6 {\"rings\": [[[-71.06636894899998, 42.3797311290... \n", "7 {\"rings\": [[[-71.05724029099997, 42.3634811770... \n", "8 {\"rings\": [[[-71.05664811599996, 42.3638840410... \n", "9 {\"rings\": [[[-71.06704979399996, 42.3443456650... " ] }, "execution_count": 72, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a SeDF to check the output\n", "hotspot_sedf = pd.DataFrame.spatial.from_featureclass(hotspot_result)\n", "hotspot_sedf.head(10)" ] }, { "cell_type": "code", "execution_count": 73, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDSOURCE_IDJoin_CountGiZScoreGiPValueNNeighborsGi_BinSHAPE
1451461469-2.4795390.01315525-2{\"rings\": [[[-71.13403509199998, 42.2809428210...
88898910-2.1064210.03516822-1{\"rings\": [[[-71.15379160799995, 42.3408491590...
01123-0.6813590.49564560{\"rings\": [[[-71.01304725199998, 42.3853109380...
171818262.4067360.016096441{\"rings\": [[[-71.09559790299994, 42.3376813040...
788243.2323710.001228262{\"rings\": [[[-71.05724029099997, 42.3634811770...
91010624.8620070.000001413{\"rings\": [[[-71.06704979399996, 42.3443456650...
\n", "
" ], "text/plain": [ " OBJECTID SOURCE_ID Join_Count GiZScore GiPValue NNeighbors Gi_Bin \\\n", "145 146 146 9 -2.479539 0.013155 25 -2 \n", "88 89 89 10 -2.106421 0.035168 22 -1 \n", "0 1 1 23 -0.681359 0.495645 6 0 \n", "17 18 18 26 2.406736 0.016096 44 1 \n", "7 8 8 24 3.232371 0.001228 26 2 \n", "9 10 10 62 4.862007 0.000001 41 3 \n", "\n", " SHAPE \n", "145 {\"rings\": [[[-71.13403509199998, 42.2809428210... \n", "88 {\"rings\": [[[-71.15379160799995, 42.3408491590... \n", "0 {\"rings\": [[[-71.01304725199998, 42.3853109380... \n", "17 {\"rings\": [[[-71.09559790299994, 42.3376813040... \n", "7 {\"rings\": [[[-71.05724029099997, 42.3634811770... \n", "9 {\"rings\": [[[-71.06704979399996, 42.3443456650... " ] }, "execution_count": 73, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Show only the first rows for each unique value of Gi_Bin\n", "hotspot_sedf.drop_duplicates(subset=\"Gi_Bin\", keep='first', inplace=False).sort_values(by=[\"Gi_Bin\"])\n", "\n", "# Tips for understanding the Gi_Bin field:\n", "# -3 : Cold Spot - 99% Confidence\n", "# -2 : Cold Spot - 95% Confidence\n", "# -1 : Cold Spot - 90% Confidence\n", "# 0 : Not Significant\n", "# 1 : Hot Spot - 90% Confidence\n", "# 2 : Hot Spot - 95% Confidence\n", "# 3 : Hot Spot - 99% Confidence" ] }, { "cell_type": "code", "execution_count": 74, "metadata": {}, "outputs": [], "source": [ "# Create a map based on Gi_Bin field to look at the results\n", "# First, we need to create a variable using Gi_Bin field value for symbology, fill-in colors are exactly the same as layer file in ArcGIS Pro\n", "outline_render = {\"type\":\"esriSLS\",\n", " \"color\":[255,255,255,100],\n", " \"width\":0.7,\n", " \"style\":\"esriSLSSolid\"}\n", "\n", "optimized_hot_spots_layer = [\n", " {\"label\":\"Cold Spot - 99% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[69,117,181,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"-3\"},\n", " {\"label\":\"Cold Spot - 95% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[132,158,186,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"-2\"},\n", " {\"label\":\"Cold Spot - 90% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[192,204,190,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"-1\"},\n", " {\"label\":\"Not Significant\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[156,156,156,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"0\"},\n", " {\"label\":\"Hot Spot - 90% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[250,185,132,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"1\"},\n", " {\"label\":\"Hot Spot - 95% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[237,117,81,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"2\"},\n", " {\"label\":\"Hot Spot - 99% Confidence\",\n", " \"symbol\":{\"type\":\"esriSFS\",\n", " \"color\":[214,47,39,255],\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"},\n", " \"value\":\"3\"}\n", "]" ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "0649d26106c84d159e33b4abd46d20d1", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(layout=Layout(height='400px', width='100%'))" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Second, create an arcade expression variable using the field of Gi_Bin\n", "# For more about Arcade Expression:https://developers.arcgis.com/javascript/latest/guide/arcade/index.html#visualization\n", "arcade = \"$feature.Gi_Bin\";\n", "\n", "# Finally, we can visualize on map with the unique values of Gi_Bin\n", "hotspot_map = gis.map(\"Boston\")\n", "hotspot_sedf.spatial.plot(hotspot_map, \n", " col='Gi_Bin', \n", " renderer_type='u-a', # 'u-a' stands for uniqe value with arcade expression\n", " unique_values=optimized_hot_spots_layer,\n", " arcade_expression=arcade)\n", "hotspot_map" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The column JOIN_COUNT in the hotspot results is the number of crime incidents aggregated into that police precinct. The hotspot analysis results also has a field for the SOURCE_ID, which is a link to the object ID of the input. We want to report our results based on the police precinct rather than the ObjectID. Using the lookup table we created earlier we can use pandas to merge the two dataframes to add the PRECINCT_WARD value back to the hotspot results table. We can aggregate and report the number of crime incidents based on " ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDSOURCE_IDJoin_CountGiZScoreGiPValueNNeighborsGi_BinSHAPEPRECINCTWARD_INDEXPRECINCTWARD_ID
01123-0.6813590.49564560{'rings': [[[-71.01304725199998, 42.3853109380...10113
1228-0.7195130.471825160{'rings': [[[-71.03131003999994, 42.3702461870...20102
23320-0.5964870.550850150{'rings': [[[-71.02715579999995, 42.3702870860...30105
34411.4451350.148420220{'rings': [[[-71.05734319999993, 42.3746231680...40203
4551-0.8241810.409837160{'rings': [[[-71.06329203599995, 42.3815201530...50205
\n", "
" ], "text/plain": [ " OBJECTID SOURCE_ID Join_Count GiZScore GiPValue NNeighbors Gi_Bin \\\n", "0 1 1 23 -0.681359 0.495645 6 0 \n", "1 2 2 8 -0.719513 0.471825 16 0 \n", "2 3 3 20 -0.596487 0.550850 15 0 \n", "3 4 4 1 1.445135 0.148420 22 0 \n", "4 5 5 1 -0.824181 0.409837 16 0 \n", "\n", " SHAPE PRECINCTWARD_INDEX \\\n", "0 {'rings': [[[-71.01304725199998, 42.3853109380... 1 \n", "1 {'rings': [[[-71.03131003999994, 42.3702461870... 2 \n", "2 {'rings': [[[-71.02715579999995, 42.3702870860... 3 \n", "3 {'rings': [[[-71.05734319999993, 42.3746231680... 4 \n", "4 {'rings': [[[-71.06329203599995, 42.3815201530... 5 \n", "\n", " PRECINCTWARD_ID \n", "0 0113 \n", "1 0102 \n", "2 0105 \n", "3 0203 \n", "4 0205 " ] }, "execution_count": 76, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Add the PRECINCT_WARD column to the hotspot results\n", "hotspot_sedf_precincts = pd.DataFrame.merge(hotspot_sedf, precinct_lookup_df, \n", " left_on='SOURCE_ID', right_on='PRECINCTWARD_INDEX')\n", "hotspot_sedf_precincts.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that the PRECINCT_WARD value and the number of crimes are in the same dataframe we can determine which precinct has the highest number of incidents. After sorting by the number of incidents we look at the top 10 precincts for number of incidents." ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PRECINCTWARD_IDJoin_CountSOURCE_ID
1910306372192
2160308245217
2340802161235
10040212911
34090512535
65140110666
2005019621
3208038733
3709048238
176180480177
\n", "
" ], "text/plain": [ " PRECINCTWARD_ID Join_Count SOURCE_ID\n", "191 0306 372 192\n", "216 0308 245 217\n", "234 0802 161 235\n", "10 0402 129 11\n", "34 0905 125 35\n", "65 1401 106 66\n", "20 0501 96 21\n", "32 0803 87 33\n", "37 0904 82 38\n", "176 1804 80 177" ] }, "execution_count": 77, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at the top 10 worst precints for assaults\n", "ten_worst = hotspot_sedf_precincts.sort_values('Join_Count', ascending=False)[:10][['PRECINCTWARD_ID', 'Join_Count', 'SOURCE_ID']].copy()\n", "ten_worst[:10]" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[372, 245, 161, 129, 125, 106, 96, 87, 82, 80]" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a list of the PRECINCTWARD_ID\n", "PRECINCTWARD_ID = [str(cur_val) for cur_val in ten_worst['PRECINCTWARD_ID'].tolist()]\n", "assault_count = ten_worst['Join_Count'].tolist()\n", "assault_count" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "With the precincts with the top 10 number of incidents we can make a plot to compare the number of incidents." ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"02d2ddc3-53c7-4295-8393-56b148fcb505\":{\"roots\":{\"references\":[{\"attributes\":{\"callback\":null},\"id\":\"32a3ec7f-40c1-4983-8fd1-dc0b0b166835\",\"type\":\"DataRange1d\"},{\"attributes\":{\"callback\":null,\"data\":{\"top\":[372,245,161,129,125,106,96,87,82,80],\"x\":[0,1,2,3,4,5,6,7,8,9]},\"selected\":{\"id\":\"1f179b59-9565-4519-9ced-6f55a35ebc34\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"abebc9a2-4b46-41ca-8fb6-8e5e55320bad\",\"type\":\"UnionRenderers\"}},\"id\":\"d73d2a40-5315-482f-bc85-b38e309cb09e\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\"},\"id\":\"04948f8e-fd34-429b-a420-762a4a4be67d\",\"type\":\"Toolbar\"},{\"attributes\":{\"formatter\":{\"id\":\"7b108d9d-f90a-479e-9cfc-85e2b600512f\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"657eea77-1186-4503-9441-f9aeb382d74f\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1582cf60-886e-4c4c-b7d6-c90a68e57ea1\",\"type\":\"BasicTicker\"}},\"id\":\"b0f9ba45-2e3f-4192-8324-ed6d000ac67e\",\"type\":\"LinearAxis\"},{\"attributes\":{\"below\":[{\"id\":\"15d3bfc0-9d0a-4ad0-9480-6e0f99b8d6e8\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"b0f9ba45-2e3f-4192-8324-ed6d000ac67e\",\"type\":\"LinearAxis\"}],\"plot_height\":250,\"renderers\":[{\"id\":\"15d3bfc0-9d0a-4ad0-9480-6e0f99b8d6e8\",\"type\":\"LinearAxis\"},{\"id\":\"c859f39b-9449-43ce-9474-3947f4bfc266\",\"type\":\"Grid\"},{\"id\":\"b0f9ba45-2e3f-4192-8324-ed6d000ac67e\",\"type\":\"LinearAxis\"},{\"id\":\"354bff93-6688-416b-9fd3-114bbd91c5d6\",\"type\":\"Grid\"},{\"id\":\"bf4e424a-2a3d-44ad-9ace-303ccedcf13d\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"eee58fe3-6694-4928-8ba2-69893a7b8e67\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"04948f8e-fd34-429b-a420-762a4a4be67d\",\"type\":\"Toolbar\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"a03813a9-930a-4335-8775-e13d1225b6c5\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"7ed2713d-f3e7-482d-8382-5c090e775e2b\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"32a3ec7f-40c1-4983-8fd1-dc0b0b166835\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"30a71f86-d985-4a55-8d68-8aa458b434df\",\"type\":\"LinearScale\"}},\"id\":\"657eea77-1186-4503-9441-f9aeb382d74f\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"fill_color\":{\"value\":\"#1f77b4\"},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.9},\"x\":{\"field\":\"x\"}},\"id\":\"63b501ab-c46d-4716-a60b-b027992175c6\",\"type\":\"VBar\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"657eea77-1186-4503-9441-f9aeb382d74f\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1582cf60-886e-4c4c-b7d6-c90a68e57ea1\",\"type\":\"BasicTicker\"}},\"id\":\"354bff93-6688-416b-9fd3-114bbd91c5d6\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"1582cf60-886e-4c4c-b7d6-c90a68e57ea1\",\"type\":\"BasicTicker\"},{\"attributes\":{\"desired_num_ticks\":10},\"id\":\"ba978902-68e2-4f8c-b334-57e9bc4ed8fe\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"abebc9a2-4b46-41ca-8fb6-8e5e55320bad\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"plot\":{\"id\":\"657eea77-1186-4503-9441-f9aeb382d74f\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"ba978902-68e2-4f8c-b334-57e9bc4ed8fe\",\"type\":\"BasicTicker\"}},\"id\":\"c859f39b-9449-43ce-9474-3947f4bfc266\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"30a71f86-d985-4a55-8d68-8aa458b434df\",\"type\":\"LinearScale\"},{\"attributes\":{\"formatter\":{\"id\":\"32377eff-0058-438d-b4a1-f6bfa0ac332a\",\"type\":\"BasicTickFormatter\"},\"major_label_overrides\":{\"0\":\"0306\",\"1\":\"0308\",\"2\":\"0802\",\"3\":\"0402\",\"4\":\"0905\",\"5\":\"1401\",\"6\":\"0501\",\"7\":\"0803\",\"8\":\"0904\",\"9\":\"1804\"},\"plot\":{\"id\":\"657eea77-1186-4503-9441-f9aeb382d74f\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"ba978902-68e2-4f8c-b334-57e9bc4ed8fe\",\"type\":\"BasicTicker\"}},\"id\":\"15d3bfc0-9d0a-4ad0-9480-6e0f99b8d6e8\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"32377eff-0058-438d-b4a1-f6bfa0ac332a\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"data_source\":{\"id\":\"d73d2a40-5315-482f-bc85-b38e309cb09e\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"63b501ab-c46d-4716-a60b-b027992175c6\",\"type\":\"VBar\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"8f84102b-cc4c-4fad-a31b-73619f433767\",\"type\":\"VBar\"},\"selection_glyph\":null,\"view\":{\"id\":\"f0bb21f9-5fe0-4d49-b602-8c9af42cfced\",\"type\":\"CDSView\"}},\"id\":\"bf4e424a-2a3d-44ad-9ace-303ccedcf13d\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"1f179b59-9565-4519-9ced-6f55a35ebc34\",\"type\":\"Selection\"},{\"attributes\":{\"plot\":null,\"text\":\"Top 10 Precincts by Total Assaults for 2017\"},\"id\":\"eee58fe3-6694-4928-8ba2-69893a7b8e67\",\"type\":\"Title\"},{\"attributes\":{\"callback\":null},\"id\":\"a03813a9-930a-4335-8775-e13d1225b6c5\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"7b108d9d-f90a-479e-9cfc-85e2b600512f\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"7ed2713d-f3e7-482d-8382-5c090e775e2b\",\"type\":\"LinearScale\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.9},\"x\":{\"field\":\"x\"}},\"id\":\"8f84102b-cc4c-4fad-a31b-73619f433767\",\"type\":\"VBar\"},{\"attributes\":{\"source\":{\"id\":\"d73d2a40-5315-482f-bc85-b38e309cb09e\",\"type\":\"ColumnDataSource\"}},\"id\":\"f0bb21f9-5fe0-4d49-b602-8c9af42cfced\",\"type\":\"CDSView\"}],\"root_ids\":[\"657eea77-1186-4503-9441-f9aeb382d74f\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"02d2ddc3-53c7-4295-8393-56b148fcb505\",\"roots\":{\"657eea77-1186-4503-9441-f9aeb382d74f\":\"b3933553-4c69-4922-934b-db7e5b4497f3\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "657eea77-1186-4503-9441-f9aeb382d74f" } }, "output_type": "display_data" } ], "source": [ "p = figure( plot_height=250, title=\"Top 10 Precincts by Total Assaults for 2017\",\n", " toolbar_location=None, tools=\"\")\n", "\n", "x_axis_values = np.arange(10)\n", "p.vbar(x=x_axis_values, top=assault_count, width=0.9)\n", "\n", "# Way to custom label the chart with dates\n", "label_dict = {}\n", "for idx, cur_label in enumerate(x_axis_values):\n", " label_dict[idx] = PRECINCTWARD_ID[idx]\n", "p.xaxis.major_label_overrides = label_dict\n", "\n", "p.xaxis[0].ticker.desired_num_ticks = 10\n", "\n", "\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the IDs of the top 10 precincts we can also filter the results to show these on the map. Now we have used the spatial data to determine the locations with the most incidents and displayed them on a map." ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 80, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Outline the top 10 precincts on the previous map\n", "hotspot_top10_sedf = hotspot_sedf[hotspot_sedf['Join_Count'] > min(assault_count)]\n", "hotspot_top10_sedf.spatial.plot(hotspot_map, cmap = [0,0,0,0], \n", " line_width=1, outline_color = [0,0,0,255])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create a Space Time Cube\n", "\n", "The [Create Space Time Cube by Aggregating Points](http://pro.arcgis.com/en/pro-app/tool-reference/space-time-pattern-mining/create-space-time-cube.htm) tool aggregates incident points into space-time bins. Within each bin, the points are counted, and specified attributes are aggregated. This will enable us to look at the crime indicent trends in both space and time." ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:20 PM\n", "Running script CreateSpaceTimeCube...\n", "WARNING 001605: Distances for Geographic Coordinates (degrees, minutes, seconds) are analyzed using Chordal Distances in meters.\n", "WARNING 110109: 9 of 6849 Input Features could not be aggregated into the Defined Polygon Locations.\n", "WARNING 110110: Features that could not be aggregated (only includes first 30): OBJECTID = 753, 1139, 3778, 3887, 4510, 4915, 5012, 5378, 6549.\n", "WARNING 110067: Your spatial reference is not compatible with CF Conventions. You may experience difficulties using the resulting space-time cube with other NetCDF tools and software.\n", "WARNING 110067: Your spatial reference is not compatible with CF Conventions. You may experience difficulties using the resulting space-time cube with other NetCDF tools and software.\n", "\n", "---------- Space Time Cube Characteristics -----------\n", "Input feature time extent 2017-01-01 00:01:00\n", " to 2017-12-31 22:20:00\n", " \n", "Number of time steps 12\n", "Time step interval 1 month\n", "Time step alignment End\n", " \n", "First time step temporal bias 0.23%\n", "First time step interval after\n", " 2016-12-31 22:20:00\n", " to on or before\n", " 2017-01-31 22:20:00\n", " \n", "Last time step temporal bias 0.00%\n", "Last time step interval after\n", " 2017-11-30 22:20:00\n", " to on or before\n", " 2017-12-31 22:20:00\n", " \n", "Cube extent across space (coordinates in meters)\n", "Min X -7925023.6542\n", "Min Y 5195136.7472\n", "Max X -7898611.5535\n", "Max Y 5220625.6613\n", "\n", "Locations 255\n", "Total observations 3060\n", "\n", "------------- Overall Data Trend - COUNT -------------\n", "Trend direction Not Significant\n", "Trend statistic 0.8248\n", "Trend p-value 0.4095\n", "\n", "Completed script Create Space Time Cube By Aggregating Points...\n", "Succeeded at Friday, March 15, 2019 5:46:22 PM (Elapsed Time: 1.76 seconds)\n" ] } ], "source": [ "# Create output path for spacetime netcdf cube\n", "crime_nc = os.path.join(outputSpace, \"crime_cube.nc\")\n", "\n", "# Create space time cube\n", "space_time_cube_result = arcpy.stpm.CreateSpaceTimeCube(crime_points, crime_nc,\n", " \"DateTime\", None, \"1 Months\", \"END_TIME\", None, None, None,\n", " \"DEFINED_LOCATIONS\", precincts_feature, \"PrecinctWardID\")\n", "print(space_time_cube_result.getMessages())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Emerging Hot Spot Analysis\n", "The [Emerging Hot Spot Analysis Tool](http://pro.arcgis.com/en/pro-app/tool-reference/space-time-pattern-mining/emerginghotspots.htm) identifies trends in the clustering of point counts in a space-time cube. The results include categorizing the temporal trend for each precinct into include new, consecutive, intensifying, persistent, diminishing, sporadic, oscillating, and historical hot and cold spots. This will help us determine locations where crime incidents are on the rise or decreasing and can help with planning enforcement efforts or rating the effecitveness of current programs." ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:22 PM\n", "Running script EmergingHotSpotAnalysis...\n", "WARNING 110020: The default Neighborhood Distance is 1969.46141 meters.\n", "\n", "----------- Input Space Time Cube Details -----------\n", "Time step interval 1 month\n", " \n", "Shape Type Polygon\n", " \n", "First time step temporal bias 0.23%\n", "First time step interval after\n", " 2016-12-31 22:20:00\n", " to on or before\n", " 2017-01-31 22:20:00\n", " \n", "Last time step temporal bias 0.00%\n", "Last time step interval after\n", " 2017-11-30 22:20:00\n", " to on or before\n", " 2017-12-31 22:20:00\n", " \n", "Number of time steps 12\n", "Number of locations analyzed 255\n", "Number of space time bins analyzed 3060\n", "-----------------------------------------------------\n", "\n", "\n", "---------------- Analysis Details ---------------\n", "Neighborhood distance 1969.46141 meters\n", "Neighborhood time step intervals 1\n", "(spanning 1 month) \n", "-------------------------------------------------\n", "\n", "\n", "--------- Summary of Results ---------\n", " HOT COLD\n", "New 0 0\n", "Consecutive 9 0\n", "Intensifying 15 0\n", "Persistent 0 10\n", "Diminishing 0 0\n", "Sporadic 10 6\n", "Oscillating 0 0\n", "Historical 1 0\n", "--------------------------------------\n", "All locations with hot or cold spot trends: 51 of 255\n", "\n", "Category Definitions \n", "-----------------------------------------------------------------------------------------------------------------\n", " \n", "Last time step is hot: \n", "- New: the most recent time step interval is hot for the first time \n", "- Consecutive: a single uninterrupted run of hot time step intervals, comprised of less than 90% of all intervals\n", "- Intensifying: at least 90% of the time step intervals are hot, and becoming hotter over time \n", "- Persistent: at least 90% of the time step intervals are hot, with no trend up or down \n", "- Diminishing: at least 90% of the time step intervals are hot, and becoming less hot over time \n", "- Sporadic: some of the time step intervals are hot \n", "- Oscillating: some of the time step intervals are hot, some are cold \n", " \n", "Last time step is not hot: \n", "- Historical: at least 90% of the time step intervals are hot, but the most recent time step interval is not \n", "-----------------------------------------------------------------------------------------------------------------\n", " \n", "Last time step is cold: \n", "- New: the most recent time step interval is cold for the first time \n", "- Consecutive: a single uninterrupted run of cold time step intervals, comprised of less than 90% of all \n", "- Intensifying: at least 90% of the time step intervals are cold, and becoming colder over time \n", "- Persistent: at least 90% of the time step intervals are cold, with no trend up or down \n", "- Diminishing: at least 90% of the time step intervals are cold, and becoming less cold over time intervals \n", "- Sporadic: some of the time step intervals are cold \n", "- Oscillating: some of the time step intervals are cold, some are hot \n", " \n", "Last time step is not cold: \n", "- Historical: at least 90% of the time step intervals are cold, but the most recent time step interval is not \n", "-----------------------------------------------------------------------------------------------------------------\n", "\n", "Completed script Emerging Hot Spot Analysis...\n", "Succeeded at Friday, March 15, 2019 5:46:24 PM (Elapsed Time: 1.14 seconds)\n" ] } ], "source": [ "# Create output path for emerging hot spot analysis\n", "crime_ehsa = os.path.join(analysis_gdb, 'assault_ehsa')\n", "\n", "# Run the Emerging Hot Spot Analysis tool\n", "ehsa_result = arcpy.stpm.EmergingHotSpotAnalysis(crime_nc, \"COUNT\", crime_ehsa)\n", "print(ehsa_result.getMessages())" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:24 PM\n", "Running script EmergingHotSpotAnalysis...\n", "WARNING 110020: The default Neighborhood Distance is 1969.46141 meters.\n", "\n", "----------- Input Space Time Cube Details -----------\n", "Time step interval 1 month\n", " \n", "Shape Type Polygon\n", " \n", "First time step temporal bias 0.23%\n", "First time step interval after\n", " 2016-12-31 22:20:00\n", " to on or before\n", " 2017-01-31 22:20:00\n", " \n", "Last time step temporal bias 0.00%\n", "Last time step interval after\n", " 2017-11-30 22:20:00\n", " to on or before\n", " 2017-12-31 22:20:00\n", " \n", "Number of time steps 12\n", "Number of locations analyzed 255\n", "Number of space time bins analyzed 3060\n", "-----------------------------------------------------\n", "\n", "\n", "---------------- Analysis Details ---------------\n", "Neighborhood distance 1969.46141 meters\n", "Neighborhood time step intervals 1\n", "(spanning 1 month) \n", "-------------------------------------------------\n", "\n", "\n", "--------- Summary of Results ---------\n", " HOT COLD\n", "New 0 0\n", "Consecutive 9 0\n", "Intensifying 15 0\n", "Persistent 0 10\n", "Diminishing 0 0\n", "Sporadic 10 6\n", "Oscillating 0 0\n", "Historical 1 0\n", "--------------------------------------\n", "All locations with hot or cold spot trends: 51 of 255\n", "\n", "Category Definitions \n", "-----------------------------------------------------------------------------------------------------------------\n", " \n", "Last time step is hot: \n", "- New: the most recent time step interval is hot for the first time \n", "- Consecutive: a single uninterrupted run of hot time step intervals, comprised of less than 90% of all intervals\n", "- Intensifying: at least 90% of the time step intervals are hot, and becoming hotter over time \n", "- Persistent: at least 90% of the time step intervals are hot, with no trend up or down \n", "- Diminishing: at least 90% of the time step intervals are hot, and becoming less hot over time \n", "- Sporadic: some of the time step intervals are hot \n", "- Oscillating: some of the time step intervals are hot, some are cold \n", " \n", "Last time step is not hot: \n", "- Historical: at least 90% of the time step intervals are hot, but the most recent time step interval is not \n", "-----------------------------------------------------------------------------------------------------------------\n", " \n", "Last time step is cold: \n", "- New: the most recent time step interval is cold for the first time \n", "- Consecutive: a single uninterrupted run of cold time step intervals, comprised of less than 90% of all \n", "- Intensifying: at least 90% of the time step intervals are cold, and becoming colder over time \n", "- Persistent: at least 90% of the time step intervals are cold, with no trend up or down \n", "- Diminishing: at least 90% of the time step intervals are cold, and becoming less cold over time intervals \n", "- Sporadic: some of the time step intervals are cold \n", "- Oscillating: some of the time step intervals are cold, some are hot \n", " \n", "Last time step is not cold: \n", "- Historical: at least 90% of the time step intervals are cold, but the most recent time step interval is not \n", "-----------------------------------------------------------------------------------------------------------------\n", "\n", "Completed script Emerging Hot Spot Analysis...\n", "Succeeded at Friday, March 15, 2019 5:46:25 PM (Elapsed Time: 1.30 seconds)\n" ] } ], "source": [ "# Create output path for emerging hot spot analysis\n", "crime_ehsa = os.path.join(analysis_gdb, 'assault_ehsa')\n", "\n", "# Run the Emerging Hot Spot Analysis tool\n", "ehsa_result = arcpy.stpm.EmergingHotSpotAnalysis(crime_nc, \"COUNT\", crime_ehsa)\n", "print(ehsa_result.getMessages())" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDLOCATIONPRECINCTWARDIDCATEGORYPATTERNPERC_HOTPERC_COLDTREND_ZTREND_PTREND_BINSUM_VALUEMIN_VALUEMAX_VALUEMEAN_VALUESTD_VALUEMED_VALUESHAPE
01010No Pattern Detected0.00.0-1.6681150.095293-1.023.00.04.01.9166671.1873171.5{\"rings\": [[[-7905136.2598, 5218873.046099998]...
12120No Pattern Detected0.00.0-1.2372180.2160060.08.00.03.00.6666670.8498370.5{\"rings\": [[[-7907169.264, 5216602.892800003],...
23230No Pattern Detected0.00.0-1.0997490.2714410.020.00.04.01.6666671.1785112.0{\"rings\": [[[-7906706.8160999995, 5216609.0552...
34340No Pattern Detected0.00.02.2736100.0229892.01.00.01.00.0833330.2763850.0{\"rings\": [[[-7910067.2622, 5217262.417300001]...
45450No Pattern Detected0.00.00.4170290.6766570.01.00.01.00.0833330.2763850.0{\"rings\": [[[-7910729.4835, 5218301.749899998]...
\n", "
" ], "text/plain": [ " OBJECTID LOCATION PRECINCTWARDID CATEGORY PATTERN \\\n", "0 1 0 1 0 No Pattern Detected \n", "1 2 1 2 0 No Pattern Detected \n", "2 3 2 3 0 No Pattern Detected \n", "3 4 3 4 0 No Pattern Detected \n", "4 5 4 5 0 No Pattern Detected \n", "\n", " PERC_HOT PERC_COLD TREND_Z TREND_P TREND_BIN SUM_VALUE MIN_VALUE \\\n", "0 0.0 0.0 -1.668115 0.095293 -1.0 23.0 0.0 \n", "1 0.0 0.0 -1.237218 0.216006 0.0 8.0 0.0 \n", "2 0.0 0.0 -1.099749 0.271441 0.0 20.0 0.0 \n", "3 0.0 0.0 2.273610 0.022989 2.0 1.0 0.0 \n", "4 0.0 0.0 0.417029 0.676657 0.0 1.0 0.0 \n", "\n", " MAX_VALUE MEAN_VALUE STD_VALUE MED_VALUE \\\n", "0 4.0 1.916667 1.187317 1.5 \n", "1 3.0 0.666667 0.849837 0.5 \n", "2 4.0 1.666667 1.178511 2.0 \n", "3 1.0 0.083333 0.276385 0.0 \n", "4 1.0 0.083333 0.276385 0.0 \n", "\n", " SHAPE \n", "0 {\"rings\": [[[-7905136.2598, 5218873.046099998]... \n", "1 {\"rings\": [[[-7907169.264, 5216602.892800003],... \n", "2 {\"rings\": [[[-7906706.8160999995, 5216609.0552... \n", "3 {\"rings\": [[[-7910067.2622, 5217262.417300001]... \n", "4 {\"rings\": [[[-7910729.4835, 5218301.749899998]... " ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at the results table\n", "ehsa_sedf = pd.DataFrame.spatial.from_featureclass(crime_ehsa)\n", "\n", "ehsa_sedf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `PATTERN` column indicates the type of pattern detected by the emerging hot spot analysis. We can use this as a group by to get the counts by category." ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "PATTERN\n", "No Pattern Detected 204\n", "Intensifying Hot Spot 15\n", "Sporadic Hot Spot 10\n", "Persistent Cold Spot 10\n", "Consecutive Hot Spot 9\n", "Sporadic Cold Spot 6\n", "Historical Hot Spot 1\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 85, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Group on the pattern column and count the number of precincts\n", "group = ehsa_sedf.groupby('PATTERN').count()['OBJECTID'].sort_values(ascending=False)\n", "group" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Check how much percentage of precincts for which we can detect some pattern through time by running Emerging Hot Spot Analysis tool" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "We can detect pattern for 51 locations out of 255 precincts, that is 20% locations.\n" ] } ], "source": [ "num_total = len(ehsa_sedf)\n", "num_with_pattern = len(ehsa_sedf.loc[ehsa_sedf['PATTERN'] != \"No Pattern Detected\"])\n", "print(\"We can detect pattern for {} locations out of {} precincts, that is {}% locations.\". format(num_with_pattern, num_total, round(num_with_pattern/num_total*100)))" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"345d4af8-2574-43c4-a9b8-4e6c107b53f6\":{\"roots\":{\"references\":[{\"attributes\":{\"plot\":null,\"text\":\"Precincts by Emerging Hotspot Result Pattern\"},\"id\":\"f7e797ab-e100-4352-b292-fef19a6159ad\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"c31e81d2-a7a0-42b6-9e1d-407e914a268b\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"1100a25b-a034-4666-a2fe-673d7adc94da\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"ecf8c17c-5312-4102-8adf-993118625819\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"formatter\":{\"id\":\"c31e81d2-a7a0-42b6-9e1d-407e914a268b\",\"type\":\"BasicTickFormatter\"},\"major_label_orientation\":0.39269908169872414,\"major_label_overrides\":{\"0\":\"No Pattern Detected\",\"1\":\"Intensifying Hot Spot\",\"2\":\"Sporadic Hot Spot\",\"3\":\"Persistent Cold Spot\",\"4\":\"Consecutive Hot Spot\",\"5\":\"Sporadic Cold Spot\",\"6\":\"Historical Hot Spot\"},\"plot\":{\"id\":\"10153dd8-a6d2-4085-9be8-8d122d88ced6\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"561d76d1-ed4e-428c-a940-29792854764b\",\"type\":\"BasicTicker\"}},\"id\":\"d03ae72d-9408-495f-a1e4-58c21e28a3e2\",\"type\":\"LinearAxis\"},{\"attributes\":{\"desired_num_ticks\":10},\"id\":\"561d76d1-ed4e-428c-a940-29792854764b\",\"type\":\"BasicTicker\"},{\"attributes\":{},\"id\":\"2d611e96-4a2d-41a0-9a16-25ec8e7f1c2e\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\"},\"id\":\"2bebd292-366a-4d5b-82d9-384174b06bf1\",\"type\":\"Toolbar\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.8},\"x\":{\"field\":\"x\"}},\"id\":\"07d591d8-a8bb-4875-a5cb-d451727d293d\",\"type\":\"VBar\"},{\"attributes\":{\"fill_color\":{\"value\":\"#1f77b4\"},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.8},\"x\":{\"field\":\"x\"}},\"id\":\"feb93139-0b9b-42ea-91bb-af54494bc261\",\"type\":\"VBar\"},{\"attributes\":{\"data_source\":{\"id\":\"f19ff14e-9862-4162-8974-4ce0457fceaa\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"feb93139-0b9b-42ea-91bb-af54494bc261\",\"type\":\"VBar\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"07d591d8-a8bb-4875-a5cb-d451727d293d\",\"type\":\"VBar\"},\"selection_glyph\":null,\"view\":{\"id\":\"9e606002-d65b-4158-a51b-e62274153c2a\",\"type\":\"CDSView\"}},\"id\":\"3636cab2-833e-4180-b825-0be51abee2b9\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null},\"id\":\"39dbcce1-1d4c-4b0a-9964-0099374928ec\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"c5a52428-f4f2-46f8-b536-d2f44e0856ae\",\"type\":\"LinearScale\"},{\"attributes\":{\"callback\":null},\"id\":\"6a3915d4-3a90-40e7-b1a0-8296a5da0f00\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"7e8cea44-76a4-4183-a98f-10f742646a67\",\"type\":\"LinearScale\"},{\"attributes\":{\"plot\":{\"id\":\"10153dd8-a6d2-4085-9be8-8d122d88ced6\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"561d76d1-ed4e-428c-a940-29792854764b\",\"type\":\"BasicTicker\"}},\"id\":\"8b8f8e9d-1709-4879-9004-2f1b9db757e0\",\"type\":\"Grid\"},{\"attributes\":{\"formatter\":{\"id\":\"ecf8c17c-5312-4102-8adf-993118625819\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"10153dd8-a6d2-4085-9be8-8d122d88ced6\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"dbe76fef-c94c-412f-8f06-540140cf3503\",\"type\":\"BasicTicker\"}},\"id\":\"fb44f8f2-c140-4d8f-b1e5-26cfb73b05a0\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"dbe76fef-c94c-412f-8f06-540140cf3503\",\"type\":\"BasicTicker\"},{\"attributes\":{\"below\":[{\"id\":\"d03ae72d-9408-495f-a1e4-58c21e28a3e2\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"fb44f8f2-c140-4d8f-b1e5-26cfb73b05a0\",\"type\":\"LinearAxis\"}],\"min_border_left\":80,\"plot_height\":250,\"renderers\":[{\"id\":\"d03ae72d-9408-495f-a1e4-58c21e28a3e2\",\"type\":\"LinearAxis\"},{\"id\":\"8b8f8e9d-1709-4879-9004-2f1b9db757e0\",\"type\":\"Grid\"},{\"id\":\"fb44f8f2-c140-4d8f-b1e5-26cfb73b05a0\",\"type\":\"LinearAxis\"},{\"id\":\"90503993-7b47-4484-ab16-230b1eb24d53\",\"type\":\"Grid\"},{\"id\":\"3636cab2-833e-4180-b825-0be51abee2b9\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"f7e797ab-e100-4352-b292-fef19a6159ad\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"2bebd292-366a-4d5b-82d9-384174b06bf1\",\"type\":\"Toolbar\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"6a3915d4-3a90-40e7-b1a0-8296a5da0f00\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"c5a52428-f4f2-46f8-b536-d2f44e0856ae\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"39dbcce1-1d4c-4b0a-9964-0099374928ec\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"7e8cea44-76a4-4183-a98f-10f742646a67\",\"type\":\"LinearScale\"}},\"id\":\"10153dd8-a6d2-4085-9be8-8d122d88ced6\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"10153dd8-a6d2-4085-9be8-8d122d88ced6\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"dbe76fef-c94c-412f-8f06-540140cf3503\",\"type\":\"BasicTicker\"}},\"id\":\"90503993-7b47-4484-ab16-230b1eb24d53\",\"type\":\"Grid\"},{\"attributes\":{\"callback\":null,\"data\":{\"top\":[204,15,10,10,9,6,1],\"x\":[0,1,2,3,4,5,6]},\"selected\":{\"id\":\"1100a25b-a034-4666-a2fe-673d7adc94da\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"2d611e96-4a2d-41a0-9a16-25ec8e7f1c2e\",\"type\":\"UnionRenderers\"}},\"id\":\"f19ff14e-9862-4162-8974-4ce0457fceaa\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"f19ff14e-9862-4162-8974-4ce0457fceaa\",\"type\":\"ColumnDataSource\"}},\"id\":\"9e606002-d65b-4158-a51b-e62274153c2a\",\"type\":\"CDSView\"}],\"root_ids\":[\"10153dd8-a6d2-4085-9be8-8d122d88ced6\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"345d4af8-2574-43c4-a9b8-4e6c107b53f6\",\"roots\":{\"10153dd8-a6d2-4085-9be8-8d122d88ced6\":\"fccc7d40-0f1e-41f8-b0f0-32eaefff2929\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "10153dd8-a6d2-4085-9be8-8d122d88ced6" } }, "output_type": "display_data" } ], "source": [ "# Visualize on bar chart \n", "p = figure(plot_height=250, title=\"Precincts by Emerging Hotspot Result Pattern\",\n", " toolbar_location=None, tools=\"\")\n", "\n", "x_axis_values = np.arange(len(group.index)) # How many unique patterns are contained in the result\n", "p.vbar(x=x_axis_values, top=group.values, width=0.8)\n", "\n", "\n", "# Way to custom label the chart with dates\n", "label_dict = {}\n", "for idx, cur_label in enumerate(x_axis_values):\n", " label_dict[idx] = group.index[idx]\n", "p.xaxis.major_label_overrides = label_dict\n", "\n", "p.xaxis[0].ticker.desired_num_ticks = 10\n", "p.xaxis.major_label_orientation = pi/8\n", "\n", "# pad the plot\n", "p.min_border_left = 80\n", "\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This plot shows again that the majority of the precinct wards do not show a distinct pattern. Precincts which are intensifying hot spots could be targets for intervention programs. The persistent cold spots reveal areas where crime incidents are low, these areas could be investigated for factors which lead to decreased incidents." ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
PRECINCTWARD_INDEXPRECINCTWARD_ID
010113
120102
230105
340203
450205
\n", "
" ], "text/plain": [ " PRECINCTWARD_INDEX PRECINCTWARD_ID\n", "0 1 0113\n", "1 2 0102\n", "2 3 0105\n", "3 4 0203\n", "4 5 0205" ] }, "execution_count": 88, "metadata": {}, "output_type": "execute_result" } ], "source": [ "precinct_lookup_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can merge the results to the lookup table to identify the preicncts which are the intensifying hot spot pattern and the persistent cold spot pattern." ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Precints showing Intensifying Hot Spot Pattern:\n", "['0303', '0307', '0504', '0503', '0902', '0803', '0903', '0807', '0306', '0302', '0308', '0801', '0511', '0505', '0301']\n", "Precints showing Persistent Cold Spot Pattern:\n", "['1908', '2007', '2019', '2012', '2014', '2018', '1909', '2208', '2011', '2008']\n" ] } ], "source": [ "intensifying_hotspot_df = ehsa_sedf[ehsa_sedf['PATTERN'] == \"Intensifying Hot Spot\"]\n", "intensifying_precincts = list(pd.DataFrame.merge(intensifying_hotspot_df, precinct_lookup_df, \n", " left_on='PRECINCTWARDID', right_on='PRECINCTWARD_INDEX')['PRECINCTWARD_ID'])\n", "\n", "presistent_cold_df = ehsa_sedf[ehsa_sedf['PATTERN'] == \"Persistent Cold Spot\"]\n", "persistent_cold_precincts = list(pd.DataFrame.merge(presistent_cold_df, precinct_lookup_df, \n", " left_on='PRECINCTWARDID', right_on='PRECINCTWARD_INDEX')['PRECINCTWARD_ID'])\n", "\n", "print(\"Precints showing Intensifying Hot Spot Pattern:\\n{}\".format(intensifying_precincts))\n", "print(\"Precints showing Persistent Cold Spot Pattern:\\n{}\".format(persistent_cold_precincts))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualize Space Time Cube in 3D\n", "\n", "The [Visualize Space Time Cube in 3D\n", "](http://pro.arcgis.com/en/pro-app/tool-reference/space-time-pattern-mining/visualizecube3d.htm) tool creates a feature class with a point for each location and each time slice. This allows you to visualize the crime incident counts over time in a 3D scene.\n" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:26 PM\n", "Running script VisualizeSpaceTimeCube3D...\n", "WARNING 110044: The time it takes to render the cube in three dimensions may vary considerably based on the number of features and the graphics card associated with your CPU.\n", "Completed script Visualize Space Time Cube in 3D...\n", "Succeeded at Friday, March 15, 2019 5:46:27 PM (Elapsed Time: 1.10 seconds)\n" ] } ], "source": [ "# Create output path for Visualize Space Time Cube in 3D\n", "crime_viz3d = os.path.join(analysis_gdb, 'assault_viz3d')\n", "# Run Visualize Space Time Cube in 3D\n", "viz3d_results = arcpy.stpm.VisualizeSpaceTimeCube3D(crime_nc, \"COUNT\", \"VALUE\", crime_viz3d)\n", "\n", "print(viz3d_results.getMessages())" ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 91, "metadata": {}, "output_type": "execute_result" } ], "source": [ "wgs_sr = arcpy.Describe('/arcgis/home/BostonCrimeAnalysis.gdb/hotspot_result').spatialReference\n", "crime_viz3d_wgs = os.path.join(analysis_gdb, 'assault_viz3d_wgs')\n", "arcpy.management.Project(crime_viz3d, crime_viz3d_wgs, wgs_sr)" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDTIME_STEPSTART_DATEEND_DATETIME_EXAGELEMENTLOCATIONPRECINCTWARDIDVALUESHAPE
0102016-12-31 22:20:012017-01-31 22:20:000.0000000013.0{'x': -71.00969216799996, 'y': 42.388864279000...
1212017-01-31 22:20:012017-02-28 22:20:0052.824201255014.0{'x': -71.00969216799996, 'y': 42.388864279000...
2322017-02-28 22:20:012017-03-31 22:20:00105.648403510011.0{'x': -71.00969216799996, 'y': 42.388864279000...
3432017-03-31 22:20:012017-04-30 22:20:00158.472604765013.0{'x': -71.00969216799996, 'y': 42.388864279000...
4542017-04-30 22:20:012017-05-31 22:20:00211.2968061020011.0{'x': -71.00969216799996, 'y': 42.388864279000...
\n", "
" ], "text/plain": [ " OBJECTID TIME_STEP START_DATE END_DATE TIME_EXAG \\\n", "0 1 0 2016-12-31 22:20:01 2017-01-31 22:20:00 0.000000 \n", "1 2 1 2017-01-31 22:20:01 2017-02-28 22:20:00 52.824201 \n", "2 3 2 2017-02-28 22:20:01 2017-03-31 22:20:00 105.648403 \n", "3 4 3 2017-03-31 22:20:01 2017-04-30 22:20:00 158.472604 \n", "4 5 4 2017-04-30 22:20:01 2017-05-31 22:20:00 211.296806 \n", "\n", " ELEMENT LOCATION PRECINCTWARDID VALUE \\\n", "0 0 0 1 3.0 \n", "1 255 0 1 4.0 \n", "2 510 0 1 1.0 \n", "3 765 0 1 3.0 \n", "4 1020 0 1 1.0 \n", "\n", " SHAPE \n", "0 {'x': -71.00969216799996, 'y': 42.388864279000... \n", "1 {'x': -71.00969216799996, 'y': 42.388864279000... \n", "2 {'x': -71.00969216799996, 'y': 42.388864279000... \n", "3 {'x': -71.00969216799996, 'y': 42.388864279000... \n", "4 {'x': -71.00969216799996, 'y': 42.388864279000... " ] }, "execution_count": 92, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a spatially enabled dataframe with the results\n", "crime_viz3d_sedf = SpatialDataFrame.from_featureclass(crime_viz3d_wgs)\n", "crime_viz3d_sedf.head()" ] }, { "cell_type": "code", "execution_count": 138, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2d075276a9d345f5aff82ef74c380594", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(jupyter_target='notebook', layout=Layout(height='400px', width='100%'), mode='3D', ready=True, tilt=49…" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Create a map in 3D mode to visualize the results\n", "viz3d_map = gis.map(\"Boston\", zoomlevel=13, mode='3D')\n", "viz3d_map.basemap='dark-gray-vector'\n", "viz3d_map" ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 94, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Change 3D scene angle\n", "viz3d_map.tilt = 50\n", "# Add data\n", "# TODO: Figure out how to intepret the VALUE field and how the layer file works for this tool\n", "crime_viz3d_sedf.spatial.plot(viz3d_map, col=\"VALUE\", cmap=\"Reds\", \n", " renderer_type='c', alpha=0.75, \n", " method='esriClassifyNaturalBreaks', line_width=0.25)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Time Series Clustering\n", "\n", "The [Time Series Clustering](http://pro.arcgis.com/en/pro-app/tool-reference/space-time-pattern-mining/time-series-clustering.htm) tool partitions a collection of time series, stored in a space-time cube, based on the similarity of time series characteristics. Time series can be clustered so they have similar values in time or similar behaviors or profiles across time (increase or decrease at the same points in time).\n", "\n", "By clustering on the incident time series areas with similar temporal patterns can be identified. As an example if some precincts had the most activity in the morning hours and others had the most activity in the evenings it would create two distinct clusters. By understanding the temporal patterns police efforts can be timed to maximize their efficiency. There are two modes for time series clustering: VALUE and PROFILE. By using VALUE, locations that have time series with similar values of the Analysis Variable at the same points in time will be clustered together. By using PROFILE, time series that have similar behaviors, fluctuations, and complexity will be clustered together. Clustering is based on the common form or shape of each location's time series. We will examine using the value mode in this analysis to find clusters with similar number of incidents." ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Start Time: Friday, March 15, 2019 5:46:33 PM\n", "Running script TimeSeriesClustering...\n", "\n", "The estimated optimal number of clusters is 3 based on the spectral gap heuristic.\n", "\n", "----------- Input Space Time Cube Details -----------\n", "Time step interval 1 month\n", " \n", "Shape Type Polygon\n", " \n", "First time step temporal bias 0.23%\n", "First time step interval after\n", " 2016-12-31 22:20:00\n", " to on or before\n", " 2017-01-31 22:20:00\n", " \n", "Last time step temporal bias 0.00%\n", "Last time step interval after\n", " 2017-11-30 22:20:00\n", " to on or before\n", " 2017-12-31 22:20:00\n", " \n", "Number of time steps 12\n", "Number of locations analyzed 255\n", "Number of space time bins analyzed 3060\n", "-----------------------------------------------------\n", "\n", "\n", "\n", "------ Trend Statistics for Average Time-Series ------\n", "Cluster ID Direction Statistic p-value\n", "1 Not Significant 1.0286 0.3037\n", "2 Not Significant 1.5122 0.1305\n", "3 Not Significant 1.0286 0.3037\n", "------------------------------------------------------\n", "\n", "Completed script Time Series Clustering...\n", "Succeeded at Friday, March 15, 2019 5:46:34 PM (Elapsed Time: 0.97 seconds)\n" ] } ], "source": [ "# Create output path for Time Series Clustering results\n", "time_series_clustering = os.path.join(analysis_gdb, 'assault_time_series_clustering')\n", "\n", "# Run time series clustering based on VALUE\n", "time_series_clustering_results = arcpy.stpm.TimeSeriesClustering(crime_nc, \"COUNT\", time_series_clustering, \"VALUE\")\n", "\n", "print(time_series_clustering_results.getMessages())" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ERROR 110191: The spectral gap decomposition returned an invalid number of clusters in your data (0) which means there are no meaningful clusters.\n", "Failed to execute (TimeSeriesClustering).\n", "\n" ] } ], "source": [ "# Apart from use VALUE similarity, we can also run time series clustering based on PROFILE(cosine) similarity\n", "time_series_clustering_profile = os.path.join(analysis_gdb, 'assault_time_series_clustering_profile')\n", "try:\n", " time_series_clustering_profile_results = arcpy.stpm.TimeSeriesClustering(crime_nc, \"COUNT\", time_series_clustering_profile, \"PROFILE\") # change 2\n", "except Exception:\n", " e = sys.exc_info()[1]\n", " print(e.args[0])\n", " \n", "# it shows that there is no significant time series clusters based on profile similarity" ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDLOCATIONPRECINCTWARDIDCLUSTER_IDCENTER_REPSHAPE
010110{\"rings\": [[[-7905136.2598, 5218873.046099998]...
121210{\"rings\": [[[-7907169.264, 5216602.892800003],...
232310{\"rings\": [[[-7906706.8160999995, 5216609.0552...
343410{\"rings\": [[[-7910067.2622, 5217262.417300001]...
454510{\"rings\": [[[-7910729.4835, 5218301.749899998]...
\n", "
" ], "text/plain": [ " OBJECTID LOCATION PRECINCTWARDID CLUSTER_ID CENTER_REP \\\n", "0 1 0 1 1 0 \n", "1 2 1 2 1 0 \n", "2 3 2 3 1 0 \n", "3 4 3 4 1 0 \n", "4 5 4 5 1 0 \n", "\n", " SHAPE \n", "0 {\"rings\": [[[-7905136.2598, 5218873.046099998]... \n", "1 {\"rings\": [[[-7907169.264, 5216602.892800003],... \n", "2 {\"rings\": [[[-7906706.8160999995, 5216609.0552... \n", "3 {\"rings\": [[[-7910067.2622, 5217262.417300001]... \n", "4 {\"rings\": [[[-7910729.4835, 5218301.749899998]... " ] }, "execution_count": 97, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load time series clustering results to a dataframe\n", "time_series_clustering_sedf = pd.DataFrame.spatial.from_featureclass(time_series_clustering)\n", "time_series_clustering_sedf.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Each of the precincts has been given a cluster ID to identifiy clusters with similar time series. Using a groupby on the CLUSTER_ID field lets us count how many precincts are in each cluster. We can also use the CLUSTER_ID to map out locations with similar time series trends." ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "CLUSTER_ID\n", "1 219\n", "2 2\n", "3 34\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 98, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Look at the number of precincts in each cluster\n", "time_series_clustering_sedf.groupby('CLUSTER_ID').count()['OBJECTID']" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAABKCAYAAAC/3fOeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvDW2N/gAACK5JREFUeJzt3X+s1XUdx/HnC1FEr4CYmSjpyvghTElSZ2iyYRaZ2hZpRRqYhs7WZq5cpYV/kLUxQVDHMn+mU0hzI3XFZQuWMbJoaDCzpJFXSfIKKN6QQt/98fkcO5/jPfee3Lnn3nlfj+1s557P53w+n+/Z3ff1/X6+3/M5igjMzMwqhvT3AMzMbGBxMJiZWcHBYGZmBQeDmZkVHAxmZlZwMJiZWcHBYAOGpJB0XD+PYY6kx/uo7fmS7u2Lts2aycFgTSXpdEnrJL0iaYek30o6uYX9r5F0aR+2f0Dewf9VUpekrZLukHRsX/XZw1hC0nZJQ6teGyrpn5L8BSV7xxwM1jSSRgCPAEuB0cBRwPXA3v4cV5M9CJwHfBEYCZwIbABm9FWH1Tv+buwCZlb9/SlgZ1+NxQYHB4M10ziAiLg/It6IiD0RsSoinqpUkHSJpKcl7ZT0K0nHdNeQpGGSFkp6Lh8VL5M0vKr8fEkbJb0qaYukT0paAJwB3CzpNUk357oTJLXnM5hnJF1Q1c5hklbmdp4APlhv4ySdBXwcOD8ifh8R+yLilYi4JSJuz3XG5PZ2SHpW0mU9tHeepM2SduUznYlVZVslXSPpKaCrh3D4KXBx1d8XA/fU9DM3f+a7Jf1N0ryqsumSnpf0HUmdud/Z9cZsg0RE+OFHUx7ACOBl4G7SUeyhNeWfAZ4FJgJDgWuBdVXlARyXny8GVpLOPA4BfgHckMtOAV4h7aSHkM5MJuSyNcClVW0eDHQAc3OfJwGdwKRc/gCwItebDLwAPF5n+34IrO3lM1gL3AocCEwBXgJm5LL5wL35+TigK2/D/sC38mdzQC7fCmwExgLD6/QVeczbgVH5sT2/FlX1ziEFnoAzgX8BJ+Wy6cA+4EZgWC7vAsb39/+TH/338BmDNU1EvAqcTtph3Qa8lI+ej8hV5pF27k9HxD7gB8CU2rMGSQIuA66KiB0RsTvX/Xyu8hXgjohoj4g3I+KFiPhznWF9GtgaEXdGOsL/I/AQMEvSfsBnge9FRFdEbCKFWj2HAf+oVyhpbN7+ayLi9YjYCPwEuKib6hcCj+Zt+A+wEBgOfLSqzpKI6IiIPT2M6XVSaF5I+nxW5tfeEhGPRsSWSNYCq0hnVtWui4i9ufxR4AJs0HIwWFPlnf6ciDiadOQ6hnT0D3AMcFOeOtkF7CAdxR5V08zhwEHAhqq6v8yvQzqK3tLgkI4BTq20k9uaDbwvtzeUdEZR8fce2noZOLKH8jFAJciq26vdvkrdt/qKiDfzOKrrdtS+qY57SFNIb5tGApA0U9L6PL21i3Qd4j1VVXZGRFfNmMc02Le9CzkYrM/ko/i7SAEBaUc3LyJGVT2GR8S6mrd2AntI0z2VeiMjoq2qnXrXAmrvxukgTf9U99kWEVeQpnn2kYKm4v09bNJq4BRJR9cp3waMlnRITXsv1Kn71plSPksaW1O30TuLfkMKrCOA4lZbScNIZ0gLgSMiYhTwGCmQKw6VdHDNmLc12Le9CzkYrGnyRd6rKzvOPLXyBWB9rrIM+LakSbl8pKTP1baTj55vAxZJem+ue5SkT+QqtwNzJc2QNCSXTchl24EPVDX3CDBO0kWS9s+PkyVNjIg3gJ8D8yUdJOl44Mv1ti8iVgPtwMOSpuZbQw+RdLmkSyKiA1gH3CDpQEknkKa97uumuRXAOXkb9geuJt29VRuSvYqIAM4FzsvPqx1AunbwErBP0kzg7G6auT7finsGafrtZ//vOOzdw8FgzbQbOBX4naQuUiBsIu30iIiHgR8BD0h6NZfNrNPWNaSLsetz3dXA+NzOE6SLyYtIF6HX8r+j75tI1w92SlqSp3XOJs2/bwNezGMYlut/DWjLr98F3NnLNs4iHXEvz31vAj6SxwcpCI/NfT0MfD8i2msbiYhngC+Rbu3tJO3Yz42If/fSf7ciYnNEbO7m9d3A10lBtJN0m+3Kmmov5rJtpBC7vIdrNjYI6O0HGGY2WEiaTrpTqt70mA1CPmMwM7OCg8HMzAqeSjIzs4LPGMzMrOBgMDOzQk+rNrbUjZvWt3xO6xuTO1va36/bx7e0vzPWLWtpfwALPjy35X1et9+ilvb32PHDeq/URDN3XtvS/ra0vdbS/gA67n++5X2OuvLAlvY3dciKlvYHwOjF6r3S2zV0xiDpu3kVyKfyipan9lB3jqQxVX/fLunJ/N4HJbXVe6+ZmfW/XoNB0mmkb0KeFBEnAGfR8xoucyjXWbkqIk7M732O9IUiMzMboBqZSjoS6IyIvQAR0QkgaSppqd420jc35wDTSN8CvU/SHuC0vOJmZS2Y4TS+/ouZmfWDRqaSVgFjJf1F0q2SzsxruywFZkXEVOAOYEFEPAj8AZgdEVMqywVLupP0tfsJ+X1mZjZA9RoMEfEaMBX4KmkhruWkdfUnA+2SNpJ+cKXuV+ojYi5peulp0rrxZmY2QDV0V1JehXINsEbSn4Argc0RcVqjHUXEG5KWA9+k94XKzMysnzRy8Xm8pA9VvTSFdOR/eL4wTV7KeFIu3036KUaUHFd5TlpB0qs2mpkNYI2cMbQBSyWNIv2oybOkaaUfA0skjcztLAY2k5YuXpYvPk8D7pY0gvTDIE8CVzR7I8zMrHl6DYaI2ED5O7QVncDHuqn/EOkXoyqmvePRmZlZy3lJDDMzKzgYzMys4GW3zcys4DMGMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzgYDAzs4KDwczMCg4GMzMrOBjMzKzwXzE/KjRdTOz3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "# Visualize the Time Series Clusters\n", "# Now we can try a different rendering technique: create a layer file based on an existing color map from MATPLOTLIB\n", "# Tips 1: color map is a collection of string values that can be given to generate a series of related colors from a defined set.\n", "# Tips 2: matplotlib color map can be viewed here: https://matplotlib.org/tutorials/colors/colormaps.html\n", "\n", "# Here we will use quantitative colormaps: often miscellaneous colors and are good for representing information which does not have ordering or relationships.\n", "# In this case we use \"Set3\", you can try other color maps as well\n", "chosen_colormap = \"Set3\"\n", "# cmaps['Qualitative'] = ['Pastel1', 'Pastel2', 'Paired', 'Accent',\n", "# 'Dark2', 'Set1', 'Set2', 'Set3',\n", "# 'tab10', 'tab20', 'tab20b', 'tab20c']\n", "\n", "# Check the color with arcgis.mapping library\n", "from arcgis.mapping import display_colormaps\n", "display_colormaps([chosen_colormap])" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "3" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# so there are 12 different colors in this colormap, change this if you use other colormap\n", "num_of_color_available = 12\n", "\n", "# define how many distinct color we need, basically, it's the number of clusters identified by running the analysis, if it's less than number of color available\n", "# otherwise, it will be the number of color available\n", "num_of_clusters = len(time_series_clustering_sedf[\"CLUSTER_ID\"].unique())\n", "num_of_color = min(num_of_color_available, num_of_clusters)\n", "num_of_color" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[141., 211., 199., 255.],\n", " [179., 222., 105., 255.],\n", " [255., 237., 111., 255.]])" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# get a list of [R,G,B,Opacity] from the chosen colormap\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "# change Set3 to the color map of your choice if necessary\n", "color_list = plt.cm.Set3(np.linspace(0, 1, num_of_color)) * 255\n", "color_list" ] }, { "cell_type": "code", "execution_count": 102, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{'label': 'cluster 1',\n", " 'symbol': {'type': 'esriSFS',\n", " 'color': [141.0, 211.0, 199.0, 255.0],\n", " 'outline': {'type': 'esriSLS',\n", " 'color': [0, 0, 0, 150],\n", " 'width': 0.7,\n", " 'style': 'esriSLSSolid'},\n", " 'style': 'esriSFSSolid'},\n", " 'value': 1},\n", " {'label': 'cluster 2',\n", " 'symbol': {'type': 'esriSFS',\n", " 'color': [179.0, 222.0, 105.0, 255.0],\n", " 'outline': {'type': 'esriSLS',\n", " 'color': [0, 0, 0, 150],\n", " 'width': 0.7,\n", " 'style': 'esriSLSSolid'},\n", " 'style': 'esriSFSSolid'},\n", " 'value': 2},\n", " {'label': 'cluster 3',\n", " 'symbol': {'type': 'esriSFS',\n", " 'color': [255.0, 237.0, 111.0, 255.0],\n", " 'outline': {'type': 'esriSLS',\n", " 'color': [0, 0, 0, 150],\n", " 'width': 0.7,\n", " 'style': 'esriSLSSolid'},\n", " 'style': 'esriSFSSolid'},\n", " 'value': 0}]" ] }, "execution_count": 102, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create a \"layer file\" thing based on the chosen colormap\n", "# Tip: It is a good cartographic practice to not have too many unique values in your legend.\n", "\n", "outline_render = {\"type\":\"esriSLS\",\n", " \"color\":[0,0,0,150],\n", " \"width\":0.7,\n", " \"style\":\"esriSLSSolid\"}\n", "\n", "time_series_cluster_layer = []\n", "\n", "for i in range(0, num_of_color):\n", " label = \"cluster \" + str(i+1)\n", " symbol = {\"type\":\"esriSFS\",\n", " \"color\":color_list[i].tolist(),\n", " \"outline\":outline_render,\n", " \"style\":\"esriSFSSolid\"}\n", " value = (i + 1) % num_of_color # this is for handeling the scenario where number of cluster is larger than the number of color available in the colormap\n", " symbology = {\"label\":label, \"symbol\":symbol, \"value\":value}\n", " time_series_cluster_layer.append(symbology)\n", "\n", "# check the symbology\n", "time_series_cluster_layer" ] }, { "cell_type": "code", "execution_count": 103, "metadata": {}, "outputs": [], "source": [ "# Second, create an arcade expression variable using the CLUSTER_ID\n", "# This will help us handel the scenario where number of clusters is larger than the number of color available in the colormap\n", "arcade = \"$feature.CLUSTER_ID % \" + str(num_of_color);" ] }, { "cell_type": "code", "execution_count": 140, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "19b2a539fbe54e4f8649a277a615acde", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(jupyter_target='notebook', layout=Layout(height='400px', width='100%'), ready=True, zoom=10.0)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Finally, Plot the precincts using unique value renderer on CLUSTER_ID\n", "time_series_cluster_map = gis.map(\"Boston\")\n", "time_series_cluster_map.basemap ='dark-gray' # Let's try another basemap\n", "time_series_cluster_map" ] }, { "cell_type": "code", "execution_count": 105, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# remove existing layers if you have try this for several times\n", "time_series_cluster_map.remove_layers()\n", "\n", "# do the plot with arcade expression and layer info\n", "time_series_clustering_sedf.spatial.plot(time_series_cluster_map, \n", " col = \"CLUSTER_ID\",\n", " renderer_type='u-a', # 'u-a' stands for uniqe value with arcade expression\n", " unique_values=time_series_cluster_layer,\n", " arcade_expression=arcade)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Examine the Time Series Trends\n", "\n", "We have examined the locations of the time series clusters. Now we will take a look at what the actual temporal patterns are throughout the year for each of the clusters. We will groupby using the cluster ID and then calculate the minimum, maximum and mean value for the time series and use the values in a plot.\n", "\n", "We will merge the time series clusteirng results using CLUSTER_ID with the aggregated point count data so we can look at the results\n", "from each of the clusters" ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "There are 3 time series clusters for each month.\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
START_DATECLUSTER_IDVALUE
02016-12-31 22:20:0118.0
12016-12-31 22:20:01225.0
22016-12-31 22:20:01313.0
32017-01-31 22:20:0117.0
42017-01-31 22:20:01230.0
52017-01-31 22:20:01316.0
\n", "
" ], "text/plain": [ " START_DATE CLUSTER_ID VALUE\n", "0 2016-12-31 22:20:01 1 8.0\n", "1 2016-12-31 22:20:01 2 25.0\n", "2 2016-12-31 22:20:01 3 13.0\n", "3 2017-01-31 22:20:01 1 7.0\n", "4 2017-01-31 22:20:01 2 30.0\n", "5 2017-01-31 22:20:01 3 16.0" ] }, "execution_count": 106, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Join the cluster ID to each location in the visualize 3D results.\n", "join_df = crime_viz3d_sedf.merge(time_series_clustering_sedf, on='LOCATION')\n", "\n", "# Group by the start date and the cluster ID so we can get the number of crimes for each month for each cluster\n", "gdf = join_df.groupby(['START_DATE', 'CLUSTER_ID'])\n", "\n", "# Get the number of cluster ids\n", "num_clusters = gdf['VALUE'].max().reset_index()['CLUSTER_ID'].unique()\n", "print(\"There are {} time series clusters for each month.\".format(max(num_clusters)))\n", "\n", "# Calculate summary statistics for each cluster\n", "max_values = gdf['VALUE'].max().reset_index()\n", "max_values.head(6)" ] }, { "cell_type": "code", "execution_count": 107, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
START_DATECLUSTER_IDVALUE
02016-12-31 22:20:0110.0
12016-12-31 22:20:01216.0
22016-12-31 22:20:0131.0
32017-01-31 22:20:0110.0
42017-01-31 22:20:01211.0
52017-01-31 22:20:0130.0
\n", "
" ], "text/plain": [ " START_DATE CLUSTER_ID VALUE\n", "0 2016-12-31 22:20:01 1 0.0\n", "1 2016-12-31 22:20:01 2 16.0\n", "2 2016-12-31 22:20:01 3 1.0\n", "3 2017-01-31 22:20:01 1 0.0\n", "4 2017-01-31 22:20:01 2 11.0\n", "5 2017-01-31 22:20:01 3 0.0" ] }, "execution_count": 107, "metadata": {}, "output_type": "execute_result" } ], "source": [ "min_values = gdf['VALUE'].min().reset_index()\n", "min_values.head(6)" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
START_DATECLUSTER_IDVALUE
02016-12-31 22:20:0111.410959
12016-12-31 22:20:01220.500000
22016-12-31 22:20:0135.235294
32017-01-31 22:20:0111.232877
42017-01-31 22:20:01220.500000
52017-01-31 22:20:0134.647059
\n", "
" ], "text/plain": [ " START_DATE CLUSTER_ID VALUE\n", "0 2016-12-31 22:20:01 1 1.410959\n", "1 2016-12-31 22:20:01 2 20.500000\n", "2 2016-12-31 22:20:01 3 5.235294\n", "3 2017-01-31 22:20:01 1 1.232877\n", "4 2017-01-31 22:20:01 2 20.500000\n", "5 2017-01-31 22:20:01 3 4.647059" ] }, "execution_count": 108, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_values = gdf['VALUE'].mean().reset_index()\n", "mean_values.head(6)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this plot we will be using both a line plot as well as a [patch](https://bokeh.pydata.org/en/latest/docs/reference/models/glyphs/patches.html). \n", "Basically patch will be the range for each month, and the line will be the mean for each month." ] }, { "cell_type": "code", "execution_count": 109, "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Working on cluster 1\n", "Working on cluster 2\n", "Working on cluster 3\n", "\n", "\n", "patch_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0]\n", "patch_y : [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 7.0, 7.0, 8.0, 9.0, 7.0, 9.0, 8.0, 8.0, 8.0, 7.0, 7.0, 8.0, 0.0]\n", "mean_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n", "mean_y : [1.41, 1.23, 1.27, 1.49, 1.53, 1.63, 1.47, 1.58, 1.59, 1.56, 1.39, 1.46]\n", "\n", "\n", "patch_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0]\n", "patch_y : [16.0, 11.0, 19.0, 18.0, 21.0, 12.0, 26.0, 30.0, 19.0, 27.0, 27.0, 10.0, 34.0, 37.0, 29.0, 32.0, 37.0, 40.0, 27.0, 32.0, 25.0, 33.0, 30.0, 25.0, 16.0]\n", "mean_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n", "mean_y : [20.5, 20.5, 26.0, 21.5, 26.5, 19.5, 33.0, 33.5, 25.5, 28.0, 32.0, 22.0]\n", "\n", "\n", "patch_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 0]\n", "patch_y : [1.0, 0.0, 0.0, 0.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 18.0, 12.0, 15.0, 24.0, 15.0, 16.0, 14.0, 14.0, 17.0, 14.0, 16.0, 13.0, 1.0]\n", "mean_x : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]\n", "mean_y : [5.24, 4.65, 5.21, 5.44, 6.71, 6.24, 6.38, 5.97, 6.06, 6.12, 5.62, 5.85]\n" ] } ], "source": [ "plot_data_list = []\n", "\n", "for cur_cluster in num_clusters:\n", " print(\"Working on cluster {}\".format(cur_cluster))\n", " cur_min_values = min_values.query(\"CLUSTER_ID == {}\".format(cur_cluster))['VALUE'].tolist()\n", " cur_max_values = max_values.query(\"CLUSTER_ID == {}\".format(cur_cluster))['VALUE'].tolist()\n", " cur_mean_values = mean_values.query(\"CLUSTER_ID == {}\".format(cur_cluster))['VALUE'].tolist()\n", " \n", " time_series_length = len(cur_max_values)\n", " patch_x = list(range(0, time_series_length, 1)) + list(range(time_series_length-1, -1, -1))\n", " patch_y = cur_min_values + cur_max_values[::-1]\n", " # Repeat the first point on the patch\n", " patch_x.append(patch_x[0])\n", " patch_y.append(patch_y[0])\n", " \n", " mean_x = list(range(0, time_series_length, 1))\n", " mean_y = [round(num,2) for num in cur_mean_values] # round to 2 decimal points\n", " \n", " plot_data_list.append({'patch_x': patch_x,\n", " 'patch_y': patch_y,\n", " 'mean_x': mean_x,\n", " 'mean_y': mean_y})\n", "\n", "# Check the result\n", "for plot_data in plot_data_list:\n", " print(\"\\n\")\n", " for key in plot_data.keys():\n", " print(key, \":\", plot_data[key])" ] }, { "cell_type": "code", "execution_count": 110, "metadata": { "scrolled": false }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"b5978f79-b83f-421f-9104-30421d76e45b\":{\"roots\":{\"references\":[{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,11,10,9,8,7,6,5,4,3,2,1,0,0],\"y\":[1.0,0.0,0.0,0.0,2.0,1.0,1.0,1.0,1.0,1.0,1.0,0.0,18.0,12.0,15.0,24.0,15.0,16.0,14.0,14.0,17.0,14.0,16.0,13.0,1.0]},\"selected\":{\"id\":\"e51a99d9-4b08-4a1e-b86e-7ffd89eacdc1\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"e157494c-00f9-4981-8e49-91fd187b22fc\",\"type\":\"UnionRenderers\"}},\"id\":\"b6f11f98-ef10-4228-877b-728450de0676\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"7835066b-c7b3-4239-ab9a-6e67f56d7d0f\",\"type\":\"ColumnDataSource\"}},\"id\":\"e3284550-cd41-43a7-9cf1-eed10cbcd1e2\",\"type\":\"CDSView\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8e8d1cf0-102b-40f1-88a7-65514be0f017\",\"type\":\"Line\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11],\"y\":[20.5,20.5,26.0,21.5,26.5,19.5,33.0,33.5,25.5,28.0,32.0,22.0]},\"selected\":{\"id\":\"94448653-63b1-437a-929d-78addbd90f42\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"e05121d4-2691-4e42-baf6-dc6188f3c904\",\"type\":\"UnionRenderers\"}},\"id\":\"7835066b-c7b3-4239-ab9a-6e67f56d7d0f\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"source\":{\"id\":\"e99da614-0bc3-4421-ac47-f8613bfdcbc4\",\"type\":\"ColumnDataSource\"}},\"id\":\"15b22fa3-36d5-43f4-87fa-b76dbdb36c89\",\"type\":\"CDSView\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#1f77b4\",\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4394819a-d18e-4172-963d-dd4cfc58ef94\",\"type\":\"Patch\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"ba975b5a-caa7-4d1a-8a0a-f2c5b3e045b4\",\"type\":\"BasicTicker\"}},\"id\":\"7d69b86c-7a83-4288-87f4-d9c627034103\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"cdeed96d-4109-4b2d-a30f-121103d1f6a7\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"bc5884d0-9e4c-4372-b837-a07a6c9ba965\",\"type\":\"HelpTool\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11],\"y\":[1.41,1.23,1.27,1.49,1.53,1.63,1.47,1.58,1.59,1.56,1.39,1.46]},\"selected\":{\"id\":\"219effcf-6c33-4806-bbdc-76f4532910d8\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"cdeed96d-4109-4b2d-a30f-121103d1f6a7\",\"type\":\"UnionRenderers\"}},\"id\":\"29d6d6dd-1cb1-43f0-9bb3-fe391199229a\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#1f77b4\",\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"f8842722-a972-41ba-9190-bd95c32b4c89\",\"type\":\"Patch\"},{\"attributes\":{},\"id\":\"a70c019f-ebf7-40af-a5f1-2253579948e7\",\"type\":\"BasicTicker\"},{\"attributes\":{\"source\":{\"id\":\"dfee8199-92c9-484c-93b0-7edc036215ca\",\"type\":\"ColumnDataSource\"}},\"id\":\"a149726a-be78-4eb9-9255-b782aad32d5c\",\"type\":\"CDSView\"},{\"attributes\":{\"data_source\":{\"id\":\"dfee8199-92c9-484c-93b0-7edc036215ca\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"4179f538-78b5-4608-b864-9cc05cf4491e\",\"type\":\"Patch\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"4394819a-d18e-4172-963d-dd4cfc58ef94\",\"type\":\"Patch\"},\"selection_glyph\":null,\"view\":{\"id\":\"a149726a-be78-4eb9-9255-b782aad32d5c\",\"type\":\"CDSView\"}},\"id\":\"f3121718-d59d-4b84-902e-a55f3c7d63da\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"74071046-4690-4be7-a910-b1ea67165f41\",\"type\":\"ResetTool\"},{\"attributes\":{},\"id\":\"219effcf-6c33-4806-bbdc-76f4532910d8\",\"type\":\"Selection\"},{\"attributes\":{\"callback\":null},\"id\":\"bb881c4a-e672-40ea-b190-28bee23583bd\",\"type\":\"DataRange1d\"},{\"attributes\":{\"data_source\":{\"id\":\"7835066b-c7b3-4239-ab9a-6e67f56d7d0f\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"838c7eb8-256c-4151-a7b6-ff4bd1780aa5\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"8e8d1cf0-102b-40f1-88a7-65514be0f017\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"e3284550-cd41-43a7-9cf1-eed10cbcd1e2\",\"type\":\"CDSView\"}},\"id\":\"c125de19-4875-48d2-8941-30ed4daf3789\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#78AAFF\",\"line_alpha\":0.1,\"line_color\":\"#78AAFF\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3c3c703a-b50f-4d04-bd13-69b065e661cf\",\"type\":\"Patch\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"18b738ba-cfb4-493b-95da-896aae94a4a1\",\"type\":\"PanTool\"},{\"id\":\"03ea7aff-baa5-466b-8fc5-7664e7cf09f9\",\"type\":\"WheelZoomTool\"},{\"id\":\"86af6fd8-e17b-4bd1-ba9a-07ce2fd50f1c\",\"type\":\"BoxZoomTool\"},{\"id\":\"b4f7d469-2ebd-405b-811c-259240f327dc\",\"type\":\"SaveTool\"},{\"id\":\"74071046-4690-4be7-a910-b1ea67165f41\",\"type\":\"ResetTool\"},{\"id\":\"bc5884d0-9e4c-4372-b837-a07a6c9ba965\",\"type\":\"HelpTool\"}]},\"id\":\"76898172-3596-4f4e-b208-23e720df4cb2\",\"type\":\"Toolbar\"},{\"attributes\":{\"data_source\":{\"id\":\"e99da614-0bc3-4421-ac47-f8613bfdcbc4\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"3c3c703a-b50f-4d04-bd13-69b065e661cf\",\"type\":\"Patch\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"f8842722-a972-41ba-9190-bd95c32b4c89\",\"type\":\"Patch\"},\"selection_glyph\":null,\"view\":{\"id\":\"15b22fa3-36d5-43f4-87fa-b76dbdb36c89\",\"type\":\"CDSView\"}},\"id\":\"b379e60d-c533-42ec-84f8-41856f079668\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#FF6455\",\"line_alpha\":0.1,\"line_color\":\"#FF6455\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4179f538-78b5-4608-b864-9cc05cf4491e\",\"type\":\"Patch\"},{\"attributes\":{},\"id\":\"ba975b5a-caa7-4d1a-8a0a-f2c5b3e045b4\",\"type\":\"BasicTicker\"},{\"attributes\":{\"line_color\":\"#FF6455\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"838c7eb8-256c-4151-a7b6-ff4bd1780aa5\",\"type\":\"Line\"},{\"attributes\":{\"callback\":null},\"id\":\"594e4f1a-ebab-4aa8-b009-ba6bdc77e112\",\"type\":\"DataRange1d\"},{\"attributes\":{\"below\":[{\"id\":\"57e75fd6-b7db-4548-80e1-63c5a55d0d2e\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"f53ba0b8-77db-4b46-b48a-e407594ad4db\",\"type\":\"LinearAxis\"}],\"plot_height\":350,\"plot_width\":800,\"renderers\":[{\"id\":\"57e75fd6-b7db-4548-80e1-63c5a55d0d2e\",\"type\":\"LinearAxis\"},{\"id\":\"9b273e8f-488a-49a7-989b-1e283043e87a\",\"type\":\"Grid\"},{\"id\":\"f53ba0b8-77db-4b46-b48a-e407594ad4db\",\"type\":\"LinearAxis\"},{\"id\":\"7d69b86c-7a83-4288-87f4-d9c627034103\",\"type\":\"Grid\"},{\"id\":\"fcde7390-fb52-4cfe-945e-4132e99b1dc7\",\"type\":\"BoxAnnotation\"},{\"id\":\"b379e60d-c533-42ec-84f8-41856f079668\",\"type\":\"GlyphRenderer\"},{\"id\":\"eee0bc65-d9af-4e8c-92b6-048660a002fe\",\"type\":\"Legend\"},{\"id\":\"7a20b70d-f1e0-4507-8a82-112f187af681\",\"type\":\"GlyphRenderer\"},{\"id\":\"f3121718-d59d-4b84-902e-a55f3c7d63da\",\"type\":\"GlyphRenderer\"},{\"id\":\"c125de19-4875-48d2-8941-30ed4daf3789\",\"type\":\"GlyphRenderer\"},{\"id\":\"157e42eb-4daf-40b2-ba2d-b7f16eb9e06f\",\"type\":\"GlyphRenderer\"},{\"id\":\"77fa4625-259b-4e47-9522-ec105c42f523\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"4972b3ee-2fee-4344-9ea5-1425f1898e33\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"76898172-3596-4f4e-b208-23e720df4cb2\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"bb881c4a-e672-40ea-b190-28bee23583bd\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"31576bad-af35-49b4-acf9-2ea11a325885\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"594e4f1a-ebab-4aa8-b009-ba6bdc77e112\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"21e04e63-05e9-4472-9dcb-b3c50bc90b32\",\"type\":\"LinearScale\"}},\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,11,10,9,8,7,6,5,4,3,2,1,0,0],\"y\":[16.0,11.0,19.0,18.0,21.0,12.0,26.0,30.0,19.0,27.0,27.0,10.0,34.0,37.0,29.0,32.0,37.0,40.0,27.0,32.0,25.0,33.0,30.0,25.0,16.0]},\"selected\":{\"id\":\"c2755c66-6315-4a08-9927-ef4d8f92b9a9\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"4fbd2ba9-c474-46b2-8f60-b4e28323072d\",\"type\":\"UnionRenderers\"}},\"id\":\"dfee8199-92c9-484c-93b0-7edc036215ca\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"plot\":null,\"text\":\"\"},\"id\":\"4972b3ee-2fee-4344-9ea5-1425f1898e33\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"31576bad-af35-49b4-acf9-2ea11a325885\",\"type\":\"LinearScale\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11,11,10,9,8,7,6,5,4,3,2,1,0,0],\"y\":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,7.0,7.0,8.0,9.0,7.0,9.0,8.0,8.0,8.0,7.0,7.0,8.0,0.0]},\"selected\":{\"id\":\"b6758ad9-1c27-4450-9c4e-2e9c02c3d18f\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"0bc71512-ac4b-44cc-ac10-ad87a6239e25\",\"type\":\"UnionRenderers\"}},\"id\":\"e99da614-0bc3-4421-ac47-f8613bfdcbc4\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"c2755c66-6315-4a08-9927-ef4d8f92b9a9\",\"type\":\"Selection\"},{\"attributes\":{\"axis_label\":\"Time Step - Monthly\",\"formatter\":{\"id\":\"c80c330e-4ca7-4325-b2bb-1af519f49ab2\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"a70c019f-ebf7-40af-a5f1-2253579948e7\",\"type\":\"BasicTicker\"}},\"id\":\"57e75fd6-b7db-4548-80e1-63c5a55d0d2e\",\"type\":\"LinearAxis\"},{\"attributes\":{},\"id\":\"4fbd2ba9-c474-46b2-8f60-b4e28323072d\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"label\":{\"value\":\"Cluster 2\"},\"renderers\":[{\"id\":\"c125de19-4875-48d2-8941-30ed4daf3789\",\"type\":\"GlyphRenderer\"}]},\"id\":\"222b32e5-04f9-446f-b91e-c17871cb0d63\",\"type\":\"LegendItem\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"fcde7390-fb52-4cfe-945e-4132e99b1dc7\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"21e04e63-05e9-4472-9dcb-b3c50bc90b32\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"e05121d4-2691-4e42-baf6-dc6188f3c904\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"label\":{\"value\":\"Cluster 1\"},\"renderers\":[{\"id\":\"7a20b70d-f1e0-4507-8a82-112f187af681\",\"type\":\"GlyphRenderer\"}]},\"id\":\"8e8a3fb4-400e-44d7-9352-289f83535e92\",\"type\":\"LegendItem\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#7DDC55\",\"line_alpha\":0.1,\"line_color\":\"#7DDC55\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5a146975-2051-498d-892f-ab13697b627f\",\"type\":\"Patch\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"ea8f798b-34f6-4456-a574-b47c8da58742\",\"type\":\"Line\"},{\"attributes\":{\"data_source\":{\"id\":\"29d6d6dd-1cb1-43f0-9bb3-fe391199229a\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1b13c1cd-a38d-4225-a51e-d2e2559c0f02\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"ea8f798b-34f6-4456-a574-b47c8da58742\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"9faaaaad-8e8f-4e30-a6cc-38331d4ddb1a\",\"type\":\"CDSView\"}},\"id\":\"7a20b70d-f1e0-4507-8a82-112f187af681\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":0.1,\"fill_color\":\"#1f77b4\",\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_dash\":[6],\"line_width\":4,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"41e97a40-df31-449a-9eff-ccda75099ded\",\"type\":\"Patch\"},{\"attributes\":{\"data_source\":{\"id\":\"b6f11f98-ef10-4228-877b-728450de0676\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"5a146975-2051-498d-892f-ab13697b627f\",\"type\":\"Patch\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"41e97a40-df31-449a-9eff-ccda75099ded\",\"type\":\"Patch\"},\"selection_glyph\":null,\"view\":{\"id\":\"0c131829-e6c1-4c33-b98a-5101f75e946c\",\"type\":\"CDSView\"}},\"id\":\"157e42eb-4daf-40b2-ba2d-b7f16eb9e06f\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"line_color\":\"#78AAFF\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1b13c1cd-a38d-4225-a51e-d2e2559c0f02\",\"type\":\"Line\"},{\"attributes\":{\"source\":{\"id\":\"b6f11f98-ef10-4228-877b-728450de0676\",\"type\":\"ColumnDataSource\"}},\"id\":\"0c131829-e6c1-4c33-b98a-5101f75e946c\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"c80c330e-4ca7-4325-b2bb-1af519f49ab2\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":[0,1,2,3,4,5,6,7,8,9,10,11],\"y\":[5.24,4.65,5.21,5.44,6.71,6.24,6.38,5.97,6.06,6.12,5.62,5.85]},\"selected\":{\"id\":\"e3f16f41-dab9-43b1-91d8-7b89253f09e9\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"c2bea421-6ea2-47bb-99b8-8a5ddf23505d\",\"type\":\"UnionRenderers\"}},\"id\":\"b1797a78-02ca-4962-822f-8d05e9c2b75b\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"axis_label\":\"Incident Count\",\"formatter\":{\"id\":\"78253177-3437-4c65-9f83-500e902a63f3\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"ba975b5a-caa7-4d1a-8a0a-f2c5b3e045b4\",\"type\":\"BasicTicker\"}},\"id\":\"f53ba0b8-77db-4b46-b48a-e407594ad4db\",\"type\":\"LinearAxis\"},{\"attributes\":{\"items\":[{\"id\":\"8e8a3fb4-400e-44d7-9352-289f83535e92\",\"type\":\"LegendItem\"},{\"id\":\"222b32e5-04f9-446f-b91e-c17871cb0d63\",\"type\":\"LegendItem\"},{\"id\":\"9909b449-c48b-4074-9eb5-2f0ac86a94d6\",\"type\":\"LegendItem\"}],\"plot\":{\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"eee0bc65-d9af-4e8c-92b6-048660a002fe\",\"type\":\"Legend\"},{\"attributes\":{\"line_color\":\"#7DDC55\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1dd485a0-caae-451e-836b-62da8db49ef4\",\"type\":\"Line\"},{\"attributes\":{\"plot\":{\"id\":\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"a70c019f-ebf7-40af-a5f1-2253579948e7\",\"type\":\"BasicTicker\"}},\"id\":\"9b273e8f-488a-49a7-989b-1e283043e87a\",\"type\":\"Grid\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"e53932c3-f84c-4d4a-9e6b-4a3374f99bc8\",\"type\":\"Line\"},{\"attributes\":{},\"id\":\"0bc71512-ac4b-44cc-ac10-ad87a6239e25\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data_source\":{\"id\":\"b1797a78-02ca-4962-822f-8d05e9c2b75b\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1dd485a0-caae-451e-836b-62da8db49ef4\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"e53932c3-f84c-4d4a-9e6b-4a3374f99bc8\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"b903d5cb-c74b-4446-bc20-7fb6645c9220\",\"type\":\"CDSView\"}},\"id\":\"77fa4625-259b-4e47-9522-ec105c42f523\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"b4f7d469-2ebd-405b-811c-259240f327dc\",\"type\":\"SaveTool\"},{\"attributes\":{\"source\":{\"id\":\"b1797a78-02ca-4962-822f-8d05e9c2b75b\",\"type\":\"ColumnDataSource\"}},\"id\":\"b903d5cb-c74b-4446-bc20-7fb6645c9220\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"03ea7aff-baa5-466b-8fc5-7664e7cf09f9\",\"type\":\"WheelZoomTool\"},{\"attributes\":{},\"id\":\"94448653-63b1-437a-929d-78addbd90f42\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"e3f16f41-dab9-43b1-91d8-7b89253f09e9\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"e51a99d9-4b08-4a1e-b86e-7ffd89eacdc1\",\"type\":\"Selection\"},{\"attributes\":{\"overlay\":{\"id\":\"fcde7390-fb52-4cfe-945e-4132e99b1dc7\",\"type\":\"BoxAnnotation\"}},\"id\":\"86af6fd8-e17b-4bd1-ba9a-07ce2fd50f1c\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"e157494c-00f9-4981-8e49-91fd187b22fc\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"label\":{\"value\":\"Cluster 3\"},\"renderers\":[{\"id\":\"77fa4625-259b-4e47-9522-ec105c42f523\",\"type\":\"GlyphRenderer\"}]},\"id\":\"9909b449-c48b-4074-9eb5-2f0ac86a94d6\",\"type\":\"LegendItem\"},{\"attributes\":{},\"id\":\"78253177-3437-4c65-9f83-500e902a63f3\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"c2bea421-6ea2-47bb-99b8-8a5ddf23505d\",\"type\":\"UnionRenderers\"},{\"attributes\":{},\"id\":\"b6758ad9-1c27-4450-9c4e-2e9c02c3d18f\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"18b738ba-cfb4-493b-95da-896aae94a4a1\",\"type\":\"PanTool\"},{\"attributes\":{\"source\":{\"id\":\"29d6d6dd-1cb1-43f0-9bb3-fe391199229a\",\"type\":\"ColumnDataSource\"}},\"id\":\"9faaaaad-8e8f-4e30-a6cc-38331d4ddb1a\",\"type\":\"CDSView\"}],\"root_ids\":[\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"b5978f79-b83f-421f-9104-30421d76e45b\",\"roots\":{\"3d79f332-2f3a-420e-a7d6-d35ae88d0623\":\"b3d8abea-0652-493a-89a6-5f82340f5181\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "3d79f332-2f3a-420e-a7d6-d35ae88d0623" } }, "output_type": "display_data" } ], "source": [ "# Establish plotting colors\n", "plotting_colors = ['#78AAFF', '#FF6455', '#7DDC55', '#FFB400', '#C864E1', '#BEA064', '#FABEC8', \n", " '#AFAFAF', '#005AE6', '#E60000', '#37A000', '#960096', '#B4FF00', '#822800', \n", " '#3C6E82', '#FF00C3', '#00E6AA', '#FFE600', '#002378', '#D78787', '#282828', \n", " '#73E1E1', '#006400', '#E1C3FF', '#966432', '#FFC88C', '#D2FFBE', '#CDE1FF', \n", " '#FFFF87', '#F0F0F0', ]\n", "\n", "p = figure(plot_height=350, plot_width=800)\n", "\n", "for idx, cur_plot in enumerate(plot_data_list):\n", " p.patch(cur_plot['patch_x'], cur_plot['patch_y'], alpha=0.1, color=plotting_colors[idx], line_dash='dashed', line_width=4)\n", " p.line(cur_plot['mean_x'], cur_plot['mean_y'], color=plotting_colors[idx], line_width=2, legend=\"Cluster \"+str(idx+1))\n", "\n", "p.xaxis.axis_label = \"Time Step - Monthly\"\n", "p.yaxis.axis_label = \"Incident Count\"\n", "\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Examining this plot shows the yearly trends of crime incidents for the detected time series clusters. You may ask why there is no patch for Cluster 3." ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "CLUSTER_ID\n", "1 219\n", "2 2\n", "3 34\n", "Name: OBJECTID, dtype: int64" ] }, "execution_count": 111, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# You can see why by looking back at the number of precincts in each cluster\n", "time_series_clustering_sedf.groupby('CLUSTER_ID').count()['OBJECTID']" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
OBJECTIDLOCATIONPRECINCTWARDIDCLUSTER_IDCENTER_REPSHAPE
91091030{\"rings\": [[[-7911147.7952, 5212701.136299998]...
1011101130{\"rings\": [[[-7912403.7838, 5212884.236900002]...
1920192030{\"rings\": [[[-7911199.420600001, 5213942.52629...
2021202130{\"rings\": [[[-7911461.109200001, 5212850.60589...
3233323330{\"rings\": [[[-7912035.4394000005, 5211065.0521...
3334333430{\"rings\": [[[-7912547.265699999, 5209658.84839...
3435343530{\"rings\": [[[-7913664.168400001, 5209543.5814]...
3738373830{\"rings\": [[[-7914198.358100001, 5210753.58269...
3839383930{\"rings\": [[[-7912459.356799999, 5211303.44799...
3940394030{\"rings\": [[[-7912441.8813000005, 5210562.7053...
4748474830{\"rings\": [[[-7915333.1019, 5209158.668099999]...
5051505130{\"rings\": [[[-7914773.013800001, 5207598.33309...
5758575830{\"rings\": [[[-7912090.3564, 5209123.319499999]...
6566656630{\"rings\": [[[-7911827.9991, 5207304.5988000035...
6869686930{\"rings\": [[[-7912559.333799999, 5207091.92440...
7071707130{\"rings\": [[[-7911781.941500001, 5207429.40269...
7172717230{\"rings\": [[[-7911560.8835, 5207111.8825], [-7...
7273727330{\"rings\": [[[-7910591.9153, 5207015.1994], [-7...
8283828330{\"rings\": [[[-7918073.421700001, 5213950.58259...
10110210110230{\"rings\": [[[-7919886.9054000005, 5212861.9746...
10710810710830{\"rings\": [[[-7912681.528100001, 5208056.77040...
11211311211330{\"rings\": [[[-7912684.676200001, 5204943.2086]...
11811911811930{\"rings\": [[[-7913213.440400001, 5203171.58649...
12012112012130{\"rings\": [[[-7910519.7347, 5205432.348899998]...
12712812712830{\"rings\": [[[-7911441.6557, 5204517.7623], [-7...
13713813713830{\"rings\": [[[-7912168.573799999, 5203452.15510...
14014114014130{\"rings\": [[[-7913997.262499999, 5202488.48200...
16716816716830{\"rings\": [[[-7911559.3039, 5205670.5825999975...
17617717617730{\"rings\": [[[-7913632.4454, 5201862.5898], [-7...
18218318218331{\"rings\": [[[-7914421.7546, 5205004.1008], [-7...
21821921821930{\"rings\": [[[-7910051.1163, 5212888.992899999]...
21922021922030{\"rings\": [[[-7910972.248199999, 5216386.0242]...
23423523423530{\"rings\": [[[-7911753.532199999, 5211990.42490...
25125225125230{\"rings\": [[[-7914645.84, 5212175.049699999], ...
\n", "
" ], "text/plain": [ " OBJECTID LOCATION PRECINCTWARDID CLUSTER_ID CENTER_REP \\\n", "9 10 9 10 3 0 \n", "10 11 10 11 3 0 \n", "19 20 19 20 3 0 \n", "20 21 20 21 3 0 \n", "32 33 32 33 3 0 \n", "33 34 33 34 3 0 \n", "34 35 34 35 3 0 \n", "37 38 37 38 3 0 \n", "38 39 38 39 3 0 \n", "39 40 39 40 3 0 \n", "47 48 47 48 3 0 \n", "50 51 50 51 3 0 \n", "57 58 57 58 3 0 \n", "65 66 65 66 3 0 \n", "68 69 68 69 3 0 \n", "70 71 70 71 3 0 \n", "71 72 71 72 3 0 \n", "72 73 72 73 3 0 \n", "82 83 82 83 3 0 \n", "101 102 101 102 3 0 \n", "107 108 107 108 3 0 \n", "112 113 112 113 3 0 \n", "118 119 118 119 3 0 \n", "120 121 120 121 3 0 \n", "127 128 127 128 3 0 \n", "137 138 137 138 3 0 \n", "140 141 140 141 3 0 \n", "167 168 167 168 3 0 \n", "176 177 176 177 3 0 \n", "182 183 182 183 3 1 \n", "218 219 218 219 3 0 \n", "219 220 219 220 3 0 \n", "234 235 234 235 3 0 \n", "251 252 251 252 3 0 \n", "\n", " SHAPE \n", "9 {\"rings\": [[[-7911147.7952, 5212701.136299998]... \n", "10 {\"rings\": [[[-7912403.7838, 5212884.236900002]... \n", "19 {\"rings\": [[[-7911199.420600001, 5213942.52629... \n", "20 {\"rings\": [[[-7911461.109200001, 5212850.60589... \n", "32 {\"rings\": [[[-7912035.4394000005, 5211065.0521... \n", "33 {\"rings\": [[[-7912547.265699999, 5209658.84839... \n", "34 {\"rings\": [[[-7913664.168400001, 5209543.5814]... \n", "37 {\"rings\": [[[-7914198.358100001, 5210753.58269... \n", "38 {\"rings\": [[[-7912459.356799999, 5211303.44799... \n", "39 {\"rings\": [[[-7912441.8813000005, 5210562.7053... \n", "47 {\"rings\": [[[-7915333.1019, 5209158.668099999]... \n", "50 {\"rings\": [[[-7914773.013800001, 5207598.33309... \n", "57 {\"rings\": [[[-7912090.3564, 5209123.319499999]... \n", "65 {\"rings\": [[[-7911827.9991, 5207304.5988000035... \n", "68 {\"rings\": [[[-7912559.333799999, 5207091.92440... \n", "70 {\"rings\": [[[-7911781.941500001, 5207429.40269... \n", "71 {\"rings\": [[[-7911560.8835, 5207111.8825], [-7... \n", "72 {\"rings\": [[[-7910591.9153, 5207015.1994], [-7... \n", "82 {\"rings\": [[[-7918073.421700001, 5213950.58259... \n", "101 {\"rings\": [[[-7919886.9054000005, 5212861.9746... \n", "107 {\"rings\": [[[-7912681.528100001, 5208056.77040... \n", "112 {\"rings\": [[[-7912684.676200001, 5204943.2086]... \n", "118 {\"rings\": [[[-7913213.440400001, 5203171.58649... \n", "120 {\"rings\": [[[-7910519.7347, 5205432.348899998]... \n", "127 {\"rings\": [[[-7911441.6557, 5204517.7623], [-7... \n", "137 {\"rings\": [[[-7912168.573799999, 5203452.15510... \n", "140 {\"rings\": [[[-7913997.262499999, 5202488.48200... \n", "167 {\"rings\": [[[-7911559.3039, 5205670.5825999975... \n", "176 {\"rings\": [[[-7913632.4454, 5201862.5898], [-7... \n", "182 {\"rings\": [[[-7914421.7546, 5205004.1008], [-7... \n", "218 {\"rings\": [[[-7910051.1163, 5212888.992899999]... \n", "219 {\"rings\": [[[-7910972.248199999, 5216386.0242]... \n", "234 {\"rings\": [[[-7911753.532199999, 5211990.42490... \n", "251 {\"rings\": [[[-7914645.84, 5212175.049699999], ... " ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Let's see which ward precinct experienced this extremely high occurance of crime incidents throughout year 2017\n", "time_series_clustering_sedf.loc[time_series_clustering_sedf['CLUSTER_ID'] == 3]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You can go back to the map above to see that's South Boston.\n", "\n", "And if you combine the count table and the chart, you can see that the majority of the ward precincts fell into Cluster 2 with very low crime incident rate, so Boston is a very safe city." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create a HTML report of the analysis\n", "This analysis follows similar patterns as above so if you are looking for detailed information about a step try looking through the code above." ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [], "source": [ "from scipy.stats import linregress # used to calculate slope" ] }, { "cell_type": "code", "execution_count": 114, "metadata": {}, "outputs": [], "source": [ "# Get the current date and time\n", "cur_date = dt.datetime.now()\n", "\n", "# Check the current week of the year\n", "cur_week = cur_date.isocalendar()[1]\n", "\n", "# Create a query string for the most recent crime incidents\n", "# If the current week is less than 13 we will need to query the last two years to get a complete data set\n", "if cur_week < 13:\n", " params = {\n", " 'sql': \"\"\"SELECT * from \"12cb3883-56f5-47de-afa5-3b1cf61b257b\" WHERE (\"OFFENSE_CODE_GROUP\" = 'Aggravated Assault' OR \"OFFENSE_CODE_GROUP\" = 'Simple Assault') AND (\"YEAR\"='{}'OR \"YEAR\" = '{}')\"\"\".format(cur_date.year, cur_date.year-1)\n", " }\n", "else:\n", " params = {\n", " 'sql': \"\"\"SELECT * from \"12cb3883-56f5-47de-afa5-3b1cf61b257b\" WHERE (\"OFFENSE_CODE_GROUP\" = 'Aggravated Assault' OR \"OFFENSE_CODE_GROUP\" = 'Simple Assault') AND \"YEAR\"='{}'\"\"\".format(cur_date.year)\n", " }" ] }, { "cell_type": "code", "execution_count": 115, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Status Code: 200\n" ] } ], "source": [ "# Build the query string and send response to server\n", "query_string = parse.urlencode(params)\n", "full_url = url + \"?\" + query_string\n", "\n", "try:\n", " resp = request.urlopen(full_url)\n", " print(\"Status Code: {}\".format(resp.code))\n", "except Exception as e:\n", " print(str(e))\n", " " ] }, { "cell_type": "code", "execution_count": 116, "metadata": {}, "outputs": [], "source": [ "# Clean the data from the API\n", "\n", "json_dict = json.loads(resp.read().decode())\n", "\n", "cur_crime_df = pd.DataFrame(json_dict['result']['records'])\n", "\n", "# Convert lat and long to numeric\n", "cur_crime_df['Long'] = pd.to_numeric(cur_crime_df['Long'])\n", "cur_crime_df['Lat'] = pd.to_numeric(cur_crime_df['Lat'])\n", "\n", "# convert hour of day to numeric\n", "cur_crime_df['HOUR'] = pd.to_numeric(cur_crime_df['HOUR'])\n", "\n", "cur_crime_df['DateTime'] = cur_crime_df.apply((lambda x: process_date(x['OCCURRED_ON_DATE'])), axis=1)\n", "\n", "# Remove locations without coordinates\n", "cur_crime_df_coords = cur_crime_df.query('(Long == Long) and (Long != -1)').copy()\n", "\n", "cur_crime_sedf = pd.DataFrame.spatial.from_xy(cur_crime_df_coords, 'Long', 'Lat')\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we have the most recent crime incident data available from the Crime Incident Reports API. The plan is to report the crimes for the past four weeks. We check the returned data for the most recent event and grab the previous four weeks from that date.\n", "\n", "Again we will use the Space Time Cube to aggregate the crime incidents into space and time bins. To create a space time cube a minimum of 10 time periods are required. We filter the data and keep only the 12 weeks before the most recent event." ] }, { "cell_type": "code", "execution_count": 117, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The most recent date in the data set is: 2019-03-14 19:44:00\n" ] } ], "source": [ "# Check the most recent date from the data\n", "most_recent_event = max(cur_crime_sedf['DateTime'])\n", "print(\"The most recent date in the data set is: {}\".format(most_recent_event))\n", "\n", "# Filter the data for the last 12 weeks\n", "analysis_period = \"84day\" # (12 weeks)\n", "most_recent_event - pd.to_timedelta(analysis_period)\n", "\n", "cur_crime_sedf = cur_crime_sedf[cur_crime_sedf['DateTime'] > most_recent_event - pd.to_timedelta(analysis_period)].copy()\n" ] }, { "cell_type": "code", "execution_count": 118, "metadata": {}, "outputs": [], "source": [ "# We can repeat the space time cube operation to aggregate into districts" ] }, { "cell_type": "code", "execution_count": 119, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'/arcgis/home/BostonCrimeAnalysis.gdb/cur_assault_points'" ] }, "execution_count": 119, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set up output paths for the analysis\n", "cur_crime_points = os.path.join(analysis_gdb, 'cur_assault_points')\n", "precincts_feature = os.path.join(analysis_gdb, 'precincts_feature')\n", "cur_hotspot_result = os.path.join(analysis_gdb, 'cur_hotspot_result')\n", "cur_crime_viz3d = os.path.join(analysis_gdb, 'cur_assault_viz3d')\n", "cur_crime_nc = os.path.join(outputSpace, \"cur_crime_cube.nc\")\n", "\n", "# Write the current crime points to a feature class for analysis\n", "cur_crime_sedf.spatial.to_featureclass(cur_crime_points)" ] }, { "cell_type": "code", "execution_count": 120, "metadata": {}, "outputs": [], "source": [ "# Aggregate into space time cube\n", "arcpy.stpm.CreateSpaceTimeCube(cur_crime_points, cur_crime_nc,\n", " \"DateTime\", None, \"1 Weeks\", \"END_TIME\", None, None, None,\n", " \"DEFINED_LOCATIONS\", precincts_feature, \"PrecinctWardID\")\n", " \n", "viz3d_results = arcpy.stpm.VisualizeSpaceTimeCube3D(cur_crime_nc, \"COUNT\", \"VALUE\", cur_crime_viz3d)\n", "\n", "# Load results to a data frame\n", "cur_crime_viz3d_sedf = pd.DataFrame.spatial.from_featureclass(cur_crime_viz3d) " ] }, { "cell_type": "code", "execution_count": 121, "metadata": {}, "outputs": [], "source": [ "# Get the last four weeks of data for this analysis\n", "last_four_weeks = list(cur_crime_viz3d_sedf['START_DATE'].unique())[-4:]" ] }, { "cell_type": "code", "execution_count": 122, "metadata": {}, "outputs": [], "source": [ "# Collect the top 5 precincts for crime incidents for the last four weeks\n", "\n", "count_list = []\n", "precinct_list = []\n", "date_list = []\n", "\n", "for cur_week in last_four_weeks:\n", " # Loop through the last four weeks and determine if any of the current week top precints\n", " # were in the previous week\n", " query_df = cur_crime_viz3d_sedf[cur_crime_viz3d_sedf['START_DATE'] == cur_week]\n", " cur_crime_by_precinct = query_df.groupby('PRECINCTWARDID')['VALUE'].sum().sort_values(ascending=False)\n", " count_list = count_list + list(cur_crime_by_precinct[:5].values)\n", " precinct_list = precinct_list + list(cur_crime_by_precinct.index[:5])\n", " date_list = date_list + [cur_week] * 5\n", " \n", "cur_crime_dict = {\"PRECINCT\": precinct_list,\n", " \"COUNT\": count_list,\n", " \"DATE\":date_list}" ] }, { "cell_type": "code", "execution_count": 123, "metadata": {}, "outputs": [], "source": [ "# Create a data frame with the crime count by precinct for the last four weeks\n", "crime_weeks_df = pd.DataFrame(cur_crime_dict)\n", "# Pull the most recent date from the data\n", "most_recent_date = max(crime_weeks_df['DATE'].unique())\n", "# Format the date for display\n", "time_string = pd.to_datetime(str(most_recent_date)) .strftime('%Y.%m.%d')\n", "# Create a subset data frame with the most recent week\n", "recent_week_df = crime_weeks_df[crime_weeks_df['DATE']==most_recent_date]\n", "# Create a subset data frame with the previous 3 weeks\n", "previous_week_df = crime_weeks_df[crime_weeks_df['DATE']!=most_recent_date]" ] }, { "cell_type": "code", "execution_count": 124, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Precinct 113 is new in the top 5. Action required\n", "Precinct 217 has been in the top 5 for crimes for the past 3 week(s)\n", "Precinct 107 is new in the top 5. Action required\n", "Precinct 74 is new in the top 5. Action required\n", "Precinct 33 has been in the top 5 for crimes for the past 1 week(s)\n", "Crime incidents are on the decrease. Send our regards to the Police Chief\n", "\n" ] } ], "source": [ "report_string = \"\"\n", "\n", "for cur_precinct in recent_week_df['PRECINCT']:\n", " \"\"\"Loop thorugh each precinct in the top 5 for the most recent week and compare to\n", " the previous 3 weeks to determine if it is new in the top 5 or a repeat\"\"\"\n", " \n", " # Group the previous 3 weeks by precint\n", " previous_week_precincts = previous_week_df.groupby('PRECINCT').count()\n", " \n", " # Check if the current precinct was in the top 5 previously\n", " if cur_precinct in previous_week_precincts.index:\n", " \n", " cur_index = list(previous_week_precincts.index).index(cur_precinct)\n", " # Count how many times this precinct has been in the top 5 in the last 4 weeks\n", " cur_number_of_times = int(previous_week_precincts.iloc[cur_index]['COUNT'])\n", " cur_string = \"Precinct {} has been in the top 5 for crimes for the past {} week(s)\\n\".format(cur_precinct, cur_number_of_times)\n", " \n", " else:\n", " cur_string = \"Precinct {} is new in the top 5. Action required\\n\".format(cur_precinct)\n", " \n", " report_string = report_string + cur_string\n", "\n", "# Determine the crime incident trend for the past 4 weeks\n", "total_crimes_by_week = crime_weeks_df.groupby('DATE')['COUNT'].sum().values \n", "crime_slope = linregress([0, 1, 2, 3], total_crimes_by_week).slope\n", "\n", "if crime_slope < 0:\n", " cur_string = \"Crime incidents are on the decrease. Send our regards to the Police Chief\\n\"\n", "else:\n", " cur_string = \"Crimes incidents are on the rise. Set a meeting with the Police Chief\\n\"\n", " \n", "report_string = report_string + cur_string\n", "print(report_string)" ] }, { "cell_type": "code", "execution_count": 125, "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"441ed8d8-6f5c-4879-af0b-5d42290fbeab\":{\"roots\":{\"references\":[{\"attributes\":{\"callback\":null,\"data\":{\"x\":{\"__ndarray__\":\"AIC+mNiOdkIAgP5gGZF2QgCAPilak3ZCAIB+8ZqVdkI=\",\"dtype\":\"float64\",\"shape\":[4]},\"y\":{\"__ndarray__\":\"AAAAAAAANkAAAAAAAAA7QAAAAAAAADhAAAAAAAAAM0A=\",\"dtype\":\"float64\",\"shape\":[4]}},\"selected\":{\"id\":\"d1d33976-eaab-4032-8d6d-2f247093625e\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"9f417167-fc71-4fe4-8d5f-b04959b484cd\",\"type\":\"UnionRenderers\"}},\"id\":\"16622291-4f77-47ea-9e11-cb82c9c22139\",\"type\":\"ColumnDataSource\"},{\"attributes\":{},\"id\":\"9f417167-fc71-4fe4-8d5f-b04959b484cd\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"fca2bf66-e051-4966-a731-5132a292f343\",\"type\":\"Line\"},{\"attributes\":{\"months\":[0,6]},\"id\":\"864470ab-254b-4a1b-91cb-e4a524d85483\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"plot\":{\"id\":\"dc4212c0-989d-4416-91d6-c2182cf9381a\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1cfd68a9-0341-458e-b253-472da54c029d\",\"type\":\"DatetimeTicker\"}},\"id\":\"f05fb595-3dd2-46ca-8289-b71f98da1b55\",\"type\":\"Grid\"},{\"attributes\":{\"line_alpha\":0.1,\"line_color\":\"#1f77b4\",\"line_width\":2,\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"33ba1296-8968-4015-833c-1f0da24eb1e7\",\"type\":\"Line\"},{\"attributes\":{\"days\":[1,8,15,22]},\"id\":\"ebd9423d-1a40-4fba-a16f-fb68ca40acee\",\"type\":\"DaysTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.5},\"line_color\":{\"value\":\"#1f77b4\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"e09aed7c-347d-4448-b074-d45df4733092\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"19fb4361-7f92-4769-917e-96848ae8d52c\",\"type\":\"PanTool\"},{\"attributes\":{\"callback\":null,\"end\":32},\"id\":\"0f619b1e-2661-4fa4-983b-f022f9b23ae9\",\"type\":\"Range1d\"},{\"attributes\":{\"base\":24,\"mantissas\":[1,2,4,6,8,12],\"max_interval\":43200000.0,\"min_interval\":3600000.0,\"num_minor_ticks\":0},\"id\":\"0e0cdaa4-9e29-4f8b-878c-b5afe7bd8476\",\"type\":\"AdaptiveTicker\"},{\"attributes\":{},\"id\":\"329242bc-08f6-48af-9bfa-5e5790aca258\",\"type\":\"DatetimeTickFormatter\"},{\"attributes\":{},\"id\":\"65d89ba5-7097-4350-a2e4-2d9865ad4ca2\",\"type\":\"YearsTicker\"},{\"attributes\":{\"below\":[{\"id\":\"554a07d7-d6bc-4f44-ba53-c19115391a3a\",\"type\":\"DatetimeAxis\"}],\"left\":[{\"id\":\"351154fb-1abe-43f5-9512-7003e3d682c4\",\"type\":\"LinearAxis\"}],\"plot_height\":250,\"renderers\":[{\"id\":\"554a07d7-d6bc-4f44-ba53-c19115391a3a\",\"type\":\"DatetimeAxis\"},{\"id\":\"f05fb595-3dd2-46ca-8289-b71f98da1b55\",\"type\":\"Grid\"},{\"id\":\"351154fb-1abe-43f5-9512-7003e3d682c4\",\"type\":\"LinearAxis\"},{\"id\":\"b9f8d212-4d30-4073-bc52-6f8a59a810a0\",\"type\":\"Grid\"},{\"id\":\"bc9d5c8b-fc2b-4c51-83f5-be948a1a9651\",\"type\":\"BoxAnnotation\"},{\"id\":\"fb41968a-6237-4735-85c6-0a963c6898fe\",\"type\":\"GlyphRenderer\"},{\"id\":\"2bbc261e-cbf6-49a9-9ef4-fcd1f67d68f2\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"cbbc5a6b-c3e4-435d-9dcd-117e2c6e9075\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"fada0c6e-bba0-4965-a5ab-73af7d252038\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"ddd202be-5f0f-4d13-87c7-90163119f1c8\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"b74052ab-4115-4a24-a577-7f20819295ff\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"0f619b1e-2661-4fa4-983b-f022f9b23ae9\",\"type\":\"Range1d\"},\"y_scale\":{\"id\":\"4a40f97d-d245-4ce1-902c-2dbf460091bc\",\"type\":\"LinearScale\"}},\"id\":\"dc4212c0-989d-4416-91d6-c2182cf9381a\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"overlay\":{\"id\":\"bc9d5c8b-fc2b-4c51-83f5-be948a1a9651\",\"type\":\"BoxAnnotation\"}},\"id\":\"9aa0c40b-29fd-48d1-8738-c5cc2e6cbbd0\",\"type\":\"BoxZoomTool\"},{\"attributes\":{},\"id\":\"3403eb30-38d0-4183-9ef5-eddce97a5b81\",\"type\":\"HelpTool\"},{\"attributes\":{\"formatter\":{\"id\":\"1a72b0b9-c2a8-4a74-be0f-081cbb091c42\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"dc4212c0-989d-4416-91d6-c2182cf9381a\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"adbc556b-27c3-4cda-b852-56b159d3358e\",\"type\":\"BasicTicker\"}},\"id\":\"351154fb-1abe-43f5-9512-7003e3d682c4\",\"type\":\"LinearAxis\"},{\"attributes\":{\"data_source\":{\"id\":\"16622291-4f77-47ea-9e11-cb82c9c22139\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"fca2bf66-e051-4966-a731-5132a292f343\",\"type\":\"Line\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"33ba1296-8968-4015-833c-1f0da24eb1e7\",\"type\":\"Line\"},\"selection_glyph\":null,\"view\":{\"id\":\"d59b6fc8-bab7-4285-b306-0b4312c8fe4f\",\"type\":\"CDSView\"}},\"id\":\"fb41968a-6237-4735-85c6-0a963c6898fe\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"plot\":null,\"text\":\"Four Week Assault Summary Report [2019.03.07]\"},\"id\":\"cbbc5a6b-c3e4-435d-9dcd-117e2c6e9075\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"4106baed-1380-42f6-b7f1-b6318da6b38f\",\"type\":\"ResetTool\"},{\"attributes\":{\"base\":60,\"mantissas\":[1,2,5,10,15,20,30],\"max_interval\":1800000.0,\"min_interval\":1000.0,\"num_minor_ticks\":0},\"id\":\"3989b059-7d37-47f7-baa9-ce59b79e029a\",\"type\":\"AdaptiveTicker\"},{\"attributes\":{},\"id\":\"d1d33976-eaab-4032-8d6d-2f247093625e\",\"type\":\"Selection\"},{\"attributes\":{\"days\":[1,15]},\"id\":\"b04e139d-1b7b-4b3d-83c4-7706fdbf8345\",\"type\":\"DaysTicker\"},{\"attributes\":{\"callback\":null,\"data\":{\"x\":{\"__ndarray__\":\"AIC+mNiOdkIAgP5gGZF2QgCAPilak3ZCAIB+8ZqVdkI=\",\"dtype\":\"float64\",\"shape\":[4]},\"y\":{\"__ndarray__\":\"AAAAAAAANkAAAAAAAAA7QAAAAAAAADhAAAAAAAAAM0A=\",\"dtype\":\"float64\",\"shape\":[4]}},\"selected\":{\"id\":\"e9e8dd98-883d-4b4f-a350-7a7532638af5\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"e496e147-f386-41d3-b585-be76f9ff3780\",\"type\":\"UnionRenderers\"}},\"id\":\"ac72c3cb-e8d0-47e0-9595-bd65ed305874\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5c0e97e0-bde2-483c-b8ae-91e926228c3a\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"b74052ab-4115-4a24-a577-7f20819295ff\",\"type\":\"LinearScale\"},{\"attributes\":{\"months\":[0,1,2,3,4,5,6,7,8,9,10,11]},\"id\":\"21096aba-1c5f-4a0b-8358-c5fca2508627\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"formatter\":{\"id\":\"329242bc-08f6-48af-9bfa-5e5790aca258\",\"type\":\"DatetimeTickFormatter\"},\"plot\":{\"id\":\"dc4212c0-989d-4416-91d6-c2182cf9381a\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1cfd68a9-0341-458e-b253-472da54c029d\",\"type\":\"DatetimeTicker\"}},\"id\":\"554a07d7-d6bc-4f44-ba53-c19115391a3a\",\"type\":\"DatetimeAxis\"},{\"attributes\":{\"data_source\":{\"id\":\"ac72c3cb-e8d0-47e0-9595-bd65ed305874\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"e09aed7c-347d-4448-b074-d45df4733092\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"5c0e97e0-bde2-483c-b8ae-91e926228c3a\",\"type\":\"Circle\"},\"selection_glyph\":null,\"view\":{\"id\":\"b3854a16-2ea0-49ea-aa3c-f67a5959a6c9\",\"type\":\"CDSView\"}},\"id\":\"2bbc261e-cbf6-49a9-9ef4-fcd1f67d68f2\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"99707987-236c-4dbc-9ec8-1ac7676bb651\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"months\":[0,2,4,6,8,10]},\"id\":\"1d5619cb-a995-4c98-87e1-21d94339bb86\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"desired_num_ticks\":5},\"id\":\"adbc556b-27c3-4cda-b852-56b159d3358e\",\"type\":\"BasicTicker\"},{\"attributes\":{\"desired_num_ticks\":4,\"num_minor_ticks\":5,\"tickers\":[{\"id\":\"082a27f6-6624-4b4d-8599-a01d773c5134\",\"type\":\"AdaptiveTicker\"},{\"id\":\"3989b059-7d37-47f7-baa9-ce59b79e029a\",\"type\":\"AdaptiveTicker\"},{\"id\":\"0e0cdaa4-9e29-4f8b-878c-b5afe7bd8476\",\"type\":\"AdaptiveTicker\"},{\"id\":\"440a2613-3de9-41d8-bf5e-0229cba8b46c\",\"type\":\"DaysTicker\"},{\"id\":\"fd240dd2-ab30-48dd-9581-95a83de7f333\",\"type\":\"DaysTicker\"},{\"id\":\"ebd9423d-1a40-4fba-a16f-fb68ca40acee\",\"type\":\"DaysTicker\"},{\"id\":\"b04e139d-1b7b-4b3d-83c4-7706fdbf8345\",\"type\":\"DaysTicker\"},{\"id\":\"21096aba-1c5f-4a0b-8358-c5fca2508627\",\"type\":\"MonthsTicker\"},{\"id\":\"1d5619cb-a995-4c98-87e1-21d94339bb86\",\"type\":\"MonthsTicker\"},{\"id\":\"424f80b1-ed2f-4475-970c-763c36dd180f\",\"type\":\"MonthsTicker\"},{\"id\":\"864470ab-254b-4a1b-91cb-e4a524d85483\",\"type\":\"MonthsTicker\"},{\"id\":\"65d89ba5-7097-4350-a2e4-2d9865ad4ca2\",\"type\":\"YearsTicker\"}]},\"id\":\"1cfd68a9-0341-458e-b253-472da54c029d\",\"type\":\"DatetimeTicker\"},{\"attributes\":{\"days\":[1,4,7,10,13,16,19,22,25,28]},\"id\":\"fd240dd2-ab30-48dd-9581-95a83de7f333\",\"type\":\"DaysTicker\"},{\"attributes\":{\"callback\":null},\"id\":\"ddd202be-5f0f-4d13-87c7-90163119f1c8\",\"type\":\"DataRange1d\"},{\"attributes\":{},\"id\":\"e9e8dd98-883d-4b4f-a350-7a7532638af5\",\"type\":\"Selection\"},{\"attributes\":{},\"id\":\"19397365-68b5-43d9-b1a6-eb1e03b83201\",\"type\":\"SaveTool\"},{\"attributes\":{\"months\":[0,4,8]},\"id\":\"424f80b1-ed2f-4475-970c-763c36dd180f\",\"type\":\"MonthsTicker\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"dc4212c0-989d-4416-91d6-c2182cf9381a\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"adbc556b-27c3-4cda-b852-56b159d3358e\",\"type\":\"BasicTicker\"}},\"id\":\"b9f8d212-4d30-4073-bc52-6f8a59a810a0\",\"type\":\"Grid\"},{\"attributes\":{\"mantissas\":[1,2,5],\"max_interval\":500.0,\"num_minor_ticks\":0},\"id\":\"082a27f6-6624-4b4d-8599-a01d773c5134\",\"type\":\"AdaptiveTicker\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"19fb4361-7f92-4769-917e-96848ae8d52c\",\"type\":\"PanTool\"},{\"id\":\"99707987-236c-4dbc-9ec8-1ac7676bb651\",\"type\":\"WheelZoomTool\"},{\"id\":\"9aa0c40b-29fd-48d1-8738-c5cc2e6cbbd0\",\"type\":\"BoxZoomTool\"},{\"id\":\"19397365-68b5-43d9-b1a6-eb1e03b83201\",\"type\":\"SaveTool\"},{\"id\":\"4106baed-1380-42f6-b7f1-b6318da6b38f\",\"type\":\"ResetTool\"},{\"id\":\"3403eb30-38d0-4183-9ef5-eddce97a5b81\",\"type\":\"HelpTool\"}]},\"id\":\"fada0c6e-bba0-4965-a5ab-73af7d252038\",\"type\":\"Toolbar\"},{\"attributes\":{\"source\":{\"id\":\"16622291-4f77-47ea-9e11-cb82c9c22139\",\"type\":\"ColumnDataSource\"}},\"id\":\"d59b6fc8-bab7-4285-b306-0b4312c8fe4f\",\"type\":\"CDSView\"},{\"attributes\":{},\"id\":\"4a40f97d-d245-4ce1-902c-2dbf460091bc\",\"type\":\"LinearScale\"},{\"attributes\":{\"days\":[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]},\"id\":\"440a2613-3de9-41d8-bf5e-0229cba8b46c\",\"type\":\"DaysTicker\"},{\"attributes\":{},\"id\":\"e496e147-f386-41d3-b585-be76f9ff3780\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"source\":{\"id\":\"ac72c3cb-e8d0-47e0-9595-bd65ed305874\",\"type\":\"ColumnDataSource\"}},\"id\":\"b3854a16-2ea0-49ea-aa3c-f67a5959a6c9\",\"type\":\"CDSView\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"bc9d5c8b-fc2b-4c51-83f5-be948a1a9651\",\"type\":\"BoxAnnotation\"},{\"attributes\":{},\"id\":\"1a72b0b9-c2a8-4a74-be0f-081cbb091c42\",\"type\":\"BasicTickFormatter\"}],\"root_ids\":[\"dc4212c0-989d-4416-91d6-c2182cf9381a\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"441ed8d8-6f5c-4879-af0b-5d42290fbeab\",\"roots\":{\"dc4212c0-989d-4416-91d6-c2182cf9381a\":\"4ab78e0c-0084-44fb-8674-31633309db57\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "dc4212c0-989d-4416-91d6-c2182cf9381a" } }, "output_type": "display_data" } ], "source": [ "# Plot crimes for the last 4 weeks\n", "# With this method I am having trouble getting the dates to line up, so I created the bar chart below as an alternative\n", "\n", "group = crime_weeks_df.groupby('DATE')['COUNT'].sum()\n", "\n", "calc_y_max = int(max(group.values) + max(group.values)*0.2)\n", "\n", "p = figure(plot_width=600, plot_height=250, title=\"Four Week Assault Summary Report [{}]\".format(time_string),\n", " x_axis_type='datetime', y_range=(0, calc_y_max))\n", "p.line(group.index, group, line_width=2)\n", "p.circle(group.index, group, alpha=0.5)\n", "\n", "p.yaxis[0].ticker.desired_num_ticks = 5\n", "p.xaxis[0].ticker.desired_num_ticks = 4\n", "\n", "show(p)" ] }, { "cell_type": "code", "execution_count": 126, "metadata": { "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "
\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "(function(root) {\n", " function embed_document(root) {\n", " \n", " var docs_json = {\"dc61eda5-2a97-4a81-831b-487d3fe23ce6\":{\"roots\":{\"references\":[{\"attributes\":{},\"id\":\"5ca5b454-ffca-4f92-93ae-6d087779ef51\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"0f9c09dd-8a1f-4902-b3f9-35d39bf17d62\",\"type\":\"Selection\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.8},\"x\":{\"field\":\"x\"}},\"id\":\"5d46be82-81ab-47c0-930d-777b01a24eed\",\"type\":\"VBar\"},{\"attributes\":{\"plot\":null,\"text\":\"Four Week Assault Summary Report [2019.03.07]\"},\"id\":\"560ea054-b720-4474-82af-643a11bec40f\",\"type\":\"Title\"},{\"attributes\":{},\"id\":\"f7e61a78-2cc4-4b4c-949d-768fe89336f1\",\"type\":\"BasicTicker\"},{\"attributes\":{\"callback\":null,\"data\":{\"top\":{\"__ndarray__\":\"AAAAAAAANkAAAAAAAAA7QAAAAAAAADhAAAAAAAAAM0A=\",\"dtype\":\"float64\",\"shape\":[4]},\"x\":[0,1,2,3]},\"selected\":{\"id\":\"0f9c09dd-8a1f-4902-b3f9-35d39bf17d62\",\"type\":\"Selection\"},\"selection_policy\":{\"id\":\"25bf8c28-f370-419f-9a08-a26d9f6e1d8f\",\"type\":\"UnionRenderers\"}},\"id\":\"cdb145ae-4c98-45ab-b39d-bd2409341757\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"desired_num_ticks\":5},\"id\":\"1ed7aafd-6661-402a-aa96-b81121109dcd\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_color\":{\"value\":\"#1f77b4\"},\"line_color\":{\"value\":\"#1f77b4\"},\"top\":{\"field\":\"top\"},\"width\":{\"value\":0.8},\"x\":{\"field\":\"x\"}},\"id\":\"c1e5a42e-fe15-4c43-bb0c-d622ff61cede\",\"type\":\"VBar\"},{\"attributes\":{\"below\":[{\"id\":\"592d4523-acbb-46d7-9e2c-75a7e2f2cb88\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"33507240-02ef-48c2-bd5b-75716497815b\",\"type\":\"LinearAxis\"}],\"plot_height\":250,\"renderers\":[{\"id\":\"592d4523-acbb-46d7-9e2c-75a7e2f2cb88\",\"type\":\"LinearAxis\"},{\"id\":\"814844a8-855a-4b01-b13d-091b9a18bc1e\",\"type\":\"Grid\"},{\"id\":\"33507240-02ef-48c2-bd5b-75716497815b\",\"type\":\"LinearAxis\"},{\"id\":\"0d68640d-657b-4385-8c78-7101ff728047\",\"type\":\"Grid\"},{\"id\":\"848e7cc9-ba01-46b4-be4c-c768aa70ad31\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"560ea054-b720-4474-82af-643a11bec40f\",\"type\":\"Title\"},\"toolbar\":{\"id\":\"3aeb00d6-322b-4fc4-902e-d2c801c8d322\",\"type\":\"Toolbar\"},\"toolbar_location\":null,\"x_range\":{\"id\":\"16f940e1-cdf5-40fe-aac5-1bd6ceffb276\",\"type\":\"DataRange1d\"},\"x_scale\":{\"id\":\"5ca5b454-ffca-4f92-93ae-6d087779ef51\",\"type\":\"LinearScale\"},\"y_range\":{\"id\":\"edc5af48-5ea1-4aeb-8ef3-792d080c5973\",\"type\":\"DataRange1d\"},\"y_scale\":{\"id\":\"766c4d5d-f4ef-4607-ace0-0999dc2f56e6\",\"type\":\"LinearScale\"}},\"id\":\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"formatter\":{\"id\":\"c208ced7-c155-44cc-9f94-0a96b898ba00\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1ed7aafd-6661-402a-aa96-b81121109dcd\",\"type\":\"BasicTicker\"}},\"id\":\"33507240-02ef-48c2-bd5b-75716497815b\",\"type\":\"LinearAxis\"},{\"attributes\":{\"plot\":{\"id\":\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"f7e61a78-2cc4-4b4c-949d-768fe89336f1\",\"type\":\"BasicTicker\"}},\"id\":\"814844a8-855a-4b01-b13d-091b9a18bc1e\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"766c4d5d-f4ef-4607-ace0-0999dc2f56e6\",\"type\":\"LinearScale\"},{\"attributes\":{},\"id\":\"59bfe6c6-f692-4287-9827-34ec45733455\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{},\"id\":\"c208ced7-c155-44cc-9f94-0a96b898ba00\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"formatter\":{\"id\":\"59bfe6c6-f692-4287-9827-34ec45733455\",\"type\":\"BasicTickFormatter\"},\"major_label_overrides\":{\"0\":\"2019.02.14\",\"1\":\"2019.02.21\",\"2\":\"2019.02.28\",\"3\":\"2019.03.07\"},\"plot\":{\"id\":\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"f7e61a78-2cc4-4b4c-949d-768fe89336f1\",\"type\":\"BasicTicker\"}},\"id\":\"592d4523-acbb-46d7-9e2c-75a7e2f2cb88\",\"type\":\"LinearAxis\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"1ed7aafd-6661-402a-aa96-b81121109dcd\",\"type\":\"BasicTicker\"}},\"id\":\"0d68640d-657b-4385-8c78-7101ff728047\",\"type\":\"Grid\"},{\"attributes\":{},\"id\":\"25bf8c28-f370-419f-9a08-a26d9f6e1d8f\",\"type\":\"UnionRenderers\"},{\"attributes\":{\"data_source\":{\"id\":\"cdb145ae-4c98-45ab-b39d-bd2409341757\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"c1e5a42e-fe15-4c43-bb0c-d622ff61cede\",\"type\":\"VBar\"},\"hover_glyph\":null,\"muted_glyph\":null,\"nonselection_glyph\":{\"id\":\"5d46be82-81ab-47c0-930d-777b01a24eed\",\"type\":\"VBar\"},\"selection_glyph\":null,\"view\":{\"id\":\"87ade4c7-8c35-4ab7-b250-d7542693cbce\",\"type\":\"CDSView\"}},\"id\":\"848e7cc9-ba01-46b4-be4c-c768aa70ad31\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"source\":{\"id\":\"cdb145ae-4c98-45ab-b39d-bd2409341757\",\"type\":\"ColumnDataSource\"}},\"id\":\"87ade4c7-8c35-4ab7-b250-d7542693cbce\",\"type\":\"CDSView\"},{\"attributes\":{\"callback\":null},\"id\":\"edc5af48-5ea1-4aeb-8ef3-792d080c5973\",\"type\":\"DataRange1d\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_inspect\":\"auto\",\"active_multi\":null,\"active_scroll\":\"auto\",\"active_tap\":\"auto\"},\"id\":\"3aeb00d6-322b-4fc4-902e-d2c801c8d322\",\"type\":\"Toolbar\"},{\"attributes\":{\"callback\":null},\"id\":\"16f940e1-cdf5-40fe-aac5-1bd6ceffb276\",\"type\":\"DataRange1d\"}],\"root_ids\":[\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\"]},\"title\":\"Bokeh Application\",\"version\":\"0.13.0\"}};\n", " var render_items = [{\"docid\":\"dc61eda5-2a97-4a81-831b-487d3fe23ce6\",\"roots\":{\"5eaa331a-b32d-4ce6-b99f-02750f8aa00b\":\"83ee749c-8ca0-4b2b-b027-5a5e71faf189\"}}];\n", " root.Bokeh.embed.embed_items_notebook(docs_json, render_items);\n", "\n", " }\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " } else {\n", " var attempts = 0;\n", " var timer = setInterval(function(root) {\n", " if (root.Bokeh !== undefined) {\n", " embed_document(root);\n", " clearInterval(timer);\n", " }\n", " attempts++;\n", " if (attempts > 100) {\n", " console.log(\"Bokeh: ERROR: Unable to run BokehJS code because BokehJS library is missing\")\n", " clearInterval(timer);\n", " }\n", " }, 10, root)\n", " }\n", "})(window);" ], "application/vnd.bokehjs_exec.v0+json": "" }, "metadata": { "application/vnd.bokehjs_exec.v0+json": { "id": "5eaa331a-b32d-4ce6-b99f-02750f8aa00b" } }, "output_type": "display_data" } ], "source": [ "p = figure(plot_height=250, title=\"Four Week Assault Summary Report [{}]\".format(time_string),\n", " toolbar_location=None, tools=\"\")\n", "p.vbar(x=[0, 1, 2, 3], top=group, width=0.8)\n", "\n", "# Way to custom label the chart with dates\n", "label_dict = {}\n", "for idx, cur_label in enumerate(group.index):\n", " label_dict[idx] = cur_label.strftime('%Y.%m.%d') \n", "p.xaxis.major_label_overrides = label_dict\n", "\n", "p.yaxis[0].ticker.desired_num_ticks = 5\n", "\n", "# Write the chart for use in the report\n", "html = file_html(p, CDN, \"my plot\")\n", "output_html = os.path.join(outputSpace, \"ReportChart.html\")\n", "with open(output_html, 'w') as f:\n", " f.write(html)\n", "\n", "show(p)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a map showing the top 5 precincts this week\n", "\n", "Create a map showing precincts in Boston and the top five precincts from this week." ] }, { "cell_type": "code", "execution_count": 142, "metadata": { "scrolled": false }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "823adb48d5ad4a38bbf22807133c310d", "version_major": 2, "version_minor": 0 }, "text/plain": [ "MapView(jupyter_target='notebook', layout=Layout(height='400px', width='100%'), ready=True, zoom=10.0)" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "current_crime_map = gis.map(\"Boston\")\n", "current_crime_map.basemap ='dark-gray'\n", "current_crime_map" ] }, { "cell_type": "code", "execution_count": 128, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 128, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Subset the last weeks area to show on the map\n", "cur_week_precincts = list(recent_week_df['PRECINCT'])\n", "cur_week_precincts\n", "\n", "precincts_sedf.spatial.plot(current_crime_map,\n", " symbol_type = \"simple\",\n", " cmap = [90,180,172,225],\n", " outline_color = [225,225,225,200],\n", " line_width = 0.5)\n", "\n", "# Select the top 5 precincts\n", "top5 = precincts_sedf[precincts_sedf['PrecinctWardID'].isin(cur_week_precincts)].copy()\n", "top5.spatial.plot(current_crime_map, \n", " cmap = [0,0,0,0], \n", " line_width=1.2, \n", " outline_color = [237,117,81,255])" ] }, { "cell_type": "code", "execution_count": 129, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 129, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Write the map to html for use in report\n", "current_crime_map.export_to_html('ReportMap.html')" ] }, { "cell_type": "code", "execution_count": 130, "metadata": {}, "outputs": [], "source": [ "# Create a html version of the report string\n", "report_html_string = \"\"\n", "\n", "# Add html line breaks to the text being written\n", "for cur_line in report_string.splitlines():\n", " report_html_string = report_html_string + cur_line + \"
\\n\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The html report template is written in html as a string below. The text is added to the report as html strings. The map and chart which have been saved as html files are embedded in iframes within the report. It is all combined and styled into the single document which can easily be shared.\n", "\n", "**Note**: This notebook uses a local reference for the embedded map and chart. So you will need to have all 3 files in the same location for it to load." ] }, { "cell_type": "code", "execution_count": 131, "metadata": {}, "outputs": [], "source": [ "# Build a html report\n", "\n", "html_report = \"\"\"\n", "\n", "\n", "\n", " \n", "\n", " Crime Analysis Report\n", " \n", " \n", "\n", " \n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "

Crime Analysis Report for {}

\n", "\n", "{}\n", "\n", "

\n", "\n", "
\n", "\n", " \n", "
\n", "\n", "
\n", "\n", "
\n", "\n", "
\n", "\n", " \n", "\n", "\"\"\".format(time_string, report_html_string)" ] }, { "cell_type": "code", "execution_count": 132, "metadata": {}, "outputs": [], "source": [ "# Write the html report\n", "f = open( 'CrimeReport_{}.html'.format(time_string), 'w' )\n", "f.write(html_report)\n", "f.close()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Conclusion\n", "This notebook comprehensively examined data from the calendar year 2017. We downloaded live data from 'Analyze Boston' portal, cleaned and prepared it for analysis. We explored both the spatial and non-spatial attributes to answer questions such prominent categories of assault, incident variability by week, by day. For the spatial analysis, we used powerful tools such as hotspot analysis, emerging hotspot analysis and time series clustering. Our analysis shows that the majority of the ward precincts fell into Cluster 2 with very low crime incident rate, so Boston is a very safe city.\n", "\n", "The Analyze Boston Crime Incident Reporting is updated frequently (while creating this notebook it was usually 2 days behind). This means that we can use the API to get the current crime incident situation in Boston. By using the most recent data available in the API we can determine if crimes are currently on the up or down trend. The top precincts for crime incidents can also be identified and we can determine which top precincts have consistently high crime incidents or new precincts with the highest number of incidents." ] } ], "metadata": { "esriNotebookRuntime": { "notebookRuntimeName": "ArcGIS Notebook Python 3 Advanced", "notebookRuntimeVersion": "10.7.1" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.7" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "**Table of Contents**", "title_sidebar": "Contents", "toc_cell": true, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }