{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Content Management: Create service report by folder" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "> * πŸ‘Ÿ Ready To Run!\n", "* πŸ”’ Requires Administrator Privileges\n", "* πŸ“¦Content Management\n", "* πŸ—„οΈ Administration\n", "* πŸ”” Notifications\n", "\n", "An organization often configures ArcGIS Enterprise with multiple server machines functioning in varying capacities. Each server contains any number of services organized in different ways, often containing hundreds of folders. It is crucial for Enterprise administrators to quickly report on the services running on each server. Administrators can create reports revealing a wealth of information about each service." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook will make a prelimary report on all the services located on the various servers organized by folder and create date." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import Libraries and Connect to your GIS" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get started, import the necessary libraries and connect to your GIS." ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "import os\n", "from urllib.parse import urlparse\n", "import datetime as dt\n", "\n", "import pandas as pd\n", "from IPython.display import display\n", "\n", "from arcgis.gis import GIS\n", "from arcgis.gis import Item" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "gis = GIS(\"home\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Access the Servers in the GIS" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> **NOTE:** You must be an administrator to perform the steps below." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A GIS Enterprise contains multiple servers comprising the GIS. As an administrator, you can access various administrative capabilities in the [`gis.admin`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html) module, including those that help manage the servers. The classes in this module are accessible from the `admin` property of the gis. \n", "\n", "The servers in an Enterprise GIS perform different roles. To perform these roles, the servers have different architectures that necessitate using different means to administer each server. We can use the Python API to list the servers according to role so we can administer them appropriately. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Using the Enterprise [`Federation`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#federation)\n", "\n", "First we can use the [`admin`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html) module (availiable as the `admin` property of the `gis`) to access the [`federation`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#federation) property. From the `federation`, we can obtain detailed information using the [`servers`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#arcgis.gis.admin.Federation.servers) property." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[{'id': 'GL2uFDL7K6bkTpSL',\n", " 'name': 'datascienceqa.esri.com',\n", " 'url': 'https://datascienceqa.esri.com/server',\n", " 'isHosted': True,\n", " 'adminUrl': 'https://datascienceqa.esri.com/server',\n", " 'serverRole': 'HOSTING_SERVER',\n", " 'serverFunction': '',\n", " 'webgisServerTrustKey': ''},\n", " {'id': 'Od7jP87qPQDaiEp9',\n", " 'name': 'dev0006138.esri.com',\n", " 'url': 'https://dev0006138.esri.com/turing',\n", " 'isHosted': False,\n", " 'adminUrl': 'https://dev0006138.esri.com/turing',\n", " 'serverRole': 'FEDERATED_SERVER',\n", " 'serverFunction': 'NotebookServer',\n", " 'webgisServerTrustKey': ''}]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fed_servers = gis.admin.federation.servers['servers']\n", "fed_servers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Using the [`ServerManager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#servermanager):" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The list of `server` dictionaries provides important information about each server. But we can directly retrieve individual [`server`]() objects to get information about the services on each server using the [`ServerManager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#servermanager) object. The [`get()`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#arcgis.gis.server.ServerManager.get) function allows direct reporting on the servers by role.\n", "\n", "First, we can use the [`servers`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#arcgis.gis.admin.PortalAdminManager.servers) property to retrieve the Server Manager." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "server_mgr = gis.admin.servers\n", "server_mgr" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we can create a lists of the `roles` and `functions` each server can play in order to create reports on the services on each type of server" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "function_list = ['ImageHosting', 'GeoAnalytics', \n", " 'RasterAnalytics', 'NotebookServer']\n", "roles = ['HOSTING_SERVER']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can print out the servers according to roles and functions:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "HOSTING_SERVER\n", "--------------------------------------------------\n", " https://datascienceqa.esri.com/server/admin\n", "NOTEBOOKSERVER\n", "--------------------------------------------------\n", " https://dev0006138.esri.com/turing/admin\n" ] } ], "source": [ "for server in server_mgr.get(role=roles[0]):\n", " print(f\"{roles[0]}\\n{'-' * 50}\")\n", " print(f\"{' ':2}{server.url}\")\n", "for fxn in function_list:\n", " server = server_mgr.get(function=fxn)\n", " if server:\n", " print(f\"{fxn.upper()}\\n{'-' * 50}\")\n", " print(f\"{' ':2}{server[0].url}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We know that the `Notebook Server` does not store services as do servers that perform the other functions in the GIS. So let's create a list of the gis servers that are not notebook servers" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "function_without_nb = [fxn for fxn in function_list \n", " if not fxn == 'NotebookServer']" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "gis_servers = []\n", "for server in server_mgr.get(role=roles[0]):\n", " gis_servers.append(server)\n", "for fxn in function_without_nb:\n", " if server_mgr.get(function=fxn):\n", " gis_servers.append(server_mgr.get(function=fxn)[0])" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "gis_servers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Access the *Services Manager* for each Server\n", "\n", "Now we have a list of the servers in our gis that store services. The [`services`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#arcgis.gis.server.Server.services) property of each server gets a [`ServiceManager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#servicemanager) object. The Services Manager allows you to manage the services on an ArcGIS Server machine and obtain useful information for administering or reporting on the services.\n", "\n", "There are many different types of services a GIS server can store. Each service type has its own set of properties. See the [Catalog](https://developers.arcgis.com/rest/services-reference/catalog.htm) reference for detailed properties on the varying types of services.\n", "\n", "We'll continue to use the [`server`]() module classes to report on the folders and services on our GIS servers.\n", "\n", "> **NOTE:** The folders on the GIS server are entirely different from the folders on the portal." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Helper functions for services information" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, let's define some helper functions to return specific types of data from the json returned by the server:\n", "\n", " * a function to return a [`datetime`](https://docs.python.org/3.7/library/datetime.html) object from the timestamps stored in the Organization" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "def datetime_of_timestamp(timestamp_value) -> dt.datetime:\n", " \"\"\"Esri stores dates as timestamps in milliseconds. Use this function to \n", " return a datetime object from a timestamp value.\"\"\"\n", " timestamp_epoch = int(timestamp_value/1000)\n", " dt_value = dt.datetime.fromtimestamp(timestamp_epoch)\n", " return dt_value" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " * a function to return the am or pm value from a datetime object" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "def am_or_pm(dt_value):\n", " if 0 <= dt_value.hour < 12:\n", " return \"AM\"\n", " if 12 <= dt_value.hour <=23:\n", " return \"PM\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " * a function to return the create date of the correpsonding item to a gis service\n", " > **NOTE:** Not all services have a correpsonding item. We'll use error-trapping to ignore those services that do not have a item since that's how we can get the create date." ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "def get_service_create_date(service):\n", " try:\n", " svc_item = Item(gis=gis,\n", " itemid=service.properties.portalProperties.portalItems[0]['itemID'])\n", " return datetime_of_timestamp(svc_item.created)\n", " except AttributeError as ae:\n", " pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Print `services` according to folder on the server\n", "\n", "Next, we'll create a list of the services in each folder on each server in the gis (that are not a Notebook Server). We define a helper function to pare down the list of folders on the server, removing the default `System` and `Utilities` folders from the list." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "def create_folder_list(server):\n", " exclude = ['System', 'Utilities']\n", " service_mgr = server.services\n", " return [folder for folder in service_mgr.folders if not folder in exclude]" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "https://datascienceqa.esri.com/server/admin\n", "==================================================\n", "\n", " Hosted\n", " Aggregation_of_tornado_lyr___Tornadoes_5011_to_tornado_lyr___StatePop_5011_dsxFeatureServer Mon Apr 15 2019 at 08:49:27 PM\n", " ALS_Clinic_90minDriveTime FeatureServer Mon Apr 15 2019 at 08:49:59 PM\n", " ALS_Clinic_City_Candidates_CA FeatureServer Mon Apr 15 2019 at 08:50:15 PM\n", " ALS_Clinics_CA FeatureServer Mon Apr 15 2019 at 08:49:48 PM\n", " ALS_Patients_CA FeatureServer Mon Apr 15 2019 at 08:50:26 PM\n", " CensusTractIncome FeatureServer Mon Apr 15 2019 at 08:50:37 PM\n", " Chennai_lakes FeatureServer Mon Apr 15 2019 at 08:50:48 PM\n", " Chennai_precipitation FeatureServer Mon Apr 15 2019 at 08:50:58 PM\n", " chennai_relief_centers FeatureServer Mon Apr 15 2019 at 08:51:09 PM\n", " Chennai_Solid_Waste_Management FeatureServer Mon Apr 15 2019 at 08:53:06 PM\n", " Commercial_Permits_since_2010 FeatureServer Mon Apr 15 2019 at 08:51:20 PM\n", " community_college_district_2 FeatureServer Mon Apr 15 2019 at 08:51:41 PM\n", " restaurants_pittsburgh FeatureServer Mon Apr 15 2019 at 08:51:52 PM\n", " set1_overwrite_HFS_csv2 FeatureServer Mon Apr 15 2019 at 11:42:57 PM\n", " set1_overwrite_manyHFS_csv_1 FeatureServer Mon Apr 15 2019 at 11:42:22 PM\n", " set1_overwrite_manyHFS_csv_2 FeatureServer Mon Apr 15 2019 at 11:42:42 PM\n", " set2_tpk_SD MapServer Tue Apr 16 2019 at 11:45:08 PM\n", " set2_vtpk_worldgreen VectorTileServer Tue Apr 16 2019 at 11:45:25 PM\n", " solid_waste_management FeatureServer Mon Apr 15 2019 at 08:52:02 PM\n", " Streets_Centerline FeatureServer Tue Apr 16 2019 at 11:43:48 PM\n", " Streets_Centerline MapServer Tue Apr 16 2019 at 11:44:39 PM\n", " SwimmingPoolLabels FeatureServer Mon Apr 15 2019 at 08:52:13 PM\n", " tornado_lyr_dsx FeatureServer Mon Apr 15 2019 at 08:52:25 PM\n", "\n", "\n", " Root\n", " SampleWorldCities MapServer Mon Apr 15 2019 at 08:30:04 PM\n", "\n", "\n", "\n", "\n" ] } ], "source": [ "for server in gis_servers:\n", " print(f\"{server.url}\\n{'=' * 50}\\n\")\n", " folders = create_folder_list(server)\n", " for folder in folders:\n", " if server.services.list(folder=folder):\n", " if folder == '/':\n", " print(f\"{' ':2}{'root'.capitalize()}\")\n", " else:\n", " print(f\"{' ':2}{folder.lower().capitalize()}\")\n", " else:\n", " pass\n", " for service in server.services.list(folder=folder):\n", " if 'portalProperties' in service.properties:\n", " try:\n", " create_date = get_service_create_date(service)\n", " if create_date is not None:\n", " print(f\"{' ':4}{service.properties.serviceName:40}\"\n", " f\"{service.properties.type:25}\"\n", " f\"{create_date.strftime('%a %b %d %Y at %I:%M:%S')}\"\n", " f\"{' ':1}{am_or_pm(create_date)}\")\n", " except RuntimeError as re:\n", " pass\n", " print(\"\\n\")\n", " print(\"\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> **NOTE:** Servers licensed to perform different functions will be duplicated in the above code." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create a Report of `services` in the GIS\n", "\n", "Printing the services to see our output is helpful, but now let's write the results into a report. We can write a function to capture the information in structures we can write out to files. First we'll write a function that returns a list of services on a server with information about the folder, type, and corresponding item create date for each service." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Write Helper Function" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "def create_service_list(server, folder):\n", " prop_list = []\n", " service_mgr = server.services\n", " for service in service_mgr.list(folder=folder):\n", " try:\n", " if 'portalProperties' in service.properties:\n", " try:\n", " prop_list.append((service.properties.serviceName[:30],\n", " folder,\n", " service.properties.type,\n", " service.properties['portalProperties']['portalItems'][0]['itemID'],\n", " get_service_create_date(service)))\n", " except RuntimeError as re:\n", " pass\n", " except KeyError as ke:\n", " prop_list.append((service.properties.serviceName,\n", " folder,\n", " service.properties.type))\n", " return prop_list" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Create a dataframe for the services on each server" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "service_dfs = {}\n", "for server in gis_servers:\n", " for folder in create_folder_list(server):\n", " if folder == '/':\n", " service_dfs[urlparse(server.url).path.split('/')[1] + '_Root'] = \\\n", " create_service_list(server, folder)\n", " else:\n", " service_dfs[urlparse(server.url).path.split('/')[1] + '_' + folder] = \\\n", " create_service_list(server, folder)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We just created a dictionary with each folder on a server as the key and a list of services (along with the metadata we specified) in the folder as the value. Let's print a list of the keys in the dictionary:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "server_Hosted\n", "server_Root\n" ] } ], "source": [ "for key,value in service_dfs.items():\n", " print(f\"{key}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we'll create a dictionary with the same folders as the keys, but instead of a list we'll create a dataframe with column headings corresponding to the service information we collected as the value:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [], "source": [ "column_names = ['serviceName', 'folder', 'serviceType', \n", " 'serviceItemId', 'createDate']" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "server_dfs = {}\n", "for server_folder, value in service_dfs.items():\n", " df = pd.DataFrame(data=service_dfs[server_folder], \n", " index=range(1, len(service_dfs[server_folder]) + 1), \n", " columns=column_names)\n", " server_dfs[server_folder] = df.sort_values(by='createDate')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can print out the dataframe of services in each folder by entering the folder name as the key. The services are ordered by the date the item was created." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
serviceNamefolderserviceTypeserviceItemIdcreateDate
1Aggregation_of_tornado_lyr___THostedFeatureServer87a5c5721d2a44b9b448d76f86ca9ce72019-04-15 20:49:27
4ALS_Clinics_CAHostedFeatureServerda518256b33b40ab88a9b6ef8d6fdc0c2019-04-15 20:49:48
2ALS_Clinic_90minDriveTimeHostedFeatureServera8e54794ff3744efbd47cdd47587d5632019-04-15 20:49:59
3ALS_Clinic_City_Candidates_CAHostedFeatureServer3c5eac34a1f947e599be28383332e0762019-04-15 20:50:15
5ALS_Patients_CAHostedFeatureServer701df587430f4f5fb9628408301492f32019-04-15 20:50:26
6CensusTractIncomeHostedFeatureServeraa0d700451844f5eb9edb92da6e1f4d42019-04-15 20:50:37
7Chennai_lakesHostedFeatureServer8a31eda0a4024afe80ec912d5a8e88492019-04-15 20:50:48
8Chennai_precipitationHostedFeatureServerbc99a9ed0325433985a058e772b2768f2019-04-15 20:50:58
9chennai_relief_centersHostedFeatureServerbd816a56ccca41ffa79d2d1e035fa2452019-04-15 20:51:09
11Commercial_Permits_since_2010HostedFeatureServer1e90436bab1944ef83b21b70379a3d332019-04-15 20:51:20
12community_college_district_2HostedFeatureServerebed14127a9e45c6b4e46fa2e081f9e42019-04-15 20:51:41
13restaurants_pittsburghHostedFeatureServerdbe7b6fd2708496380e89a282e6039972019-04-15 20:51:52
19solid_waste_managementHostedFeatureServerca32ecc92fe849a4b19d701aabad73302019-04-15 20:52:02
22SwimmingPoolLabelsHostedFeatureServer5d20cbd384c54c2392697c1d15b8b17e2019-04-15 20:52:13
23tornado_lyr_dsxHostedFeatureServerdb7dee8a971e4e29b86c99cb6adc5b362019-04-15 20:52:25
10Chennai_Solid_Waste_ManagementHostedFeatureServer8be83137196d44c78c1f96591daadcb12019-04-15 20:53:06
15set1_overwrite_manyHFS_csv_1HostedFeatureServer2a089c2253754833a94de7c32aec7e442019-04-15 23:42:22
16set1_overwrite_manyHFS_csv_2HostedFeatureServer624d493287ce4108b4eae08280f536dc2019-04-15 23:42:42
14set1_overwrite_HFS_csv2HostedFeatureServer5d1467d6c1624043bf4e1cff6b8e34162019-04-15 23:42:57
20Streets_CenterlineHostedFeatureServer063200292d9e4e24a0d71f469737adb02019-04-16 23:43:48
21Streets_CenterlineHostedMapServer76570246a6fc4d73a73e7142ab6080222019-04-16 23:44:39
17set2_tpk_SDHostedMapServer773aaea1b53e4798903d3b55e2f29dac2019-04-16 23:45:08
18set2_vtpk_worldgreenHostedVectorTileServer81e1e5820fa14bdfa89b674fbc89b1622019-04-16 23:45:25
\n", "
" ], "text/plain": [ " serviceName folder serviceType \\\n", "1 Aggregation_of_tornado_lyr___T Hosted FeatureServer \n", "4 ALS_Clinics_CA Hosted FeatureServer \n", "2 ALS_Clinic_90minDriveTime Hosted FeatureServer \n", "3 ALS_Clinic_City_Candidates_CA Hosted FeatureServer \n", "5 ALS_Patients_CA Hosted FeatureServer \n", "6 CensusTractIncome Hosted FeatureServer \n", "7 Chennai_lakes Hosted FeatureServer \n", "8 Chennai_precipitation Hosted FeatureServer \n", "9 chennai_relief_centers Hosted FeatureServer \n", "11 Commercial_Permits_since_2010 Hosted FeatureServer \n", "12 community_college_district_2 Hosted FeatureServer \n", "13 restaurants_pittsburgh Hosted FeatureServer \n", "19 solid_waste_management Hosted FeatureServer \n", "22 SwimmingPoolLabels Hosted FeatureServer \n", "23 tornado_lyr_dsx Hosted FeatureServer \n", "10 Chennai_Solid_Waste_Management Hosted FeatureServer \n", "15 set1_overwrite_manyHFS_csv_1 Hosted FeatureServer \n", "16 set1_overwrite_manyHFS_csv_2 Hosted FeatureServer \n", "14 set1_overwrite_HFS_csv2 Hosted FeatureServer \n", "20 Streets_Centerline Hosted FeatureServer \n", "21 Streets_Centerline Hosted MapServer \n", "17 set2_tpk_SD Hosted MapServer \n", "18 set2_vtpk_worldgreen Hosted VectorTileServer \n", "\n", " serviceItemId createDate \n", "1 87a5c5721d2a44b9b448d76f86ca9ce7 2019-04-15 20:49:27 \n", "4 da518256b33b40ab88a9b6ef8d6fdc0c 2019-04-15 20:49:48 \n", "2 a8e54794ff3744efbd47cdd47587d563 2019-04-15 20:49:59 \n", "3 3c5eac34a1f947e599be28383332e076 2019-04-15 20:50:15 \n", "5 701df587430f4f5fb9628408301492f3 2019-04-15 20:50:26 \n", "6 aa0d700451844f5eb9edb92da6e1f4d4 2019-04-15 20:50:37 \n", "7 8a31eda0a4024afe80ec912d5a8e8849 2019-04-15 20:50:48 \n", "8 bc99a9ed0325433985a058e772b2768f 2019-04-15 20:50:58 \n", "9 bd816a56ccca41ffa79d2d1e035fa245 2019-04-15 20:51:09 \n", "11 1e90436bab1944ef83b21b70379a3d33 2019-04-15 20:51:20 \n", "12 ebed14127a9e45c6b4e46fa2e081f9e4 2019-04-15 20:51:41 \n", "13 dbe7b6fd2708496380e89a282e603997 2019-04-15 20:51:52 \n", "19 ca32ecc92fe849a4b19d701aabad7330 2019-04-15 20:52:02 \n", "22 5d20cbd384c54c2392697c1d15b8b17e 2019-04-15 20:52:13 \n", "23 db7dee8a971e4e29b86c99cb6adc5b36 2019-04-15 20:52:25 \n", "10 8be83137196d44c78c1f96591daadcb1 2019-04-15 20:53:06 \n", "15 2a089c2253754833a94de7c32aec7e44 2019-04-15 23:42:22 \n", "16 624d493287ce4108b4eae08280f536dc 2019-04-15 23:42:42 \n", "14 5d1467d6c1624043bf4e1cff6b8e3416 2019-04-15 23:42:57 \n", "20 063200292d9e4e24a0d71f469737adb0 2019-04-16 23:43:48 \n", "21 76570246a6fc4d73a73e7142ab608022 2019-04-16 23:44:39 \n", "17 773aaea1b53e4798903d3b55e2f29dac 2019-04-16 23:45:08 \n", "18 81e1e5820fa14bdfa89b674fbc89b162 2019-04-16 23:45:25 " ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "server_dfs['server_Hosted']" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Write each dataframe out to a file to create report for further visualization." ] }, { "cell_type": "code", "execution_count": 51, "metadata": {}, "outputs": [], "source": [ "path = os.path.join(os.path.sep, 'arcgis', 'home')" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [], "source": [ "timestamp = str(dt.datetime.now())\n", "files_written = []\n", "for server_folder, df_val in server_dfs.items():\n", " file_path = os.path.join(path, f\"{server_folder}_{timestamp}.csv\")\n", " files_written.append(file_path)\n", " df_val.to_csv(file_path, \n", " index_label='service_oid')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, publish the files you've made" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", " server_Hosted_2019-04-17 20:28:30.113562\n", " \n", "
CSV by portaladmin\n", "
Last Modified: April 17, 2019\n", "
0 comments, 0 views\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", " server_Root_2019-04-17 20:28:30.113562\n", " \n", "
CSV by portaladmin\n", "
Last Modified: April 17, 2019\n", "
0 comments, 0 views\n", "
\n", "
\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for file_path in files_written:\n", " filename = os.path.basename(file_path)\n", " output_item = gis.content.add(item_properties={'type':'CSV'},\n", " data=file_path)\n", " display(output_item)" ] } ], "metadata": { "esriNotebookRuntime": { "notebookRuntimeName": "ArcGIS Notebook Python 3 Standard", "notebookRuntimeVersion": "10.7.1" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.2" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }