{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Administration: Notify of license expiration\n",
"\n",
"> * ✏️ Needs Configuration\n",
"* 🔒 Requires Administrator Privileges\n",
"* 📝 Administration\n",
"* 🔔 Notifications\n",
"\n",
"## Introduction\n",
"\n",
"Enterprise Administrators can schedule this notebook to inform them of the status of Server licenses within their Enterprise deployment. If the license will expire within 30 days, a notification is sent to the administrator."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To get started, import the necessary libraries and connect to our GIS:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"import datetime as dt\n",
"import os\n",
"import csv\n",
"import smtplib\n",
"import logging\n",
"log = logging.getLogger()\n",
"\n",
"import requests\n",
"import pandas as pd\n",
"\n",
"from arcgis.gis import GIS\n",
"\n",
"gis = GIS(\"home\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notifications\n",
"\n",
"To optimize your notebook's utility, consider configuring it to alert someone if any licenses are set to expire within the next 30 days. We will achieve this by connecting to an external SMTP server and sending out emails, but you can write any code that connects to any external service to send out notifications. __View the Notifications notebook in the examples gallery for more information.__\n",
"\n",
"First, we'll configure the necessary variables and functions to send an email. We'll then query the Enterprise for the status of licenses, and if any are going to expire within 30 days, we'll send an email."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Security\n",
"\n",
"Before writing code that connects to any external service, it is vital to keep security in mind. Notebooks are commonly shared, code is copied and pasted, and passwords and private keys can inadvertently end up in the wrong hands.\n",
"\n",
"**Passwords and private keys should NEVER be stored in plaintext in a notebook**. There are many paradigms to securely store passwords and private keys. We will be storing all 'secrets' in a [private CSV item](https://www.esri.com/arcgis-blog/products/arcgis-online/sharing-collaboration/managing-security-and-findability-of-items-with-the-arcgis-sharing-model/) that is only accessible by you and your GIS administrator. \n",
"\n",
"> __Note__: that although this is more secure than storing your passwords as a plain string in a notebook, this is __NOT__ a totally secure solution—items aren't encrypted in the underlying server where they are stored. Consider storing your secrets in an encrypted files in `/arcgis/home/`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Make a private, empty `csv` file to store password information. Use the datetime module to ensure a unique file name:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'1548193826'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"now = dt.datetime.now()\n",
"now_ts = str(int(now.timestamp()))\n",
"now_ts"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"secrets_file = 'secrets_' + now_ts + '.csv'"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Secret CSV Item ID = 'b2b0604160964109bf4a5a0ad2500abe'\n"
]
},
{
"data": {
"text/html": [
"
\n",
"
\n",
" \n",
" \n",
" \n",
"
\n",
"\n",
"
\n",
" secrets_1548193826\n",
" \n",
" CSV by portaladmin\n",
" Last Modified: January 22, 2019\n",
" 0 comments, 0 views\n",
"
\n",
"
\n",
" "
],
"text/plain": [
""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Then, make an empty .csv file with headers\n",
"with open(secrets_file, \"w\") as f:\n",
" f.write(\"secret_key,secret_value\\n\")\n",
"\n",
"# Publish it as a private item only you and admins can see\n",
"secret_csv_item = gis.content.add({'access':'private'}, secrets_file)\n",
"secret_csv_item.protect()\n",
"print(\"Secret CSV Item ID = '{}'\".format(secret_csv_item.id))\n",
"secret_csv_item"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"SECRET_CSV_ITEM_ID = '5c7b389393394cc7ac91c91cdced92a6'"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"def add_secret(secret_key, secret_value, gis=gis,\n",
" secret_csv_item_id = SECRET_CSV_ITEM_ID):\n",
" \"\"\"Appends a new secret to the secrets.csv item\"\"\"\n",
" item = gis.content.get(secret_csv_item_id)\n",
" home_dir = os.path.join(os.path.sep, \"arcgis\", \"home\")\n",
" with open(item.download(home_dir, 'a') as local_item_file:\n",
" local_item_file.write(\"{},{}\\n\".format(secret_key, secret_value))\n",
" return item.update({}, local_item_file.name)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"add_secret(\"smtp_email_password\", \"\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__You MUST modify the below cell__ to update `secret_csv_item_id`, `smtp_server_url`, `username`, and any other information needed to connect to your external smtp server. This includes making sure your _secrets.csv_ file item contains the `smtp_email_password` entry."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"path = r\"/arcgis/home\"\n",
"SECRET_CSV_ITEM_ID = \"\"\n",
"def get_secrets(gis=gis,\n",
" secret_csv_item_id = SECRET_CSV_ITEM_ID):\n",
" \"\"\"Returns the secrets.csv file as a dict of secret_key : secret_value\"\"\"\n",
" item = gis.content.get(secret_csv_item_id)\n",
" with open(item.download(), 'r') as local_item_file:\n",
" reader = csv.DictReader(local_item_file)\n",
" return { rows['secret_key'] : rows['secret_value'] for rows in reader}\n",
"\n",
"def send_email_smtp(recipients, message,\n",
" subject=\"Message from your Notebook\"):\n",
" \"\"\"Sends the `message` string to all of the emails in the \n",
" `recipients` using the configured SMTP email server.\"\"\"\n",
" # Set up server and credential variables\n",
" smtp_server_url = \"smtp.mail.yahoo.com\"\n",
" smtp_server_port = 587\n",
" sender= \"someone@example.com\"\n",
" username = \"someone@example.com\"\n",
" secrets = get_secrets()\n",
" password = secrets[\"smtp_email_password\"]\n",
"\n",
" # Instantiate our server, configure the necessary security\n",
" server = smtplib.SMTP(smtp_server_url, smtp_server_port)\n",
" server.ehlo()\n",
" server.starttls() # Needed if TLS is required for the SMTP server\n",
" server.login(username, password)\n",
"\n",
" # For each recipient, construct the message and attempt to send\n",
" did_succeed = True\n",
" for recipient in recipients:\n",
" try:\n",
" message_body = '\\r\\n'.join(['To: {}'.format(recipient),\n",
" 'From: {}'.format(sender),\n",
" 'Subject: {}'.format(subject),\n",
" \"\",\n",
" '{}'.format(message)])\n",
" print(f\"{message_body}\")\n",
" server.sendmail(sender, [recipient], message_body)\n",
" except Exception as e:\n",
" log.warning(\"Failed sending message to {}\".format(recipient))\n",
" log.warning(e)\n",
" did_succeed = False\n",
" \n",
" # Cleanup and return\n",
" server.quit()\n",
" return did_succeed"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get a list of the Enterprise servers\n",
"\n",
"The `gis.admin` property allows administrators to use the [`PortalAdminManager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#portaladminmanager) to access the root for administering the portal, including managing all the servers. We can use the `servers` property to create an instance of the [`Server Manager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#servermanager). From the Server Manager, we can `list()` all the servers to then access information on the licenses."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"gis_server_mgr = gis.admin.servers"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[,\n",
" ]"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"gis_servers = gis_server_mgr.list()\n",
"gis_servers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Server Licensing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create an instance of a [`Server`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.server.html#server) to represent the system's hosting server:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"hosting_server = gis_servers[1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Access the Server License Manager\n",
"\n",
"Access the [`System`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#system) class with the `system` property on the server to work with the [`LicenseManager`](https://esri.github.io/arcgis-python-api/apidoc/html/arcgis.gis.admin.html#licensemanager). The `licenses` property returns the status of the ArcGIS Server and Extensions licenses, including the expiration dates.\n",
"\n",
"\n",
"The license manager is available as a Python dictionary. Retrieve the license manager and use the `keys()` method to return the key names to use to retrieve information from the server:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"dict_keys(['edition', 'level', 'datafeature', 'extensions', 'features'])"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"server_lic_resource = hosting_server.system.licenses\n",
"server_lic_resource.keys()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return the licenses for ArcGIS Server and all extensions using the license manager `features` key."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Datetimes\n",
"\n",
"A core component of this notebook will be testing whether a license expires within 30 days. The below code cells create helper functions that use the `datetime` library to calculate how many days until the licenses expire.\n",
"\n",
"The first function returns a datetime object of the expiration date:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"def datetime_of_timestamp(timestamp_value) -> datetime:\n",
" \"\"\"Esri stores dates as timestamps in milliseconds. Use this \n",
" function to return a datetime object from a timestamp value.\"\"\"\n",
" timestamp_epoch = int(timestamp_value/1000)\n",
" dt_value = dt.datetime.fromtimestamp(timestamp_epoch)\n",
" return dt_value"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a `timedelta` object to represent a 30 day time period and a representation for the current datetime."
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"# Create the timedelta object representing a 30 day time period\n",
"days_30 = dt.timedelta(days=30)\n",
"\n",
"# Create the datetime representing current time\n",
"now = dt.datetime.now()"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"def expires_in_30_days(datetime_obj):\n",
" \"\"\"Return a boolean value to represent whether a datetime is within 30 \n",
" days from now.\"\"\"\n",
" if datetime_obj < now:\n",
" print(\"Date has already past. This function determines whether a \\\n",
" future date is within 30 days from now.\")\n",
" else:\n",
" if datetime_obj - now <= days_30:\n",
" return True\n",
" else:\n",
" return False"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Server Licensing"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"server_license_status = {}\n",
"for feature_license in server_lic_resource['features']:\n",
" date_of_expiration = datetime_of_timestamp(feature_license['expiration'])\n",
" if date_of_expiration - now <= days_30:\n",
" server_license_status[feature_license['displayName']] = expires_in_30_days(date_of_expiration)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'ArcGIS Server Advanced Core': True,\n",
" 'Workflow Manager Extension for ArcGIS Server': True,\n",
" 'StreetMap Premium North America for ArcGIS Enterprise': True,\n",
" 'Business Analyst Extension for ArcGIS Server': True,\n",
" 'ArcGIS for INSPIRE': True,\n",
" 'GeoAnalytics Server Core': True,\n",
" 'StreetMap Premium Europe for ArcGIS Enterprise': True,\n",
" 'Esri Defense Mapping for Server': True,\n",
" 'Esri Production Mapping for Server': True,\n",
" 'Network Analyst Extension for ArcGIS Server': True,\n",
" 'StreetMap Premium Mid East and Africa for ArcGIS Enterprise': True,\n",
" 'SDE Enterprise Core': True,\n",
" 'ArcGIS Aviation: Airports for Server': True,\n",
" 'Utility Network Management for ArcGIS Server': True,\n",
" 'World Geocoder for ArcGIS Enterprise': True,\n",
" 'ArcGIS Aviation: Charting for Server': True,\n",
" 'ArcGIS Maritime: Charting for Server': True,\n",
" 'Location Referencing for Server': True,\n",
" 'StreetMap Premium Asia Pacific for ArcGIS Enterprise': True,\n",
" 'ArcGIS Data Reviewer Server': True,\n",
" 'Esri Roads and Highways for ArcGIS Server': True,\n",
" 'ArcGIS Maritime: Bathymetry for Server': True,\n",
" 'StreetMap Premium Japan for ArcGIS Enterprise': True,\n",
" 'ArcGIS Server Enterprise': True,\n",
" 'Image Server Core': True,\n",
" 'Data Interoperability Extension for ArcGIS Server': True,\n",
" 'StreetMap Premium Latin America for ArcGIS Enterprise': True,\n",
" 'Roadway Reporter for Esri Roads and Highways': True}"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"server_license_status"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Notifications \n",
"Send email to appropriate personnel if any licenses expire within 30 days"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Use pandas dataframes to write the license status ouput to a file and add to Portal\n",
"\n",
"#### Enterprise Server Licenses\n",
"Check to see if the dictionary is empty. If it is not, create a pandas dataframe, write it out to a csv file and add it to \n",
"the portal as an item. Then send an email containing the item id value of\n",
"the csv file containing the names of licenses expiring in 30 days.:"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"To: someone@example.com\r\n",
"From: someone_else@example.com\r\n",
"Subject: Message from your Notebook\r\n",
"\r\n",
"See Item for Server Licenses expiring in 30 days:\n",
"\tItem Id: 977d93d8bbeb4961906449876ea4ec3c\n"
]
}
],
"source": [
"if server_license_status:\n",
" out_dir = r\"/arcgis/home/\"\n",
" file_name = \"enterprise_server_license_status_\" + now_ts + \".csv\"\n",
" \n",
" server_license_df = pd.DataFrame(data=server_license_status, \n",
" index=['exp_within_30_days']).T\n",
" server_license_df.to_csv(os.path.join(out_dir, file_name), \n",
" index_label=\"Server Feature\")\n",
" server_license_exp = gis.content.add({'access':'private'}, \n",
" os.path.join(out_dir, file_name))\n",
" \n",
" server_exp_message = (f\"See Item for Server Licenses expiring in 30 days:\"\n",
" f\"\\n\\tItem Id: {server_license_exp.itemid}\")\n",
" send_email_smtp(recipients=['someone@example.com'], \n",
" message=server_exp_message)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conclusion\n",
"This notebook queries the `gis.admin` module to find out the status of licenses for the server component of the Enterprise. If any licenses are set to expire within 30 days, the notebook sends an email notfication to appropriate personnel."
]
}
],
"metadata": {
"esriNotebookRuntime": {
"notebookRuntimeName": "ArcGIS Notebook Python 3 Standard",
"notebookRuntimeVersion": "10.7.1"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}