train_and_deploy.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Starbucks Rewards: Predicting Consumer Responses"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project Overview"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project seeks to determines how do we take data and discover what are the offers that excite people? We want to know what is the most valuable offer there is, not just for the customers as a whole but at an individual personal level.\n",
    "\n",
    "Link to an academic paper where machine learning was applied to this type of problem: http://ceur-ws.org/Vol-3026/paper18.pdf"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Statement"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict if someone will reply to an offer. Transaction data and demographic information must be combined and gitHub\n",
    "customers will also have access to the data. We will analyze the attributes of customers to create customer classifications. This is a multi-class classification problem so the key metric we will use is f1-score. The simulating dataset only has one product, while Starbucks offers dozens of products. Therefore, this data set is a simplified version of the real Starbucks app."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "import pandas as pd\n",
    "import sagemaker\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sagemaker.xgboost.estimator import XGBoost\n",
    "from sklearn import preprocessing\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Exploration"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataset details\n",
    "\n",
    "profile.json\n",
    "* age (int) - age of the customer \n",
    "* became_member_on (int) - date when customer created an app account\n",
    "* gender (str) - gender of the customer (note some entries contain 'O' for other rather than M or F)\n",
    "* id (str) - customer id\n",
    "* income (float) - customer's income\n",
    "\n",
    "transcript.json\n",
    "* event (str) - record description (ie transaction, offer received, offer viewed, etc.)\n",
    "* person (str) - customer id\n",
    "* time (int) - time in hours since the start of the test. The data begins at time t=0\n",
    "* value - (dict of strings) - either an offer id or transaction amount depending on the record\n",
    "\n",
    "portfolio.json\n",
    "* id (string) - offer id\n",
    "* offer_type (string) - a type of offer ie BOGO, discount, informational\n",
    "* difficulty (int) - the minimum required to spend to complete an offer\n",
    "* reward (int) - the reward is given for completing an offer\n",
    "* duration (int) - time for the offer to be open, in days\n",
    "* channels (list of strings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "portfolio = pd.read_json('data/portfolio.json', orient='records', lines=True)\n",
    "profile = pd.read_json('data/profile.json', orient='records', lines=True)\n",
    "transcript = pd.read_json('data/transcript.json', orient='records', lines=True)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Profile Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(profile.columns)\n",
    "profile.hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "profile.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "profile.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "profile['gender'].value_counts().plot(kind='bar')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 17000 records where the average age is 63 years old and the average income is $65000 and there are 2175 missing data points from income and gender. We will need to consider what we do with these values later one. We can safely ignore the id as these don't mean anything. As for became_member_on we can confirm that most customers became a member in 2017 and 2018.\n",
    "\n",
    "With M corresponding to male, we can see that males accounts for the majority of the customers. **This may be a problem for our model, but if our accuracy is too low, then we can always normalize the presence of each area in the dataset to get predictions that aren’t skewed towards male customers**."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transcript Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript['event'].value_counts().plot(kind='bar')"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Given the 300000+ records, it makes sense that offer completed has the smallest value count and transactions has the most since there can't be more offers completed compared to offer viewed and offer viewed. We also see the test lasted 714 hours but this doesn't seem useful for our analysis and modeling stage."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Portfolio Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "portfolio.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "portfolio['reward'].value_counts()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that there are 8 rows with rewards but 2 that are just informational. So even if an offer was completed there would be no reward and discounts gives rewards with values 2,3, and 5 and have multiple diffuculties."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Missing Values"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Previously we observed that the missing values come from gender and income. We will add a 4th category for gender as U for unknown and fill the income with the mean since income values are appear to have a normal distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "profile['gender'] = profile['gender'].fillna('U')\n",
    "profile['income'] = profile['income'].fillna(profile['income'].mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "profile.info()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outliers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(profile[profile['age']>100])/len(profile)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('percentage of centenarians {0:.0%}%'.format(len(profile[profile['age']>100])/len(profile)))\n",
    "print('average age of customer', 63)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that so far the only possible outliers come form age as there are many Centenarian. Also out of a US population of approximately 300 million, there were approximatly 90000 centenarians (age 100+) or a prevalence of 0.3%. From above we see out of the customers make up 13% but the average age is high as well so we can ignore age for now."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: We will only consider customers who received offers. Customers who did not recieve an offer could not have viewed or responed to offfer so they offer no valid information for the puproses of predicting consumer response to offers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_transcript = transcript[transcript['event'].str.startswith('o')]\n",
    "new_transcript.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split / Explode a column of dictionaries into separate columns \n",
    "new_transcript = pd.concat([new_transcript.drop(['value'], axis=1), new_transcript['value'].apply(pd.Series)], axis=1)\n",
    "new_transcript.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_transcript = new_transcript.dropna(subset = [\"offer_id\"])\n",
    "\n",
    "temp1 = portfolio.rename(columns={\"id\": \"offer_id\"})\n",
    "temp2 = profile.rename(columns={\"person\": \"id\"})\n",
    "\n",
    "data = pd.merge(new_transcript, temp1, on=\"offer_id\").rename(columns={\"person\": \"id\"})\n",
    "data = pd.merge(data, temp2, on=\"id\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating additional features\n",
    "data['became_member_on'] = pd.to_datetime(data['became_member_on'],format='%Y%m%d')\n",
    "\n",
    "data[\"year\"] = data.became_member_on.dt.year\n",
    "data[\"month\"] = data.became_member_on.dt.month\n",
    "data[\"day\"] = data.became_member_on.dt.day\n",
    "\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "to_drop = ['offer id',\n",
    "           'event',\n",
    "           'id',\n",
    "           'became_member_on',\n",
    "           'reward_y',\n",
    "           'reward_x',\n",
    "           'time',\n",
    "           'offer_type']\n",
    "           \n",
    "data.drop(to_drop, inplace=True, axis=1)\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_vals = {offer_id for offer_id in data['offer_id']}\n",
    "print(target_vals)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set target to be offer_id\n",
    "data['target'] = data['offer_id']\n",
    "data.drop('offer_id', inplace=True, axis=1)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert object type to avoid error\n",
    "lbl = preprocessing.LabelEncoder()\n",
    "data['gender'] = lbl.fit_transform(data['gender'].astype(str))\n",
    "data['target'] = lbl.fit_transform(data['target'].astype(str))\n",
    "data['channels'] = lbl.fit_transform(data['gender'].astype(str))\n",
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data.drop_duplicates()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Class distributions of offers completed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_count = defaultdict(int)\n",
    "for offer in data['target']:\n",
    "    if offer in offer_count:\n",
    "        offer_count[offer]+=1\n",
    "    else:\n",
    "        offer_count[offer] = 1\n",
    "\n",
    "class_count = pd.DataFrame.from_dict(offer_count, orient='index')\n",
    "bar_plot = class_count.plot.bar(title=\"Number of occurrences\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_csv('data.csv', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train, test = train_test_split(data, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train.to_csv('data/train/train.csv', index = False)\n",
    "test.to_csv('data/test/test.csv', index = False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload to S3 bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_session = sagemaker.Session()\n",
    "\n",
    "bucket = \"my-project-bucket-123\"\n",
    "region =\"us-east-1\" \n",
    "role = \"arn:aws:iam::657240468511:role/service-role/AmazonSageMaker-ExecutionRole-20211225T133929\"\n",
    "\n",
    "print(\"Default Bucket: {}\".format(bucket))\n",
    "print(\"AWS Region: {}\".format(region))\n",
    "print(\"RoleArn: {}\".format(role))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_path_to_data = sagemaker.Session().upload_data(bucket=bucket, \n",
    "                                                  path='data/train', \n",
    "                                                  key_prefix='train')\n",
    "\n",
    "s3_path_to_data = sagemaker.Session().upload_data(bucket=bucket, \n",
    "                                                  path='data/test', \n",
    "                                                  key_prefix='test')\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Benchmark"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using multiclass logistic regression against which we can benchmark."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "benchmark_data = pd.read_csv ('data.csv')\n",
    "benchmark_train, benchmark_test = train_test_split(data, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "model = LogisticRegression(multi_class='multinomial', solver='saga', max_iter=10000 )\n",
    "X = benchmark_train.iloc[:, :-1]\n",
    "y = benchmark_train.iloc[:, -1:]\n",
    "model.fit(X, y.values.ravel())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import f1_score\n",
    "\n",
    "preds = model.predict(benchmark_test.iloc[:,:-1])\n",
    "score=f1_score(benchmark_test[\"target\"], preds, average='weighted')\n",
    "print(f\"f1-score: {score}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Training\n",
    "**Note:** You will need to use the `train.py` script to train your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = 's3://{}/output'.format(bucket)\n",
    "input_data = 's3://{}/'.format(bucket)\n",
    "\n",
    "metric_definitions = [{'Name': 'validation:f1', 'Regex': '.*\\[[0-9]+\\].*#011validation-f1:([-+]?[0-9]*\\.?[0-9]+(?:[eE][-+]?[0-9]+)?).*'}]\n",
    "\n",
    "xgb_estimator = XGBoost(\n",
    "    entry_point=\"train.py\",\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.2xlarge\",\n",
    "    framework_version=\"1.3-1\",\n",
    "    output_path=output_path,\n",
    "\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_input = input_data + \"train\"\n",
    "test_input = input_data + \"test\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_estimator.fit({'train': train_input, 'validation': test_input})"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standout suggestions"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameter Tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "from sagemaker.tuner import (\n",
    "    IntegerParameter,\n",
    "    CategoricalParameter,\n",
    "    ContinuousParameter,\n",
    "    HyperparameterTuner,\n",
    ")\n",
    "\n",
    "hyperparameter_ranges = {\n",
    "    \"max_depth\": IntegerParameter(2, 8),\n",
    "    \"eta\": ContinuousParameter(0.1, 0.5),\n",
    "    \"num_round\" : CategoricalParameter([10, 50, 100]),\n",
    "}\n",
    "\n",
    "objective_metric_name = \"validation:f1\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_estimator = XGBoost(\n",
    "    entry_point=\"train.py\",\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.2xlarge\",\n",
    "    framework_version=\"1.3-1\",\n",
    "    output_path=output_path,\n",
    "\n",
    ")\n",
    "\n",
    "tuner = HyperparameterTuner(\n",
    "    xgb_estimator,\n",
    "    objective_metric_name,\n",
    "    hyperparameter_ranges, \n",
    "    max_jobs=2, \n",
    "    max_parallel_jobs=4 \n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tuner.fit({'train': train_input, \"validation\": test_input})"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The final results compared to the benchmark result is higher by a large margin. \n",
    "Therefore the final model and solution is significant enough to have adequately solved the problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tuner.describe()['BestTrainingJob']['FinalHyperParameterTuningJobObjectiveMetric']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_estimator = tuner.describe()['BestTrainingJob']['TunedHyperParameters']\n",
    "print(best_estimator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_depth = int(best_estimator[\"max_depth\"])\n",
    "eta = float(best_estimator['eta'])\n",
    "num_round = int(best_estimator['num_round'][1:-1])\n",
    "\n",
    "print(\"max_depth: {}\".format(max_depth))\n",
    "print(\"eta: {}\".format(eta))\n",
    "print(\"num_round: {}\".format(num_round))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparameters = {\n",
    "    \"max_depth\": max_depth,\n",
    "    \"eta\": eta,\n",
    "    \"num_round\": num_round\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = 's3://{}/output'.format(bucket)\n",
    "input_data = 's3://{}/'.format(bucket)\n",
    "\n",
    "xgb_estimator = XGBoost(\n",
    "    entry_point=\"train.py\",\n",
    "    hyperparameters=hyperparameters,\n",
    "    role=role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.2xlarge\",\n",
    "    framework_version=\"1.3-1\",\n",
    "    output_path=output_path,\n",
    "\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_estimator.fit({'train': train_input, \"validation\": test_input})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pt_model_data = xgb_estimator.model_data\n",
    "print(\"Model artifact saved at:\\n\", pt_model_data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Deploying and Querying"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import gmtime, strftime\n",
    "\n",
    "timestamp_prefix = strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "endpoint_name = \"inference-pipeline-ep-\" + timestamp_prefix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor=xgb_estimator.deploy(instance_type=\"ml.m5.large\", initial_instance_count=1, endpoint_name=endpoint_name) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.predictor import Predictor\n",
    "\n",
    "payload = '0,10,7,0,69,70000.0,2018,5,14'\n",
    "\n",
    "predictor = Predictor(\n",
    "    endpoint_name=endpoint_name,\n",
    "    sagemaker_session=sagemaker_session,\n",
    "    serializer=sagemaker.serializers.CSVSerializer(),\n",
    "    content_type=\"text/csv\",\n",
    "    accept=\"application/json\")\n",
    "\n",
    "print(predictor.predict(payload))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.g4dn.xlarge",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}