examples/audio-cluster-dim-reduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DSC160 Data Science and the Arts - Twomey - Spring 2020 - [dsc160.roberttwomey.com](http://dsc160.roberttwomey.com)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Audio Feature Extraction, Clustering, Dimensional Reduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook walks you through feature extraction from a small collection of audio clips representing different genres (pop, classical, jazz, metal, rock). \n",
    "\n",
    "It is a simplified version of the more comprehensive approach outlined in Tzanetakis and Cook's 2002 paper [Musical Genre Classification of Audio Signals](https://pdfs.semanticscholar.org/4ccb/0d37c69200dc63d1f757eafb36ef4853c178.pdf) from IEEE Transactions on Audio and Speech Processing. Many of the techniques described in that paper (timbral, beat, and pitch features) can be implemented using librosa and our numpy/scipy toolkits.\n",
    "\n",
    "This notebook works with the `mini-genres` dataset, a smaller version of the GITZAN dataset used in Tzanetakis and Cook's paper (see references at the end of this notebook). You can download minigenres here: http://opihi.cs.uvic.ca/sound/mini-genres.tar.bz2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Import necessary modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn\n",
    "\n",
    "import librosa\n",
    "import librosa.display\n",
    "\n",
    "from IPython.display import Audio\n",
    "\n",
    "import requests\n",
    "import os\n",
    "\n",
    "import sklearn\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download the dataset and unzip\n",
    "\n",
    "create data directory if it doesn't exist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists('../data/'):\n",
    "    os.makedirs('../data/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!wget -O ../data/mini-genres.tar.gz2 http://opihi.cs.uvic.ca/sound/mini-genres.tar.bz2 \n",
    "#!tar -xvf ../data/mini-genres.tar.gz2 -C ../data/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the Audio Data\n",
    "\n",
    "This section lets you explore two different examples from that dataset to observe differences in their time and frequency domain representations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using `librosa.load`, read in one of the files from the mini-genres dataset: `classical/classical.00001.au`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = \"../data/mini-genres/classical/classical.00001.au\"\n",
    "x, fs = librosa.load(filename, duration=30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the waveform for this file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "librosa.display.waveplot(x, sr=fs)\n",
    "plt.title('Sample')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `Audio` class to make a playable widget for this file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Audio(data=x, rate=fs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate the mel spectogram, using a log scale for magnitude:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hop_length = 256\n",
    "S = librosa.feature.melspectrogram(x, sr=fs, n_fft=4096, hop_length=hop_length)\n",
    "logS = librosa.power_to_db(abs(S))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the log mel spectogram:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(15, 5))\n",
    "librosa.display.specshow(logS, sr=fs, hop_length=hop_length, x_axis='time', y_axis='mel')\n",
    "plt.colorbar(format='%+2.0f dB')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repeat with another file from minigenres, `pop/pop.00004.au`\n",
    "\n",
    "Load the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = \"../data/mini-genres/pop/pop.00004.au\"\n",
    "x, fs = librosa.load(filename, duration=30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the waveform:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(12,6))\n",
    "librosa.display.waveplot(x, sr=fs)\n",
    "plt.title('Sample')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Make a playable widget for the file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Audio(data=x, rate=fs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculate and display the mel spectogram, using a log scale for magnitude:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# calculate mel spectogram:\n",
    "hop_length = 256\n",
    "S = librosa.feature.melspectrogram(x, sr=fs, n_fft=4096, hop_length=hop_length)\n",
    "logS = librosa.power_to_db(abs(S))\n",
    "\n",
    "# plot the mel spectogram\n",
    "plt.figure(figsize=(15, 5))\n",
    "librosa.display.specshow(logS, sr=fs, hop_length=hop_length, x_axis='time', y_axis='mel')\n",
    "plt.colorbar(format='%+2.0f dB')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Observations\n",
    "\n",
    "What differences do you see between the wave plots for these two audio files? Do they have a difference in amplitude, envelope?\n",
    "\n",
    "What differences do you see between the two audio files in the frequency domain (mel spectogram)? For instance, can you see the difference between pure tonal sound in the classic work (repeated intensities in time at the same frequency band), and spectogram information related to the vocals in the pop work (warbly lines in spectogram)?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Audio Feature Extraction\n",
    "\n",
    "In this section you will iterate over your downloaded images and calculate a number of image statistics, saving the results in a pandas dataframe.\n",
    "\n",
    "Here we have a function `extract_features()` that takes filename as an input and returns a list of audio stats calculated with librosa, including: \n",
    "  - MFCCs (5 coefficients)  \n",
    "  - Chroma (5 features)\n",
    "  - spectral centroid\n",
    "  - spectral bandwidth\n",
    "  - spectral roll-off\n",
    "  - zero crossing rate\n",
    "  \n",
    "Each of these is averaged (using np.mean) across the copmlete audio file, to produce one set of features for the file. That features contains 14 values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_features(filename):\n",
    "    y, sr = librosa.load(filename)\n",
    "    spec_cent = librosa.feature.spectral_centroid(y=y, sr=sr)\n",
    "    spec_bw = librosa.feature.spectral_bandwidth(y=y, sr=sr)\n",
    "    rolloff = librosa.feature.spectral_rolloff(y=y, sr=sr)\n",
    "    zcr = librosa.feature.zero_crossing_rate(y)\n",
    "\n",
    "    features = [np.mean(spec_cent), np.mean(spec_bw), \\\n",
    "                np.mean(rolloff), np.mean(zcr)]\n",
    "    \n",
    "    # chrome with n notes\n",
    "    chroma_stft = librosa.feature.chroma_stft(y=y, sr=sr, n_chroma=5)\n",
    "    # mfcc with n mfccs\n",
    "    mfcc = librosa.feature.mfcc(y=y, sr=sr, n_mfcc=5)\n",
    "    \n",
    "    for c in chroma_stft:\n",
    "        features.append(np.mean(c))\n",
    "    for c in mfcc:\n",
    "        features.append(np.mean(c))\n",
    "\n",
    "    return features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test this with single files from the mini-genres dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feat1 = extract_features('../data/mini-genres/classical/classical.00001.au')\n",
    "feat1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feat2 = extract_features('../data/mini-genres/pop/pop.00004.au')\n",
    "feat2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Extract Features from the dataset\n",
    "\n",
    "For this part, you loop over the 50 files in the `mini-genres` dataset, using the `extract_features()` function from above, and store the results in a pandas dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "AUDIO_DIR = '../data/mini-genres/'\n",
    "rows_list = []\n",
    "for directory in os.listdir(AUDIO_DIR):\n",
    "#     print(directory)\n",
    "    dirpath = os.path.join(AUDIO_DIR, directory)\n",
    "    if os.path.isdir(dirpath):\n",
    "        for filename in os.listdir(dirpath):\n",
    "#             print(filename)\n",
    "            if filename.endswith(\".au\"):\n",
    "                stats_dict = {}\n",
    "                feats = extract_features(os.path.join(AUDIO_DIR, directory, filename))\n",
    "                spec_cent, spec_bw, rolloff, zcr = feats[:4]\n",
    "                chroma = feats[4:9]\n",
    "                mfccs = feats[9:]\n",
    "                \n",
    "                stats_dict['c0'] = chroma[0]\n",
    "                stats_dict['c1'] = chroma[1]\n",
    "                stats_dict['c2'] = chroma[2]\n",
    "                stats_dict['c3'] = chroma[3]\n",
    "                stats_dict['c4'] = chroma[4]\n",
    "\n",
    "                stats_dict['m0'] = mfccs[0]\n",
    "                stats_dict['m1'] = mfccs[1]\n",
    "                stats_dict['m2'] = mfccs[2]\n",
    "                stats_dict['m3'] = mfccs[3]\n",
    "                stats_dict['m4'] = mfccs[4]\n",
    "\n",
    "                stats_dict['filename'] = filename\n",
    "                stats_dict['genre'] = directory\n",
    "                stats_dict['spec_cent'] = spec_cent\n",
    "                stats_dict['spec_bw'] = spec_bw\n",
    "                stats_dict['rolloff'] = rolloff\n",
    "                stats_dict['zcr'] = zcr\n",
    "                rows_list.append(stats_dict)\n",
    "\n",
    "summary_stats = pd.DataFrame(rows_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary_stats.head(15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "what do the summary stats look like for one of our files?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary_stats[summary_stats['filename'] == 'pop.00006.au']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Scale Features\n",
    "\n",
    "We will use the `sklearn.preprocessing` `StandardScaler` to scale all features to mean of `0.0` and std_dev of `1.0` across the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "grab just the stats from our data frame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "just_stats = summary_stats[[ 'c0', 'c1', 'c2', 'c3', 'c4', \\\n",
    "                            'm0', 'm1', 'm2', 'm3', 'm4', \\\n",
    "                            'zcr', 'spec_bw', 'spec_cent', 'rolloff']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scale the stats using the scaler:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_stats = scaler.fit_transform(just_stats)\n",
    "scaled_stats[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot the Scaled Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lets plot the scaled features for the first file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.bar(x=range(len(scaled_stats[0])), height=scaled_stats[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and see if they differ for the second file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.bar(x=range(len(scaled_stats[11])), height=scaled_stats[11])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: features 0-4 are the chroma, features 5-9 are the MFCC coefficients, and 10-13 are the zero crossing through spectral rolloff, "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Store features with Column names back in Data Frame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's add the column names back in and make a dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "col_names = [ 'c0', 'c1', 'c2', 'c3', 'c4', \\\n",
    "              'm0', 'm1', 'm2', 'm3', 'm4', \\\n",
    "              'zcr', 'spec_bw', 'spec_cent', 'rolloff']\n",
    "\n",
    "scaled_stats = pd.DataFrame(scaled_stats, columns = col_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_stats['genre'] = summary_stats['genre']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_stats.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display Features\n",
    "\n",
    "In this section we will produce bivariate plots, colored by genre.\n",
    "\n",
    "First will our stats by genre using `df.groupby()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = scaled_stats.groupby('genre')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot spectral bandwidth against spectral centroid, coloring by genre:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a plot \n",
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling\n",
    "\n",
    "# loop over groups\n",
    "for name, group in groups:\n",
    "    ax.plot(group.spec_cent, group.spec_bw, marker='o', linestyle='', label=name)\n",
    "ax.legend()\n",
    "plt.title('Spectral Centroid vs Spectral Bandwidth (by genre)', fontsize=16);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot spec_bw against zcr, coloring/labelling with genre:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling\n",
    "for name, group in groups:\n",
    "    ax.plot(group.zcr, group.spec_bw, marker='o', linestyle='', label=name)\n",
    "ax.legend()\n",
    "plt.title('Zero-Crossing Rate vs Spectral Bandwidth (by genre)', fontsize=16);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot zcr against roloff, coloring by genre:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling\n",
    "for name, group in groups:\n",
    "    ax.plot(group.zcr, group.rolloff, marker='o', linestyle='', label=name)\n",
    "ax.legend()\n",
    "plt.title('Zero-Crossing Rate vs Spectral Rolloff (by genre)', fontsize=16);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extension: Explore pairs of features across the dataset. Which features seem most useful for distinguishing between clips?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dimensional reduction\n",
    "\n",
    "This section uses the dimensional reduction technique UMAP to map the 14 dimensional feature data to a two dimensional embedding. This produces two coordinates for each audio file, based on the feature data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install umap-learn --user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import umap.umap_ as umap\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reducer = umap.UMAP()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding = reducer.fit_transform(scaled_stats.drop(['genre'], axis=1))\n",
    "embedding.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_stats['umap1'] = embedding[:,0]\n",
    "scaled_stats['umap2'] = embedding[:,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaled_stats.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clustering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "for_clustering = scaled_stats[['umap1', 'umap2']]\n",
    "for_clustering.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = KMeans(n_clusters=5, random_state=0).fit(for_clustering)\n",
    "summary_stats['cluster'] = kmeans.labels_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "summary_stats.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = summary_stats.groupby('cluster')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for name, group in groups:\n",
    "    print(\"cluster num {}: {} items\".format(name, len(group)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot by genre label\n",
    "using umap coordinates for x and y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = summary_stats.groupby('genre')\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling\n",
    "for name, group in groups:\n",
    "    ax.plot(group.umap1, group.umap2, marker='o', linestyle='', label=name)\n",
    "plt.title('UMAP projection of the Audio data (by Genre Label)', fontsize=16);\n",
    "ax.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot by cluster label\n",
    "\n",
    "using umap coordinates for x and y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "groups = summary_stats.groupby('cluster')\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling\n",
    "for name, group in groups:\n",
    "    ax.plot(group.umap1, group.umap2, marker='o', linestyle='', label=name)\n",
    "ax.legend()\n",
    "plt.title('UMAP projection of the Audio data (by Cluster number)', fontsize=16);\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "alterntive way to plot (using cluster as color with seaborn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plt.figure(figsize=(10,10))\n",
    "# plt.scatter(embedding[:, 0], embedding[:, 1], c=[sns.color_palette()[x] for x in summary_stats.cluster.to_list()])\n",
    "# plt.gca().set_aspect('equal', 'datalim')\n",
    "# plt.title('UMAP projection of the Audio data (by Cluster)', fontsize=16);\n",
    "# plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extensions\n",
    "- Evaluating:\n",
    "  - Plot both the genre label and k means cluster on the same graph, evaluating how well the clustering based on our features detected the distinct genres.\n",
    "  - Evaluate the accuracy of the k-means clustering, comparing cluster labels to genre labels.\n",
    "- Features:\n",
    "   - Try with a different number of MFCCs (20) and Chroma (12), repeating the exercise. How does this change your results?\n",
    "   - Try with other features\n",
    "- Clustering:\n",
    "  - Try with a different clustering method (affinity clustering, HDBSCAN)\n",
    "- How could we improve the results?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "* For a more comprehensive approach to genre recognition, see [Tzanetakis and Cook 'Musical Genre Classification of Audio Signals'](https://pdfs.semanticscholar.org/4ccb/0d37c69200dc63d1f757eafb36ef4853c178.pdf) from IEEE Transactions on Audio and Speech Processing 2002.\n",
    "* GITZAN dataset (from the above paper): http://opihi.cs.uvic.ca/sound/genres.tar.gz\n",
    "  * The dataset consists of 1000 audio tracks each 30 seconds long. It contains 10 genres namely, blues, classical, country, disco, hiphop, jazz, reggae, rock, metal and pop. Each genre consists of 100 sound clips.\n",
    "* `mini-genres` is a smaller subset of the above audio tracks: http://opihi.cs.uvic.ca/sound/mini-genres.tar.bz2"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}