examples/basic-image-stats.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DSC160 Data Science and the Arts - Twomey - Spring 2020 - [dsc160.roberttwomey.com](http://dsc160.roberttwomey.com)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Image Stats\n",
    "\n",
    "This notebook demonstrates how to read, write, and calculate simple statistics on digital images. \n",
    "\n",
    "It uses the [numpy](https://numpy.org/), [scikit-image](https://scikit-image.org/), and [matplotlib](https://matplotlib.org/) libraries. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this command, the output of plotting commands is displayed inline within frontends like the Jupyter notebook, directly below the code cell that produced it. The resulting plots will then also be stored in the notebook document."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Digital Images as Arrays of Numbers\n",
    "\n",
    "Digital images are arrays of numbers (stored as floats, integers, etc.) with shape (Height, Width) and some number of image channels.\n",
    "\n",
    "This section uses numpy to make a simple 9 x 15 pixel, single channel image (black and white), with values in a checkerboard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an empty array (zeros) to hold the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "check = np.zeros((9, 15))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set every other position and every other row to be 1 (checkerboard pattern)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "check[::2, 1::2] = 1\n",
    "check[1::2, ::2] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the shape of our image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "check.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many pixels are in our image (aka what is the resolution, in pixels?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pixelcount = check.shape[0] * check.shape[1]\n",
    "pixelcount"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "another way to do this is with .size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "check.size"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the image using matplotlib/pyplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.imshow(check, cmap='gray', interpolation='nearest') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NOTE: The resulting image is 15 pixels wide by 9 pixels tall. In numpy indexing the first dimension corresponds to rows, and the second dimension corresponds to columns. This is different from how we usually think of images (and coordinates) as X, Y."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading a Black and White Image\n",
    "\n",
    "This section reads in a simple grayscale image from the skimage data dir and displays it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage import io\n",
    "import skimage\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the input image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filename = os.path.join(skimage.data_dir, 'camera.png')\n",
    "camera = io.imread(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the width?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the resolution?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pixelcount = camera.shape[0] * camera.shape[1]\n",
    "pixelcount"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.imshow(camera)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why are the colors so strange? Pyplot is using a Blue-Green colormap (mapping grayscale to blue-green). \n",
    "\n",
    "Let's try again, specifying a grayscale colormap:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.imshow(camera, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading a Color Image\n",
    "\n",
    "This section reads in an RGB (Red-Green-Blue) image and displays it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scipy.misc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read in a color image from the scipy miscellanous examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "face = scipy.misc.face()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the image shape?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "face.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NOTE: the image is 1024 pixels wide, 768 pixels tall, and has an additional 3 channels of information. Those three channels correspond to the Red, Blue, and Green color intensities of the image. This is the most typical format for color images. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the data type of the pixels?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "face.dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's show the image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ax = plt.imshow(face)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the resolution of the image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pixelcount = face.shape[0]*face.shape[1] # resolution\n",
    "pixelcount"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Technically there are 2,359,296 pixels (786432 * 3 channels), but we typically describe images based on their x and y resolution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's calculate some statistics. What are the max, mean, standard deviation and variance of the RGB values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.max(face, axis=(0,1)) # max rgb value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.mean(face, axis=(0,1)) # mean rgb value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.std(face, axis=(0,1)) # standard deviation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.var(face, axis=(0,1)) # variance of rgb value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These are not particularly useful! We want to remap the image to some more useful format."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert an RGB image to HSV (Hue, Saturation, Value)\n",
    "\n",
    "This section takes an RGB image (red, green, blue) and converts it to a HSV image (hue, saturation, value). Whereas each channel in the original correspond to one of the above colors, here the first channel will correspond to the Hue (color), Saturation (how intense the color is from white to pure hue), and Value (how light or dark the image is)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### HSV Color space \n",
    "\n",
    "HSV is closer to how humans perceive color. It has three components: hue, saturation, and value. This color space describes colors (hue) in terms of their shade (saturation) and their brightness value.\n",
    "\n",
    "#### Hue\n",
    "\n",
    "Hue is the color portion of the model, expressed as a number on a circular color wheel. In terms of 0 to 360 degrees: \n",
    "\n",
    "- Red falls between 0 and 60 degrees.\n",
    "- Yellow falls between 61 and 120 degrees.\n",
    "- Green falls between 121-180 degrees.\n",
    "- Cyan falls between 181-240 degrees.\n",
    "- Blue falls between 241-300 degrees.\n",
    "- Magenta falls between 301-360 degrees\n",
    "\n",
    "#### Saturation\n",
    "Saturation describes the amount of gray in a particular color, from 0 to 100 percent. Reducing this component toward zero introduces more gray and produces a faded effect. Sometimes, saturation appears as a range from just 0-1, where 0 is gray, and 1 is a primary color.\n",
    "\n",
    "#### Value\n",
    "\n",
    "Value works in conjunction with saturation and describes the brightness or intensity of the color, from 0-100 percent, where 0 is completely black, and 100 is the brightest and reveals the most color."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage import data\n",
    "from skimage.color import rgb2hsv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert our racoon image from RGB to HSV using the skimage rgb2hsv function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rgb_img = face\n",
    "hsv_img = rgb2hsv(rgb_img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Slice out the Hue, Saturation, and Value channels of our converted image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hue_img = hsv_img[:, :, 0]\n",
    "saturation_img = hsv_img[:,:, 1]\n",
    "value_img = hsv_img[:, :, 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display the H, S, V images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, (ax0, ax1, ax2, ax3) = plt.subplots(ncols=4, figsize=(16, 8))\n",
    "\n",
    "ax0.imshow(rgb_img)\n",
    "ax0.set_title(\"RGB image\")\n",
    "ax0.axis('off')\n",
    "ax1.imshow(hue_img, cmap='hsv')\n",
    "ax1.set_title(\"Hue channel (H)\")\n",
    "ax1.axis('off')\n",
    "ax2.imshow(saturation_img)\n",
    "ax2.set_title(\"Saturation channel (S)\")\n",
    "ax2.axis('off')\n",
    "ax3.imshow(value_img)\n",
    "ax3.set_title(\"Value channel (V)\")\n",
    "ax3.axis('off')\n",
    "\n",
    "fig.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the Hue channel looks like an extremely saturated version of whatever color that pixel has. The Saturation channel is darker for more grayscale parts of the image and lighter for more colorful parts of the image. The Value channel is light where the image is bright, and dark where the image is dark, independent of color."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the shape of one single channel of our new channels?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "value_img.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note numpy drops the third coordinate if we are dealing with a single channel image (whereas the RGB or HSV explicitly lists the third channel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Statistics on an HSV Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Image statistics calculated on the Hue, Saturation, and Value (Brightness) are more useful for us in image analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.max(hue_img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.min(hue_img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the average color (hue) of the image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_hue = np.mean(hue_img, axis=(0,1))\n",
    "mean_hue"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the average saturation of the image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_saturation = np.mean(saturation_img, axis=(0,1))\n",
    "mean_saturation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the average value (brightness) of the image?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean_brightness = np.mean(value_img)\n",
    "mean_brightness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the variange of the value channel?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_brightness = np.var(value_img) # variance of brightness channel\n",
    "var_brightness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NOTE: we can do a manual rgb -> luminance (grayscale) conversion using rgb2gray method, and check how close the result is to the Value channel produced by the skimage rgb2hsv conversion.\n",
    "\n",
    "The rgb2gray uses the following calculation:\n",
    "$$Y = 0.2125 R + 0.7154 G + 0.0721B$$\n",
    "\n",
    "from [https://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_rgb_to_gray.html](https://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_rgb_to_gray.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage.color import rgb2gray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gray_img = rgb2gray(face)\n",
    "ax = plt.imshow(gray_img, cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One simple way to compare two images (and see where they differ) is to calculate their difference, in this case the Value image produced by the rgb2hsv function, and the rgb2gray above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diff = value_img - gray_img\n",
    "ax = plt.imshow(diff, cmap=\"gray\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... this difference image just shows us that they gray image and value image are somewhat different. we will use the value image for our analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "- Scikit-image has a great [crash course on NumPy for images](https://scikit-image.org/docs/dev/user_guide/numpy_images.html)\n",
    "- Scikit-image docs on [RGB to HSV conversion](https://scikit-image.org/docs/dev/auto_examples/color_exposure/plot_rgb_to_hsv.html)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}