{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d321d996",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 43679 -- Interactive Visualization\n",
    "# 2025 - 2026\n",
    "# 2nd semester\n",
    "# Lab 1 - EDA (guided)\n",
    "# ver 1.0 - 2026-02-20 Initial version\n",
    "# ver 1.1 - 2026-02-23  Added more comments and explanations\n",
    "# ver 1.2 - 2026-02-24  Added code for additional visualizations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d44c354e",
   "metadata": {},
   "source": [
    "# Lab 01<br>Task 1: Exploratory Data Analysis with Pandas & Seaborn\n",
    "\n",
    "This task serves two purposes. It introduces you to some of the basic tools to start understanding datasets and shows you why descriptive statistics may not be enough to understand the nature of a dataset.\n",
    "\n",
    "Also, this task also walks you through some basic visualizations of the datasets to show how the type of visualization matters when trying to understand the data.\n",
    "\n",
    "Additionally, this simple first task also serves the purpose of getting you acquainted with Jupyter notebooks.\n",
    "\n",
    "**Dataset:** `datasaurus.csv`\n",
    "\n",
    "---\n",
    "\n",
    "### Objectives\n",
    "\n",
    "By the end of this task you will be able to:\n",
    "- Use `pandas` to inspect a dataset's structure, types, and summary statistics\n",
    "- Apply grouped aggregations to compare subsets of data\n",
    "- Use `seaborn` to produce scatter plots that reveal structure invisible to statistics\n",
    "- Articulate *why* visualisation is an essential — not optional — step in data analysis\n",
    "\n",
    "---\n",
    "\n",
    "### Context\n",
    "\n",
    "The **Datasaurus Dozen** is a collection of 13 small datasets created by Matejka & Fitzmaurice (2017) to demonstrate a modern version of Anscombe's Quartet.\n",
    "\n",
    "This task will take you through the same journey a data analyst faces: you will start with raw numbers, run the usual summaries, and then discover, through visualisation, that numbers alone were hiding the story.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "350a4fd8",
   "metadata": {},
   "source": [
    "## Part 1: Load and Inspect the Data\n",
    "\n",
    "Start by importing the libraries you need and loading the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ed1a7a01",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Configure plot style\n",
    "sns.set_theme(style='whitegrid', palette='tab10')\n",
    "plt.rcParams['figure.dpi'] = 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9cf77ef2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>dataset</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>dino</td>\n",
       "      <td>55.3846</td>\n",
       "      <td>97.1795</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>dino</td>\n",
       "      <td>51.5385</td>\n",
       "      <td>96.0256</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>dino</td>\n",
       "      <td>46.1538</td>\n",
       "      <td>94.4872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>dino</td>\n",
       "      <td>42.8205</td>\n",
       "      <td>91.4103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>dino</td>\n",
       "      <td>40.7692</td>\n",
       "      <td>88.3333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>dino</td>\n",
       "      <td>38.7179</td>\n",
       "      <td>84.8718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>dino</td>\n",
       "      <td>35.6410</td>\n",
       "      <td>79.8718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>dino</td>\n",
       "      <td>33.0769</td>\n",
       "      <td>77.5641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>dino</td>\n",
       "      <td>28.9744</td>\n",
       "      <td>74.4872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>dino</td>\n",
       "      <td>26.1538</td>\n",
       "      <td>71.4103</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  dataset        x        y\n",
       "0    dino  55.3846  97.1795\n",
       "1    dino  51.5385  96.0256\n",
       "2    dino  46.1538  94.4872\n",
       "3    dino  42.8205  91.4103\n",
       "4    dino  40.7692  88.3333\n",
       "5    dino  38.7179  84.8718\n",
       "6    dino  35.6410  79.8718\n",
       "7    dino  33.0769  77.5641\n",
       "8    dino  28.9744  74.4872\n",
       "9    dino  26.1538  71.4103"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the dataset\n",
    "df = pd.read_csv('datasaurus.csv')\n",
    "\n",
    "# Preview the first rows\n",
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2e51209",
   "metadata": {},
   "source": [
    "### 1.1. Structure and data types\n",
    "\n",
    "Before computing anything, always understand what you are working with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6a45f4e3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape: (1846, 3)\n",
      "\n",
      "Dtypes:\n",
      "dataset     object\n",
      "x          float64\n",
      "y          float64\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "# Shape of the dataset (rows, columns)\n",
    "print('Shape:', df.shape)\n",
    "\n",
    "# Column names and data types\n",
    "print('\\nDtypes:')\n",
    "print(df.dtypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d01329b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unique datasets: 13\n",
      "\n",
      "Rows per dataset:\n",
      "dataset\n",
      "dino          142\n",
      "away          142\n",
      "h_lines       142\n",
      "v_lines       142\n",
      "x_shape       142\n",
      "star          142\n",
      "high_lines    142\n",
      "dots          142\n",
      "circle        142\n",
      "bullseye      142\n",
      "slant_up      142\n",
      "slant_down    142\n",
      "wide_lines    142\n",
      "Name: count, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# How many unique sub-datasets are there, and how many rows does each contain?\n",
    "print('Unique datasets:', df['dataset'].nunique())\n",
    "print('\\nRows per dataset:')\n",
    "print(df['dataset'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1545a53f",
   "metadata": {},
   "source": [
    "### 1.2. Overall summary statistics\n",
    "\n",
    "Use `describe()` to get a global numerical summary of `x` and `y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a92b670e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1846.00</td>\n",
       "      <td>1846.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>54.27</td>\n",
       "      <td>47.84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>16.71</td>\n",
       "      <td>26.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>15.56</td>\n",
       "      <td>0.02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>41.07</td>\n",
       "      <td>22.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>52.59</td>\n",
       "      <td>47.59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>67.28</td>\n",
       "      <td>71.81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>98.29</td>\n",
       "      <td>99.69</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             x        y\n",
       "count  1846.00  1846.00\n",
       "mean     54.27    47.84\n",
       "std      16.71    26.85\n",
       "min      15.56     0.02\n",
       "25%      41.07    22.56\n",
       "50%      52.59    47.59\n",
       "75%      67.28    71.81\n",
       "max      98.29    99.69"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Summary statistics for the entire dataset\n",
    "df[['x', 'y']].describe().round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16b1a9e3",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 2: Grouped Statistics: The Reveal\n",
    "\n",
    "The dataset column holds 13 different named groups. Let's compute summary statistics **per group** and see if the groups differ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e7693c95",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"2\" halign=\"left\">x</th>\n",
       "      <th colspan=\"2\" halign=\"left\">y</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dataset</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>away</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bullseye</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>circle</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.76</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dino</th>\n",
       "      <td>54.26</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dots</th>\n",
       "      <td>54.26</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>h_lines</th>\n",
       "      <td>54.26</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>high_lines</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>slant_down</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>slant_up</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>star</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.93</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>v_lines</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>wide_lines</th>\n",
       "      <td>54.27</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.83</td>\n",
       "      <td>26.94</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>x_shape</th>\n",
       "      <td>54.26</td>\n",
       "      <td>16.77</td>\n",
       "      <td>47.84</td>\n",
       "      <td>26.93</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                x             y       \n",
       "             mean    std   mean    std\n",
       "dataset                               \n",
       "away        54.27  16.77  47.83  26.94\n",
       "bullseye    54.27  16.77  47.83  26.94\n",
       "circle      54.27  16.76  47.84  26.93\n",
       "dino        54.26  16.77  47.83  26.94\n",
       "dots        54.26  16.77  47.84  26.93\n",
       "h_lines     54.26  16.77  47.83  26.94\n",
       "high_lines  54.27  16.77  47.84  26.94\n",
       "slant_down  54.27  16.77  47.84  26.94\n",
       "slant_up    54.27  16.77  47.83  26.94\n",
       "star        54.27  16.77  47.84  26.93\n",
       "v_lines     54.27  16.77  47.84  26.94\n",
       "wide_lines  54.27  16.77  47.83  26.94\n",
       "x_shape     54.26  16.77  47.84  26.93"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Compute mean and standard deviation of x and y for each sub-dataset\n",
    "grouped_stats = (\n",
    "    df.groupby('dataset')[['x', 'y']]\n",
    "    .agg(['mean', 'std'])\n",
    "    .round(2)\n",
    ")\n",
    "\n",
    "grouped_stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "837a2552",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataset\n",
      "away         -0.06\n",
      "bullseye     -0.07\n",
      "circle       -0.07\n",
      "dino         -0.06\n",
      "dots         -0.06\n",
      "h_lines      -0.06\n",
      "high_lines   -0.07\n",
      "slant_down   -0.07\n",
      "slant_up     -0.07\n",
      "star         -0.06\n",
      "v_lines      -0.07\n",
      "wide_lines   -0.07\n",
      "x_shape      -0.07\n",
      "Name: corr(x,y), dtype: float64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sss\\AppData\\Local\\Temp\\ipykernel_34380\\2163207487.py:2: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.\n",
      "  correlation = df.groupby('dataset').apply(lambda g: g['x'].corr(g['y'])).round(2)\n"
     ]
    }
   ],
   "source": [
    "# Also compute the Pearson correlation between x and y per group\n",
    "correlation = df.groupby('dataset').apply(lambda g: g['x'].corr(g['y'])).round(2)\n",
    "correlation.name = 'corr(x,y)'\n",
    "print(correlation)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c40be027",
   "metadata": {},
   "source": [
    "> **Question:** Look at the table above. Are the 13 datasets statistically different from each other?  \n",
    "> Write your answer in the cell below before moving on.\n",
    "\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc4c40dd",
   "metadata": {},
   "source": [
    "<!-- ## Part 3: Now Let us Actually Look at the Data\n",
    "\n",
    "We will focus on three sub-datasets: **`dino`**, **`star`**, and **`bullseye`**. These three were chosen because they produce a dramatic visual contrast despite their identical statistics.\n",
    "\n",
    "Later, feel free to explore the remaining 10 groups. -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d4fde0b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rows in subset: 426\n"
     ]
    }
   ],
   "source": [
    "# Filter to the three focus datasets\n",
    "focus = ['dino', 'star', 'bullseye']\n",
    "df_focus = df[df['dataset'].isin(focus)].copy()\n",
    "\n",
    "print(f'Rows in subset: {len(df_focus)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86d8b1b6",
   "metadata": {},
   "source": [
    "### 3.1 — Individual scatter plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c2f4c527",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 81764 characters]",
      "text/plain": [
       "<Figure size 1500x500 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 3, figsize=(15, 5), sharey=True)\n",
    "\n",
    "colors = sns.color_palette('tab10', 3)\n",
    "\n",
    "for ax, name, color in zip(axes, focus, colors):\n",
    "    subset = df_focus[df_focus['dataset'] == name]\n",
    "    ax.scatter(subset['x'], subset['y'], color=color, alpha=0.7, s=40, edgecolors='white', linewidths=0.4)\n",
    "    ax.set_title(name, fontsize=14, fontweight='bold')\n",
    "    ax.set_xlabel('x')\n",
    "    ax.set_ylabel('y')\n",
    "\n",
    "fig.suptitle('Same statistics, completely different data', fontsize=16, fontweight='bold', y=1.02)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "538ecb6f",
   "metadata": {},
   "source": [
    "### 3.2 — Side-by-side with statistics overlay\n",
    "\n",
    "Let's add the mean and standard deviation annotations to make the point explicit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d677b3ec",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 106340 characters]",
      "text/plain": [
       "<Figure size 1500x550 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 3, figsize=(15, 5.5), sharey=True)\n",
    "\n",
    "for ax, name, color in zip(axes, focus, colors):\n",
    "    subset = df_focus[df_focus['dataset'] == name]\n",
    "    \n",
    "    ax.scatter(subset['x'], subset['y'], color=color, alpha=0.65, s=40,\n",
    "               edgecolors='white', linewidths=0.4, label='observations')\n",
    "    \n",
    "    # Mean crosshair\n",
    "    mx, my = subset['x'].mean(), subset['y'].mean()\n",
    "    ax.axvline(mx, color='black', linestyle='--', linewidth=1.0, alpha=0.6)\n",
    "    ax.axhline(my, color='black', linestyle='--', linewidth=1.0, alpha=0.6)\n",
    "    ax.scatter([mx], [my], color='black', s=80, zorder=5, label=f'mean ({mx:.1f}, {my:.1f})')\n",
    "    \n",
    "    # Stats box\n",
    "    stats_text = (\n",
    "        f\"mean x = {subset['x'].mean():.2f}\\n\"\n",
    "        f\"mean y = {subset['y'].mean():.2f}\\n\"\n",
    "        f\"sd x   = {subset['x'].std():.2f}\\n\"\n",
    "        f\"sd y   = {subset['y'].std():.2f}\\n\"\n",
    "        f\"corr   = {subset['x'].corr(subset['y']):.2f}\"\n",
    "    )\n",
    "    ax.text(0.03, 0.97, stats_text, transform=ax.transAxes,\n",
    "            fontsize=8.5, verticalalignment='top', fontfamily='monospace',\n",
    "            bbox=dict(boxstyle='round,pad=0.4', facecolor='white', alpha=0.85, edgecolor='grey'))\n",
    "    \n",
    "    ax.set_title(name, fontsize=14, fontweight='bold')\n",
    "    ax.set_xlabel('x')\n",
    "    ax.set_ylabel('y')\n",
    "\n",
    "fig.suptitle('Datasaurus Dozen — statistics are identical, shapes are not',\n",
    "             fontsize=14, fontweight='bold', y=1.01)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e295910e",
   "metadata": {},
   "source": [
    "> **❓ Question:** What would a data analyst have concluded if they had only looked at the summary statistics table?  \n",
    "> What does this tell you about when and why visualisation is necessary?\n",
    "\n",
    "*(Double-click to write your answer here)*\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86dea1fb",
   "metadata": {},
   "source": [
    "## Part 4 — Small Multiples: All 13 Datasets at Once\n",
    "\n",
    "Seaborn's `FacetGrid` makes it easy to produce a *small multiples* plot — the same chart type repeated for each group. This is a powerful pattern for comparing distributions across many categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d7eb9f5a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 181396 characters]",
      "text/plain": [
       "<Figure size 1500x900 with 13 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "g = sns.FacetGrid(df, col='dataset', col_wrap=5, height=3, aspect=1.0,\n",
    "                  sharex=False, sharey=False)\n",
    "g.map(sns.scatterplot, 'x', 'y', alpha=0.6, s=18, color='steelblue', edgecolor='white', linewidth=0.2)\n",
    "g.set_titles(col_template='{col_name}', size=10)\n",
    "g.figure.suptitle('All 13 Datasaurus Dozen datasets — identical statistics',\n",
    "                   fontsize=13, fontweight='bold', y=1.01)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "becc716d",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Some Exploration\n",
    "\n",
    "For each chart type below, run the cell and then answer the key question:\n",
    "\n",
    "> **Does this chart type reveal the structural differences between datasets, or does it hide them?**\n",
    "\n",
    "---\n",
    "\n",
    "### Histograms\n",
    "\n",
    "Plot the marginal distribution of `x` and `y` separately for each focus dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "83a2bc01",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 67056 characters]",
      "text/plain": [
       "<Figure size 1500x800 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(2, 3, figsize=(15, 8), sharey=True)\n",
    "\n",
    "for col_idx, var in enumerate(['x', 'y']):\n",
    "    for ax, name, color in zip(axes[col_idx], focus, colors):\n",
    "        subset = df_focus[df_focus['dataset'] == name]\n",
    "        sns.histplot(subset[var], ax=ax, color=color, bins=15, kde=False)\n",
    "        ax.set_title(f'{name} — {var}', fontsize=12, fontweight='bold')\n",
    "        ax.set_xlabel(var)\n",
    "\n",
    "fig.suptitle('Histograms — marginal distributions of x and y per dataset',\n",
    "             fontsize=13, fontweight='bold', y=1.01)\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exploration_md",
   "metadata": {},
   "source": [
    "> **Answer:** Partially. Histograms show the marginal distribution of one variable at a time, so they reveal that the datasets differ along each axis individually. But they lose all information about the *relationship* between x and y — you cannot see the dinosaur or the star from a histogram alone. They reveal more than summary statistics, but less than a scatterplot.\n",
    "\n",
    "---\n",
    "\n",
    "### KDE plots\n",
    "\n",
    "Overlay density curves for the three focus datasets on the same axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "3cc44f9f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 138076 characters]",
      "text/plain": [
       "<Figure size 1400x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
    "\n",
    "for ax, var in zip(axes, ['x', 'y']):\n",
    "    sns.kdeplot(data=df_focus, x=var, hue='dataset', ax=ax, fill=True, alpha=0.3, linewidth=1.5)\n",
    "    ax.set_title(f'KDE of {var} — three focus datasets', fontsize=12, fontweight='bold')\n",
    "    ax.set_xlabel(var)\n",
    "\n",
    "fig.suptitle('KDE plots — overlaid density curves per dataset',\n",
    "             fontsize=13, fontweight='bold')\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exploration_md",
   "metadata": {},
   "source": [
    "> **Answer:** Same limitation as histograms — KDE plots show the marginal density of one variable at a time. The three curves look somewhat different from each other (especially for y), but you cannot reconstruct the actual shapes from them. The structural difference between dino, star, and bullseye is heavily underrepresented.\n",
    "\n",
    "---\n",
    "\n",
    "### Pair plots\n",
    "\n",
    "Plot all pairwise combinations of variables, coloured by dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "exploration_cell",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 177940 characters]",
      "text/plain": [
       "<Figure size 811.375x700 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "g = sns.pairplot(df_focus, hue='dataset', plot_kws={'alpha': 0.5, 's': 20},\n",
    "                 diag_kind='kde', height=3.5)\n",
    "g.figure.suptitle('Pair plot — dino, star, bullseye', fontsize=13,\n",
    "                   fontweight='bold', y=1.01)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exploration_md",
   "metadata": {},
   "source": [
    "> **Answer:** Yes — the off-diagonal scatter plot (x vs y) fully reveals the structural differences, showing the dinosaur, star, and bullseye shapes clearly. The diagonal KDE plots add the marginal distributions. For a dataset with only two variables the pair plot is essentially a scatter plot with extras, but the pattern scales well to datasets with many variables.\n",
    "\n",
    "---\n",
    "\n",
    "### Box plots\n",
    "\n",
    "Summarise the distribution of `x` and `y` per dataset using box plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "exploration_cell",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\sss\\AppData\\Local\\Temp\\ipykernel_34380\\1218167074.py:4: FutureWarning: \n",
      "\n",
      "Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.\n",
      "\n",
      "  sns.boxplot(data=df_focus, x='dataset', y=var, ax=ax,\n",
      "C:\\Users\\sss\\AppData\\Local\\Temp\\ipykernel_34380\\1218167074.py:4: FutureWarning: \n",
      "\n",
      "Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `x` variable to `hue` and set `legend=False` for the same effect.\n",
      "\n",
      "  sns.boxplot(data=df_focus, x='dataset', y=var, ax=ax,\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 38168 characters]",
      "text/plain": [
       "<Figure size 1400x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(14, 5))\n",
    "\n",
    "for ax, var in zip(axes, ['x', 'y']):\n",
    "    sns.boxplot(data=df_focus, x='dataset', y=var, ax=ax,\n",
    "                palette='tab10', width=0.5, linewidth=1.2)\n",
    "    ax.set_title(f'Box plot of {var} per dataset', fontsize=12, fontweight='bold')\n",
    "    ax.set_xlabel('dataset')\n",
    "    ax.set_ylabel(var)\n",
    "\n",
    "fig.suptitle('Box plots — do they reveal the structural differences?',\n",
    "             fontsize=13, fontweight='bold')\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exploration_md",
   "metadata": {},
   "source": [
    "> **Answer:** No — and this is the most important result. The three box plots look nearly identical for both x and y: same median, same IQR, same whiskers. Box plots summarise only five statistics per group (min, Q1, median, Q3, max), so they suffer the same blindspot as the summary statistics table. The dinosaur, star, and bullseye are completely invisible. Some chart types hide structure rather than revealing it — and box plots are a prime example."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c09cd29",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Key Takeaways\n",
    "\n",
    "- Summary statistics (mean, SD, correlation) can be completely identical across datasets with totally different structure\n",
    "- Visualisation is not a finishing step — it is a **diagnostic step** that must happen early\n",
    "- Different chart types reveal different aspects: scatterplots show point-level structure, histograms show marginal distributions, box plots summarise spread but can hide shape\n",
    "- The small multiples pattern (FacetGrid) is a powerful way to compare many groups at a glance\n",
    "\n",
    "--> In **Task 2**, you will move to a real-world dataset with real problems — and discover that the \"hard work\" you just did manually can be partially automated."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
