{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8368ca49",
   "metadata": {},
   "source": [
    "# Chapter 2: Interactive Visualisation Lab\n",
    "\n",
    "Follow Section 2.5 after the three-hour data-engineering class. Reuse the original university ranking sample to learn Plotly Express, Graph Objects and a local Dash callback.\n",
    "\n",
    "Run cells in order with pandas, plotly and dash installed. Each code cell matches the Book and Slides. The final cells define the local Dash app; they deliberately leave the server stopped so Run All finishes. Download `topic2_dash_app.py` and run it in a terminal to open the application.\n",
    "\n",
    "Explain the sample, units, missingness and comparison behind each chart. This is descriptive university data, not evidence of a causal effect."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3c691f7",
   "metadata": {},
   "source": [
    "## 1. Prepare chart columns before drawing\n",
    "\n",
    "Reuse the original university ranking file. Parse numeric fields, keep explicit names and select a stated sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27462ce9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "viz_url = \"https://statpython.pages.dev/data/\"\n",
    "viz_raw = pd.read_csv(viz_url + \"university.csv\")\n",
    "viz_clean = viz_raw.assign(\n",
    "    rank=pd.to_numeric(viz_raw[\"world_rank\"].str.replace(\"=\", \"\", regex=False), errors=\"coerce\"),\n",
    "    international=pd.to_numeric(viz_raw[\"international\"], errors=\"coerce\"))\n",
    "viz_top = viz_clean.loc[viz_clean[\"year\"].eq(2016) & viz_clean[\"rank\"].le(100)].copy()\n",
    "print(len(viz_top), viz_top[[\"rank\", \"citations\", \"teaching\"]].isna().sum().to_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdb4c434",
   "metadata": {},
   "source": [
    "After removing the tied-rank `=` marker, the sample contains `100` universities and no missing rank, citations or teaching values. Rank bands elsewhere in the source become missing numeric ranks rather than invented exact positions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "070f7749",
   "metadata": {},
   "source": [
    "## 2. Express draws a histogram from a named column\n",
    "\n",
    "Map one numeric column to the horizontal axis and name the sample in the title."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "690c5677",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_hist = px.histogram(viz_top, x=\"citations\", nbins=20,\n",
    "                        labels={\"citations\": \"Citation score\"},\n",
    "                        title=\"Citation scores: 2016 top 100\")\n",
    "viz_hist.update_layout(yaxis_title=\"University count\")\n",
    "viz_hist.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e7dd0aa",
   "metadata": {},
   "source": [
    "Hover reveals the displayed bins. `nbins` guides automatic binning; use explicit `xbins` through Graph Objects when exact shared edges matter.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/histogram.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c09e75d",
   "metadata": {},
   "source": [
    "## 3. Colour and hover labels retain the row meaning\n",
    "\n",
    "Map country to colour and retain university identity in hover information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10ba3dcd",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_scatter = px.scatter(viz_top, x=\"teaching\", y=\"citations\", color=\"country\",\n",
    "                         hover_name=\"university_name\", hover_data=[\"rank\"],\n",
    "                         labels={\"teaching\": \"Teaching score\", \"citations\": \"Citation score\"},\n",
    "                         title=\"Teaching and citations: 2016 top 100\")\n",
    "viz_scatter.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bba3719",
   "metadata": {},
   "source": [
    "Categorical colour distinguishes countries; a numeric colour column creates a continuous scale. Hover identifies the observation behind each point.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/scatter.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4cb2e882",
   "metadata": {},
   "source": [
    "## 4. Text labels and annotations serve different purposes\n",
    "\n",
    "Label a small subset of observations. Add a separate annotation for the sample definition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23e8540c",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_labels = px.scatter(viz_top.head(10), x=\"teaching\", y=\"citations\",\n",
    "                        text=\"university_name\", title=\"Top 10: selected labels\")\n",
    "viz_labels.update_traces(textposition=\"top center\")\n",
    "viz_labels.add_annotation(x=0, y=1.14, xref=\"paper\", yref=\"paper\",\n",
    "                          text=\"2016 exact ranks 1–10\", showarrow=False)\n",
    "viz_labels.update_layout(margin={\"t\": 100})\n",
    "viz_labels.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b28c1469",
   "metadata": {},
   "source": [
    "`text` labels data rows. An annotation can explain the chart in data coordinates or normalised paper coordinates.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/labels.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fbcd098",
   "metadata": {},
   "source": [
    "## 5. Multiple y columns create comparable lines on one scale\n",
    "\n",
    "Express accepts a list of value columns. Here both scores use the same 0–100 scale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f9389f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_lines = px.line(viz_top.sort_values(\"rank\"), x=\"rank\",\n",
    "                    y=[\"teaching\", \"research\"],\n",
    "                    labels={\"value\": \"Score\", \"variable\": \"Measure\", \"rank\": \"World rank\"},\n",
    "                    title=\"Teaching and research by 2016 rank\")\n",
    "viz_lines.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd5013fc",
   "metadata": {},
   "source": [
    "Sort the horizontal coordinate before connecting points. This is a rank profile, not a time trend.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/lines.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2b76518",
   "metadata": {},
   "source": [
    "## 6. A third coordinate adds a variable and a viewing problem\n",
    "\n",
    "Rotate the 3D view to inspect teaching, research and citations together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9010930a",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_3d = px.scatter_3d(viz_top, x=\"teaching\", y=\"research\", z=\"citations\",\n",
    "                       color=\"country\", hover_name=\"university_name\",\n",
    "                       title=\"Three scores: 2016 top 100\")\n",
    "viz_3d.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "859af61b",
   "metadata": {},
   "source": [
    "Rotation changes apparent distances and overlap. Use a 2D comparison or facets when a reader must compare values precisely.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/scatter3d.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f84c4b51",
   "metadata": {},
   "source": [
    "## 7. Facets repeat the same chart for comparable groups\n",
    "\n",
    "Keep the three countries with the most observations in this sample and share axis ranges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "710b6103",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_countries = viz_top[\"country\"].value_counts().head(3).index\n",
    "viz_faceted_data = viz_top.loc[viz_top[\"country\"].isin(viz_countries)]\n",
    "viz_facets = px.scatter(viz_faceted_data, x=\"teaching\", y=\"citations\",\n",
    "                        facet_col=\"country\", hover_name=\"university_name\",\n",
    "                        range_x=[0, 100], range_y=[0, 100],\n",
    "                        title=\"Same scales across selected countries\")\n",
    "viz_facets.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba3b807d",
   "metadata": {},
   "source": [
    "Faceting partitions rows into panels. Shared scales support comparison; different group sizes remain part of the interpretation.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/facets.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b8aecd2",
   "metadata": {},
   "source": [
    "## 8. Animation needs stable identities and stable axes\n",
    "\n",
    "Compare the top 20 in each year. University name identifies the same institution across frames."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e51aafd",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_animated_data = viz_clean.loc[viz_clean[\"rank\"].le(20)].sort_values(\n",
    "    [\"year\", \"university_name\"])\n",
    "viz_animation = px.scatter(viz_animated_data, x=\"teaching\", y=\"citations\",\n",
    "                           color=\"country\", hover_name=\"university_name\",\n",
    "                           animation_frame=\"year\", animation_group=\"university_name\",\n",
    "                           range_x=[0, 100], range_y=[0, 100],\n",
    "                           title=\"Top 20 each year; membership can change\")\n",
    "viz_animation.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7816c15f",
   "metadata": {},
   "source": [
    "The six frames cover 2011–2016. An institution can enter or leave the selected top 20; a disappearing point is not automatically missing source data.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/animation.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42771c70",
   "metadata": {},
   "source": [
    "## 9. Graph Objects makes the trace and layout explicit\n",
    "\n",
    "Build one trace and one layout, then combine them in a Figure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3d1aacd",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_trace = go.Scatter(x=viz_top[\"teaching\"], y=viz_top[\"citations\"],\n",
    "                       mode=\"markers\", text=viz_top[\"university_name\"],\n",
    "                       name=\"2016 top 100\")\n",
    "viz_layout = go.Layout(title=\"Explicit trace and layout\",\n",
    "                       xaxis_title=\"Teaching score\", yaxis_title=\"Citation score\")\n",
    "viz_go = go.Figure(data=[viz_trace], layout=viz_layout)\n",
    "viz_go.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3fe7a17",
   "metadata": {},
   "source": [
    "The trace supplies data and its drawing mode. The layout supplies the shared chart presentation.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/graph-objects.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d017af00",
   "metadata": {},
   "source": [
    "## 10. A figure can also be described as a dictionary\n",
    "\n",
    "The same structure uses data and layout keys. Use documented properties, then let `go.Figure` validate them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab661068",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_spec = {\"data\": [{\"type\": \"scatter\", \"mode\": \"markers\",\n",
    "                      \"x\": viz_top[\"teaching\"], \"y\": viz_top[\"citations\"]}],\n",
    "            \"layout\": {\"title\": {\"text\": \"Dictionary figure\"},\n",
    "                       \"xaxis\": {\"title\": {\"text\": \"Teaching score\"}},\n",
    "                       \"yaxis\": {\"title\": {\"text\": \"Citation score\"}}}}\n",
    "viz_dictionary = go.Figure(viz_spec)\n",
    "print(len(viz_dictionary.data), viz_dictionary.layout.title.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "110d7b31",
   "metadata": {},
   "source": [
    "One trace survives and the title is `Dictionary figure`. Dictionaries are useful for serialisation; they follow the same figure schema."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "460513c3",
   "metadata": {},
   "source": [
    "## 11. Add a trace and update the existing figure\n",
    "\n",
    "Construct a rank profile, add the second score and refine titles without rebuilding the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "30b2e12a",
   "metadata": {},
   "outputs": [],
   "source": [
    "viz_ordered = viz_top.sort_values(\"rank\")\n",
    "viz_updated = go.Figure()\n",
    "viz_updated.add_trace(go.Scatter(x=viz_ordered[\"rank\"], y=viz_ordered[\"teaching\"],\n",
    "                                 mode=\"lines\", name=\"Teaching\"))\n",
    "viz_updated.add_trace(go.Scatter(x=viz_ordered[\"rank\"], y=viz_ordered[\"research\"],\n",
    "                                 mode=\"lines\", name=\"Research\"))\n",
    "viz_updated.update_layout(title=\"Two score profiles\", xaxis_title=\"World rank\",\n",
    "                          yaxis_title=\"Score\", legend_title=\"Measure\")\n",
    "viz_updated.update_traces(line={\"width\": 2})\n",
    "viz_updated.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0b235de",
   "metadata": {},
   "source": [
    "`add_trace` changes the plotted data series. `update_layout` changes shared presentation. `update_traces` changes trace properties.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/updated.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87c57858",
   "metadata": {},
   "source": [
    "## 12. make_subplots assigns traces to named grid positions\n",
    "\n",
    "Place the teaching and research profiles in separate panels with a shared score range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0fcdf59a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from plotly.subplots import make_subplots\n",
    "viz_subplots = make_subplots(rows=1, cols=2,\n",
    "                             subplot_titles=[\"Teaching\", \"Research\"])\n",
    "viz_subplots.add_trace(go.Scatter(x=viz_ordered[\"rank\"], y=viz_ordered[\"teaching\"],\n",
    "                                  mode=\"lines\", name=\"Teaching\"), row=1, col=1)\n",
    "viz_subplots.add_trace(go.Scatter(x=viz_ordered[\"rank\"], y=viz_ordered[\"research\"],\n",
    "                                  mode=\"lines\", name=\"Research\"), row=1, col=2)\n",
    "viz_subplots.update_xaxes(title_text=\"World rank\")\n",
    "viz_subplots.update_yaxes(title_text=\"Score\", range=[0, 100])\n",
    "viz_subplots.update_layout(title=\"Separate panels, comparable scales\", showlegend=False)\n",
    "viz_subplots.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "978707db",
   "metadata": {},
   "source": [
    "Plotly subplot positions start at row 1, column 1. Matplotlib's Axes array uses Python's zero-based indexing.\n",
    "\n",
    "[Open the interactive chart](https://statpython.pages.dev/data/topic2-visualisation/subplots.html){target=\"_blank\"}."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd78a737",
   "metadata": {},
   "source": [
    "## 13. Dash composes a webpage around a figure\n",
    "\n",
    "The guided extension uses a dropdown, a graph and one callback. Run it locally after downloading the complete script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a6886fe",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dash import Dash, dcc, html, Input, Output\n",
    "app = Dash(__name__)\n",
    "app.layout = html.Div([\n",
    "    html.H2(\"University scores: 2016 top 100\"),\n",
    "    dcc.Dropdown(id=\"country\", options=[\"All\"] + sorted(viz_top[\"country\"].unique()),\n",
    "                 value=\"All\", clearable=False),\n",
    "    dcc.Graph(id=\"scores\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ab858a8",
   "metadata": {},
   "source": [
    "This defines the component tree. A figure layout formats a chart; `app.layout` determines which controls and charts appear on the page."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b620aa65",
   "metadata": {},
   "source": [
    "## 14. A callback maps the selected country to a new figure\n",
    "\n",
    "The function filters rows, draws the same two score columns and returns a figure. It leaves the source table unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d0aff6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "@app.callback(Output(\"scores\", \"figure\"), Input(\"country\", \"value\"))\n",
    "def update_scores(country):\n",
    "    country_mask = viz_top[\"country\"].eq(country) | (country == \"All\")\n",
    "    selected = viz_top.loc[country_mask]\n",
    "    return px.scatter(selected, x=\"teaching\", y=\"citations\",\n",
    "                      color=\"country\", hover_name=\"university_name\",\n",
    "                      range_x=[0, 100], range_y=[0, 100],\n",
    "                      title=f\"2016 top 100 · {country} · n={len(selected)}\")\n",
    "\n",
    "# In the downloaded local script, start the server with:\n",
    "# app.run(debug=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a4863b5",
   "metadata": {},
   "source": [
    "Download [the complete Dash app](https://statpython.pages.dev/data/topic2_dash_app.py), then run `python topic2_dash_app.py` locally. The script uses `app.run(debug=True)`. Stop it with Ctrl+C."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad3fef66",
   "metadata": {},
   "source": [
    "## Practice: make your chart auditable\n",
    "\n",
    "1. Select a different year and report how many complete score pairs remain.\n",
    "2. Compare two countries using shared scales and visible sample counts.\n",
    "3. Replace one Express chart with Graph Objects, preserving its inputs.\n",
    "4. Add a selector to the local app and explain which rows it includes.\n",
    "\n",
    "The data-engineering companion remains `Chapters1_2_DataEngineering.ipynb`. Use that notebook for availability-aware features, walk-forward training and portfolio reconciliation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}