Clarification of the scope and the text

message_ix/tests/test_tutorials.py

@@ -1,4 +1,5 @@
 import sys
+from shutil import copyfile
 
 import numpy as np
 import pytest
@@ -23,6 +24,8 @@
         [("solve-objective-value", 369297.75)],
         {},
     ),
+    (("westeros", "westeros_baseline_using_xlsx_import_part1"), [], {}),
+    (("westeros", "westeros_baseline_using_xlsx_import_part2"), [], {}),
     (("westeros", "westeros_emissions_bounds"), [], {}),
     (("westeros", "westeros_emissions_taxes"), [], {}),
     (("westeros", "westeros_firm_capacity"), [], {}),
@@ -49,6 +52,15 @@
 # Short, readable IDs for the tests
 ids = [arg[0][-1] for arg in tutorials]
 
+# List of data files required to run tutorials
+data_files = [
+    "westeroes_baseline_demand.xlsx",
+    "westeroes_baseline_technology_basic.xlsx",
+    "westeroes_baseline_technology_constraint.xlsx",
+    "westeroes_baseline_technology_economic.xlsx",
+    "westeroes_baseline_technology_historic.xlsx",
+]
+
 
 @pytest.fixture
 def nb_path(request, tutorial_path):
@@ -76,6 +88,12 @@ def test_tutorial(nb_path, cell_values, run_args, tmp_path, tmp_env):
         [str(nb_path.parent), tmp_env.get("PYTHONPATH", "")]
     )
 
+    # Copy necessary data files to tmp_path
+    print(nb_path)
+    if "westeros_baseline_using_xlsx_import_part2" in nb_path.parts[-1]:
+        for fil in data_files:
+            copyfile(nb_path.parent / fil, tmp_path / fil)
+
     # The notebook can be run without errors
     nb, errors = run_notebook(nb_path, tmp_path, tmp_env, **run_args)
     assert errors == []

tutorial/README.rst

@@ -110,6 +110,10 @@ uses it to illustrate a range of framework features.
    #. Add-on technologies: Add the possibility of co-generation for the coal
       power plant, by allowing it to produce heat via a passout-turbine
       (:tut:`westeros/westeros_addon_technologies.ipynb`).
+   #. Build the baseline scenario using data stored in xlsx files to populate sets and
+      parameters (:tut:`westeros_baseline_using_xlsx_import_part1.ipynb`).
+   #. Build the baseline scenario using data stored in mutliple xlsx files to populate
+      sets and parameters (:tut:`westeros_baseline_using_xlsx_import_part2.ipynb`).
 
 #. After the MESSAGE model has solved, use the :mod:`.message_ix.reporting`
    module to ‘report’ results, e.g. do post-processing, plotting, and other

tutorial/westeros/westeros_baseline_using_xlsx_import_part1.ipynb

@@ -0,0 +1,255 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "# Westeros Tutorial Part 4a - Building a scenario by importing data from Excel (I)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Scope of this tutorial\n",
+    "\n",
+    "This tutorial explains how the Westeros Baseline scenario can be built using data imported from Excel (.xlsx) files. The figure below compares the workflows of the original westeroes tutorial, where data was added using the [add_par()](https://docs.messageix.org/projects/ixmp/en/stable/api.html?highlight=add_set()#ixmp.Scenario.add_par) and [add_set()](https://docs.messageix.org/projects/ixmp/en/stable/api.html?highlight=add_set()#ixmp.Scenario.add_set) functions. The same scenario can also be created by importing data from an Excel file, using [ixmp.Scenario.read_excel()](https://docs.messageix.org/projects/ixmp/en/latest/api.html#ixmp.Scenario.read_excel) as shown in the figure below.\n",
+    "\n",
+    "<img src='_static/westeroes_baseline_xlsx_workflow_part1.jpg'>\n",
+    "\n",
+    "We also demonstrate how to export data to an Excel file.  In the first part of this tutorial we are load the scenario which we created in `westeros_baseline.ipynb`, and export the scenario data to an Excel file.\n",
+    "\n",
+    "In the sceond part, we import the data from the Excel file to create a new scenario.\n",
+    "\n",
+    "### Requirements\n",
+    "\n",
+    "Please add some notes for the requirements of running this tutorial\n",
+    "\n",
+    "\n",
+    "\n",
+    "### Online documentation\n",
+    "\n",
+    "The documentation of the MESSAGEix framework is available at [https://docs.messageix.org](https://docs.messageix.org)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Similar to other tutorials, we start by importing all the packages we need."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import pandas as pd\n",
+    "import ixmp\n",
+    "import message_ix\n",
+    "\n",
+    "%matplotlib inline"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# Loading Modeling platform\n",
+    "mp = ixmp.Platform()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1. Loading the Westeros baseline scenario"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "base = message_ix.Scenario(mp, model='Westeros Electrified', scenario='baseline')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2. Export the data to an Excel file\n",
+    "We export all the scenario data (sets and parameters) to a single Excel file called `westeros_baseline_data.xlsx`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data_file = 'westeros_baseline_data.xlsx'\n",
+    "base.to_excel(data_file)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You can open the created Excel file on your machine and navigate through the Excel sheets and make yourself familiar with the format that the data have been stored in the file."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 3. Create a new scenario"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# Creating a new, empty scenario\n",
+    "scenario = message_ix.Scenario(mp, model='Westeros Electrified', \n",
+    "                               scenario='baseline_xlsx', version='new')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Step 4: Importing data from Excel"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "Instead of using the `message_ix.Scenario.add_set()` and `add_par()` for adding data to a MESSAGEix parameter, we import data from an xlsx file. The arguments `add_units` is set to `True`, so that any units which have not yet been specified in the modeling platform can be defined automatically."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "scenario.read_excel(data_file, add_units=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Time to Solve the Model"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "scenario.set_as_default()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "scenario.solve()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "jupyter": {
+     "name": "solve-objective-value"
+    },
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "scenario.var('OBJ')['lvl']"
+   ]
+  }
+ ],
+ "metadata": {
+  "anaconda-cloud": {},
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}