Consistent GRIB handle usage in field

docs/source/how-tos/grib/grib_modify_metadata.ipynb

@@ -3,37 +3,49 @@
   {
    "cell_type": "markdown",
    "id": "5d21c14f-b851-4437-a1f1-1daac37a9e49",
-   "metadata": {},
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
    "source": [
     "# GRIB: modifying metadata"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "ac126208-e64e-4658-b69a-83e9b387464b",
-   "metadata": {},
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
    "source": [
     "This notebook demonstrates how to modify the metadata in GRIB fields.\n",
     "\n",
     "First we read some GRIB data containing pressure level fields."
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "0edd73c7-9358-47cd-96b4-cf1e0d1cb720",
-   "metadata": {},
-   "source": []
-  },
   {
    "cell_type": "code",
    "execution_count": 1,
    "id": "c11cc522-d388-4f39-a2d4-2f7b4c03517e",
-   "metadata": {},
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
    "outputs": [
     {
      "data": {
       "application/vnd.jupyter.widget-view+json": {
-       "model_id": "01b91013e30e499da1ed373f75523982",
+       "model_id": "6bb5a5212f7742f6bd69894ec6a167b3",
        "version_major": 2,
        "version_minor": 0
       },
@@ -262,7 +274,7 @@
     "tags": []
    },
    "source": [
-    "Only the field component metadata keys can be used in :py:meth:`~earthkit.data.core.field.Field.set` and raw metadata keys are not allowed to use. E.g. the field was created from GRIB data so it has the raw metadata key ``metadata.shortName`` but we cannot set it. If you need to change the GRIB metadata see the \"Changing raw GRIB metadata\" section below."
+    "Only the field component metadata keys can be used in :py:meth:`~earthkit.data.core.field.Field.set` and raw metadata keys are not allowed to use. E.g. since the field was created from GRIB data it has the raw (GRIB) metadata key ``metadata.shortName`` but we cannot set it. If you need to change the GRIB metadata see the \"Changing raw GRIB metadata\" section below."
    ]
   },
   {
@@ -319,7 +331,7 @@
     "tags": []
    },
    "source": [
-    "Setting keys for the time field component allows using multiple formats. By default a \"datetime\" key takes a datatime.datetime object and a \"step\" key takes a datatime.timedelta object."
+    "Setting keys for the \"time\" field component allows using multiple formats. By default a \"datetime\" key takes a datatime.datetime object and a \"step\" key takes a datatime.timedelta object."
    ]
   },
   {
@@ -591,7 +603,7 @@
     "tags": []
    },
    "source": [
-    "It is allowed to set individual field components with :py:meth:`~earthkit.data.core.field.Field.set`. The simplest way is to specify them as a dict. E.g. the following cell sets a new \"time\" component on the field."
+    "It is allowed to set whole individual field components with :py:meth:`~earthkit.data.core.field.Field.set`. The simplest way is to specify them as a dict. E.g. the following cell sets a new \"time\" component on the field."
    ]
   },
   {
@@ -793,8 +805,199 @@
   },
   {
    "cell_type": "markdown",
-   "id": "39666d1c-7db3-4e97-9cfc-55b3b8d71f7d",
+   "id": "e16dda68-bb85-4d0f-af9c-b814450a81c7",
+   "metadata": {},
+   "source": [
+    "## Modified fields and the associated GRIB message"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "id": "56528846-a9e1-43a8-a299-e33b64aabd55",
+   "metadata": {
+    "editable": true,
+    "raw_mimetype": "text/restructuredtext",
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "source": [
+    "When a field was created from GRIB data the associated GRIB message can be accessed via the field with :func:`~earthkit.data.core.field.Field.message`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "id": "e403ad78-821d-4b14-8cae-4b2afef55093",
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "b'GRIB\\x00\\x00\\x96\\x01\\x00\\x00'"
+      ]
+     },
+     "execution_count": 11,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "f.message()[:10]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d9aa4f53-01e1-46de-a0c1-ebb87970cf38",
+   "metadata": {},
+   "source": [
+    "Having modified the field metadata this GRIB message is not updated and we cannot access it any longer in the new field. The same is true for any raw GRIB metadata."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "8cddbcda-ea5c-4482-baa7-3535f94752a6",
    "metadata": {},
+   "outputs": [],
+   "source": [
+    "f1 = f.set({\"vertical.level\": 500})\n",
+    "f1.message()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "8750f04b-b106-493b-96ad-2c02c413137d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "f1.get(\"metadata.shortName\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "id": "5faa2075-b6ae-48a1-a07b-0b83055459e2",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "'Key metadata.shortName not found in field'\n"
+     ]
+    }
+   ],
+   "source": [
+    "try:\n",
+    "    f1.metadata(\"shortName\")\n",
+    "except KeyError as e:\n",
+    "    print(e)"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "id": "1c32e5c7-fc47-491b-bf2b-476512791831",
+   "metadata": {
+    "editable": true,
+    "raw_mimetype": "text/restructuredtext",
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "source": [
+    "If we want to keep a valid associated GRIB message in the modified field we need to call :func:`~earthkit.data.core.field.Field.sync`. This will create a new GRIB handle, update the relevant metadata in it and create a new field out of it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "5a2a63ec-0d95-4b3e-ba13-039f09234c81",
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['t', 500]"
+      ]
+     },
+     "execution_count": 15,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "f1 = f1.sync()\n",
+    "f1.get([\"metadata.shortName\", \"metadata.level\"])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "id": "f17ffe64-45d4-4bdc-a174-26e8a4fa28d7",
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "['t', 500]"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "f1.metadata([\"shortName\", \"level\"])"
+   ]
+  },
+  {
+   "cell_type": "raw",
+   "id": "af6311d3-74b0-4793-976e-d02a907f5113",
+   "metadata": {
+    "editable": true,
+    "raw_mimetype": "text/restructuredtext",
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
+   "source": [
+    "Alternatively, if your workflow is strictly GRIB-bound you can carry out the filed modification via the :py:class:`~earthkit.data.encoders.grib.GribEncoder` as shown in the next chapter. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "39666d1c-7db3-4e97-9cfc-55b3b8d71f7d",
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
    "source": [
     "## Changing raw GRIB metadata"
    ]
@@ -817,7 +1020,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 17,
    "id": "82e780fe-3539-4af9-ad4f-585739577dcc",
    "metadata": {
     "editable": true,
@@ -882,7 +1085,7 @@
        "0            1000            pressure               0          regular_ll  "
       ]
      },
-     "execution_count": 11,
+     "execution_count": 17,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -897,8 +1100,14 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "e16dda68-bb85-4d0f-af9c-b814450a81c7",
-   "metadata": {},
+   "id": "60b11297-7499-445d-9b71-2d59709aed25",
+   "metadata": {
+    "editable": true,
+    "slideshow": {
+     "slide_type": ""
+    },
+    "tags": []
+   },
    "outputs": [],
    "source": []
   }