From 06ed8f9ca9f9d2b5ead9e9b8610310cefb0c8ebd Mon Sep 17 00:00:00 2001 From: Pavel Perezhogin Date: Thu, 2 Jun 2022 19:15:59 -0400 Subject: [PATCH] Jansen link changed --- notebooks/online_metrics.ipynb | 45 +++------------------------------- 1 file changed, 3 insertions(+), 42 deletions(-) diff --git a/notebooks/online_metrics.ipynb b/notebooks/online_metrics.ipynb index b0cad0b..6e5d2e1 100644 --- a/notebooks/online_metrics.ipynb +++ b/notebooks/online_metrics.ipynb @@ -2,7 +2,6 @@ "cells": [ { "cell_type": "markdown", - "id": "09a96a8e-1b3f-4e04-b55a-5b350e42bf43", "metadata": {}, "source": [ "# Online Metrics\n", @@ -13,7 +12,6 @@ { "cell_type": "code", "execution_count": 1, - "id": "5120a32e-6e75-48af-b415-44dfbf5bbb52", "metadata": {}, "outputs": [], "source": [ @@ -28,7 +26,6 @@ }, { "cell_type": "markdown", - "id": "a657f112-f1ae-4a9f-a472-438a419205d0", "metadata": {}, "source": [ "## Load datasets\n", @@ -41,7 +38,6 @@ { "cell_type": "code", "execution_count": 2, - "id": "c50d6f55-0617-4627-b880-f052ae37d2dd", "metadata": {}, "outputs": [], "source": [ @@ -56,7 +52,6 @@ }, { "cell_type": "markdown", - "id": "a6846d73-938a-4b8e-97db-ff5ebbdd3700", "metadata": {}, "source": [ "## Define parameterizations\n", @@ -64,7 +59,7 @@ "Here we'll define three kinds of parameterizations:\n", "\n", "1. The hybrid symbolic model we learned in the paper (with the exact set of weights we fitted)\n", - "1. A backscatter parameterization based on [Jansen et al. 2015](https://doi.org/10.1016/j.ocemod.2015.05.007) and adapted by [Pavel Perezhogin](https://github.com/Pperezhogin)\n", + "1. A backscatter parameterization based on [Jansen et al. 2015](https://www.sciencedirect.com/science/article/pii/S1463500315001341) and adapted by [Pavel Perezhogin](https://github.com/Pperezhogin)\n", "1. A classic Smagorinsky parameterization (which is just the dissipation portion of the backscatter model).\n", "\n", "Note that the latter two parameterizations are [already available in `pyqg`](https://pyqg.readthedocs.io/en/latest/api.html#pyqg.parameterizations.Smagorinsky). (We import them here with a tiny bit of adaptation so we can automatically run a simulation and save intermediate snapshots.)" @@ -73,7 +68,6 @@ { "cell_type": "code", "execution_count": 3, - "id": "195b1bb2-afd0-4d5b-874a-50fb8e50b78a", "metadata": {}, "outputs": [], "source": [ @@ -125,7 +119,6 @@ }, { "cell_type": "markdown", - "id": "083d7778-ccb6-45a4-afbf-66ae932f243d", "metadata": {}, "source": [ "We'll also consider a fully convolutional neural network parameterization (see the paper for more details), but we'll load some simulations rather than running them to save time.\n", @@ -138,7 +131,6 @@ { "cell_type": "code", "execution_count": 4, - "id": "e287a522-2844-4d2d-8f4c-2631a5d64968", "metadata": {}, "outputs": [], "source": [ @@ -149,7 +141,6 @@ { "cell_type": "code", "execution_count": 6, - "id": "a7291673-5459-4a8d-9c21-c8a6fd00953b", "metadata": {}, "outputs": [ { @@ -175,7 +166,6 @@ { "cell_type": "code", "execution_count": 5, - "id": "91387819-3670-41dc-b121-fa451abc150f", "metadata": {}, "outputs": [ { @@ -203,7 +193,6 @@ { "cell_type": "code", "execution_count": 7, - "id": "97a89c69-7a08-4217-8395-afa1c022435e", "metadata": { "tags": [] }, @@ -231,7 +220,6 @@ { "cell_type": "code", "execution_count": 30, - "id": "5e5a512f-01bb-4fb9-b6de-453537c874a3", "metadata": {}, "outputs": [], "source": [ @@ -242,7 +230,6 @@ }, { "cell_type": "markdown", - "id": "10c96543-e3e3-4f68-be57-334205a58cc4", "metadata": {}, "source": [ "## Compute similarity metrics\n", @@ -267,7 +254,6 @@ { "cell_type": "code", "execution_count": 8, - "id": "4bf1227c-c72b-45bd-820c-bfb52b08d15f", "metadata": {}, "outputs": [ { @@ -305,7 +291,6 @@ }, { "cell_type": "markdown", - "id": "72a7e996-4e22-4555-8d7d-90ec66d70cb5", "metadata": {}, "source": [ "The Smagorinsky parameterization has negative similarity scores for most of our metrics. That means it's bringing simulation quantities further from high-res.\n", @@ -316,7 +301,6 @@ { "cell_type": "code", "execution_count": 9, - "id": "0be2dbf9-1df2-4bb4-910b-dbef62806364", "metadata": {}, "outputs": [ { @@ -354,7 +338,6 @@ }, { "cell_type": "markdown", - "id": "2e3a3dc6-2545-44ff-81bb-f852319ee114", "metadata": {}, "source": [ "These are quite good.\n", @@ -365,7 +348,6 @@ { "cell_type": "code", "execution_count": 10, - "id": "44ac97f6-96dc-4312-a01c-a8b309ec595f", "metadata": {}, "outputs": [ { @@ -403,7 +385,6 @@ }, { "cell_type": "markdown", - "id": "e45ad0d8-01a8-4237-9f90-b256f125333e", "metadata": {}, "source": [ "Also quite good, and somewhat hard to distinguish from the backscatter model.\n", @@ -414,7 +395,6 @@ { "cell_type": "code", "execution_count": 31, - "id": "db2a4fe0-66e8-402e-bc51-36016528c215", "metadata": {}, "outputs": [ { @@ -452,7 +432,6 @@ }, { "cell_type": "markdown", - "id": "3dbad53f-5886-425d-a6a6-b214d82089d1", "metadata": {}, "source": [ "Once again, very good, and with higher worst-case scores. It's essentially matching everything.\n", @@ -463,7 +442,6 @@ { "cell_type": "code", "execution_count": 37, - "id": "6c857a14-4797-4556-aa17-bf6590f3fe88", "metadata": {}, "outputs": [ { @@ -499,7 +477,6 @@ }, { "cell_type": "markdown", - "id": "b06d00a1-a874-4049-bfd0-7ae0160e416c", "metadata": {}, "source": [ "We can see that three of our four parameterizations are helping essentially across the board, though the FCNN is most consistent. Interestingly, neither the backscatter nor the hybrid symbolic model lessen distributional differences between the upper-layer enstrophy.\n", @@ -512,7 +489,6 @@ { "cell_type": "code", "execution_count": 12, - "id": "6144a329-3144-44df-8fa6-3a27ed65d0aa", "metadata": {}, "outputs": [], "source": [ @@ -523,7 +499,6 @@ { "cell_type": "code", "execution_count": 13, - "id": "2577ce09-b0be-4d5a-a5c5-6df445ea195d", "metadata": {}, "outputs": [], "source": [ @@ -537,7 +512,6 @@ { "cell_type": "code", "execution_count": 14, - "id": "46fb27b8-9d98-4cde-9c32-862ea5e38c37", "metadata": {}, "outputs": [ { @@ -563,7 +537,6 @@ { "cell_type": "code", "execution_count": 15, - "id": "764f8e89-baf5-46b7-ac0d-4bf4699f4717", "metadata": {}, "outputs": [ { @@ -589,7 +562,6 @@ { "cell_type": "code", "execution_count": 16, - "id": "57c60943-e524-48e3-93a3-6a33af76994b", "metadata": {}, "outputs": [ { @@ -615,7 +587,6 @@ { "cell_type": "code", "execution_count": 38, - "id": "60113988-d0bd-4589-8d6d-5b59292ffce5", "metadata": {}, "outputs": [], "source": [ @@ -625,7 +596,6 @@ { "cell_type": "code", "execution_count": 17, - "id": "1823478a-b8de-455c-a3a6-2ed78f9cd55b", "metadata": {}, "outputs": [ { @@ -663,7 +633,6 @@ }, { "cell_type": "markdown", - "id": "6600dc6d-aa29-462c-82b6-3b0a5aede365", "metadata": {}, "source": [ "The Smagorinsky parameterization still gets negative similarity scores, though they aren't any _more_ negative than before." @@ -672,7 +641,6 @@ { "cell_type": "code", "execution_count": 18, - "id": "773e3506-0bd1-4e60-a768-4d3c38c2990f", "metadata": {}, "outputs": [ { @@ -710,7 +678,6 @@ }, { "cell_type": "markdown", - "id": "cac766e9-a223-4baa-ac86-a1437a4214ea", "metadata": {}, "source": [ "The backscatter model gets marginally positive similarity scores in this case (it's not super consistent across re-runs), though it also has a number of fairly negative scores. Overall, it's not generalizing exactly the way we would hope. " @@ -719,7 +686,6 @@ { "cell_type": "code", "execution_count": 19, - "id": "407c8fbc-d533-421d-9738-0f87b544c80c", "metadata": {}, "outputs": [ { @@ -757,7 +723,6 @@ }, { "cell_type": "markdown", - "id": "6fbb9958-42cc-4c53-921a-ccce405d6907", "metadata": {}, "source": [ "The symbolic regression model has much more consistently positive scores, indicating better generalization ability!" @@ -766,7 +731,6 @@ { "cell_type": "code", "execution_count": 39, - "id": "3a5d286a-0280-4f71-a774-f70f9bbb9f8d", "metadata": {}, "outputs": [ { @@ -804,7 +768,6 @@ }, { "cell_type": "markdown", - "id": "494d9da6-d099-4eca-9a8e-077f53d2377b", "metadata": {}, "source": [ "The neural network gets very negative similarity scores, indicating that it's not succeeding in transferring. The lower-layer enstrophy in particular gets a similarity score of -34. When plotting, we'll need to manually set the $y$ limits, or else the plot will be unreadable:" @@ -813,7 +776,6 @@ { "cell_type": "code", "execution_count": 42, - "id": "d9942a09-d941-463a-8294-615a2d7f838b", "metadata": {}, "outputs": [ { @@ -850,7 +812,6 @@ }, { "cell_type": "markdown", - "id": "96979017-d1c0-4879-8467-8c85fd10efef", "metadata": {}, "source": [ "The symbolic regression model does best by most of the metrics, though the backscatter model does well on many, and interestingly the FCNN does get closest on the upper-layer enstrophy again. This is consistent with a result we see in [the FCNN notebook](./neural_networks.ipynb), where its upper-layer $R^2$s remain high when transferring to jet configuration while its lower-layer $R^2$s plummet.\n", @@ -861,7 +822,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3 (ipykernel)", + "display_name": "Python 3", "language": "python", "name": "python3" }, @@ -875,7 +836,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.5" + "version": "3.8.6" } }, "nbformat": 4,