Skip to content

Commit

Permalink
readmi
Browse files Browse the repository at this point in the history
  • Loading branch information
@ChrisLisbon
ChrisLisbon committed Jan 29, 2025
1 parent 4a67aa4 commit e4ab2c2
Show file tree
Hide file tree
Showing 62 changed files with 9,803 additions and 2 deletions.
60 changes: 58 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,58 @@
# sea_ice_transformers
This repository contains code for the research of transformer effectiveness for spatio-temporal forecasting task in comparison with 3d and 2d CNN models. The experiments was set up for sea ice concentration long-term prediction in Arctic seas.
# Understanding the Limitations of Deep Transformer Models for Sea Ice Forecasting - ICCS 2025

## Content

This repo contains code and results of long-term sea ice concentration forecasting of four models: [2D CNN](cnn_forecaster_2d), [3D CNN](cnn_forecaster_3d),
[TimeSformer](timesformer) and [SwinLSTM](swinlstm).

Code and results for toy example with loop video also presented for [all models](toy_example).

Supplementary materials as PDF can be loaded [as file](Supplementary%20materials.pdf).


## Sea ice concentration data

We used OSI SAF Global Sea Ice Concentration product as source data. The spatial resolution of the images is reduced to 14 km.
Five Arctic seas were selected as test areas, its spatial position presented on image:

![arctic_map](data/arctic_map.png)

## Sea ice - Results

Table with main results of 52 weeks ahead forecast with implemented models and its comparison with SOTA
numerical (SEAS5) and deep learning (IceNet) solutions for sea ice prediction problem:

| Metric | MAE<br/>SEAS5 | MAE<br/>2D CNN | MAE<br/>3D CNN | MAE<br/>TimeS former | SSIM<br/>SEAS5 | SSIM<br/>2D CNN | SSIM<br/>3D CNN | SSIM<br/>TimeS former | Accuracy<br/>Ice Net | Accuracy<br/>2D CNN | Accuracy<br/>3D CNN | Accuracy<br/>TimeS former |
|-----------------------|-----------|----------------|---------|--------------|----------------|---------|---------|--------------|----------------------|---------|---------|--------------|
| **Kara Sea** | 0.093 | **0.076** | 0.076 | 0.109 | 0.653 | **0.683** | 0.663 | 0.581 | 0.918 | **0.945** | 0.943 | 0.929 |
| **Barents Sea** | 0.073 | 0.063 | **0.060** | 0.129 | 0.634 | **0.684** | 0.672 | 0.489 | 0.906 | 0.922 | **0.944** | 0.916 |
| **Laptev Sea** | 0.101 | **0.068** | 0.072 | 0.146 | 0.703 | **0.722** | 0.706 | 0.608 | 0.967 | **0.982** | 0.980 | 0.966 |
| **East-Siberian Sea** | 0.098 | 0.074 | **0.069** | 0.177 | **0.723** | 0.718 | 0.714 | 0.685 | 0.980 | **0.990** | 0.990 | 0.988 |
| **Chukchi Sea** | **0.067** | 0.075 | 0.073 | 0.147 | **0.780** | 0.713 | 0.719 | 0.588 | 0.974 | 0.979 | **0.981** | 0.962 |


Visualization of one timestep of prediction for Kara sea presented on image:

![anime](data/kara_trans_error.png)

Convergence plots for models, extended tables with metrics by quarters and SwinLSTM prediction results are
presented in [file](Supplementary%20materials.pdf).

## Loop video prediction - Results

As a synthetic data gif-file with loop video was used. Images was set to gray scale and resized to 45x45 pixels.

![anime](data/anime_10f.gif)
![anime](data/anime_10f_fullsize.gif)


To understand convergence process of 2D CNN and TimeSformer and get reason of TimeSformer's quality lack inference
for each epoch during optimization was produced.

2D CNN convergence process:
![anime](data/2d_cnn_anime.gif)

TimeSformer convergence process:
![anime](data/timesformer_anime_convergence.gif)

Convergence plots and metrics estimation are in [supplementary](Supplementary%20materials.pdf) and in the paper.
Binary file added Supplementary materials.pdf
View file
Binary file not shown.
24 changes: 24 additions & 0 deletions cnn_forecaster_2d/data_loader.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import os
from datetime import datetime

import numpy as np


def get_timespatial_series(sea_name, start_date, stop_date):
"""
Function for loading spatiotemporal data for sea
"""
datamodule_path = '/path_to_data/'
files_path = f'{datamodule_path}/{sea_name}'
timespatial_series = []
dates_series = []
for file in os.listdir(files_path):
date = datetime.strptime(file, f'osi_%Y%m%d.npy')
if start_date <= date.strftime('%Y%m%d') < stop_date:
array = np.load(f'{files_path}/{file}')
timespatial_series.append(array)
dates_series.append(date)
else:
break
timespatial_series = np.array(timespatial_series)
return timespatial_series, dates_series
23 changes: 23 additions & 0 deletions cnn_forecaster_2d/metrics_ts_averaged.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import pandas as pd

# This script provide averaging of metrics by quarters

l1_df = pd.DataFrame()
ssim_df = pd.DataFrame()
accuracy_df = pd.DataFrame()

for sea_name in ['kara', 'barents', 'laptev', 'eastsib', 'chukchi']:
sea_df = pd.read_csv(f'results/{sea_name}_metrics(20200101-20230101).csv')
l1_df[sea_name] = sea_df['l1']
ssim_df[sea_name] = sea_df['ssim']
accuracy_df[sea_name] = sea_df['accuracy']
l1_df['dates'] = pd.to_datetime(sea_df['dates'])
ssim_df['dates'] = pd.to_datetime(sea_df['dates'])
accuracy_df['dates'] = pd.to_datetime(sea_df['dates'])

l1_df = l1_df.resample('Q', on='dates').mean()
ssim_df = ssim_df.resample('Q', on='dates').mean()
accuracy_df = accuracy_df.resample('Q', on='dates').mean()

accuracy_df

209 changes: 209 additions & 0 deletions cnn_forecaster_2d/results/barents_metrics(20200101-20230101).csv
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
dates,l1,ssim,accuracy
2020-01-01,0.0451,0.6941,0.9755
2020-01-08,0.0448,0.7169,0.9663
2020-01-15,0.0576,0.6885,0.9545
2020-01-22,0.0551,0.685,0.9556
2020-01-29,0.0804,0.6773,0.9445
2020-02-05,0.0726,0.6568,0.9462
2020-02-12,0.0559,0.6693,0.9636
2020-02-19,0.0708,0.6223,0.9412
2020-02-26,0.0873,0.6103,0.9354
2020-03-04,0.082,0.6415,0.9383
2020-03-11,0.0811,0.6458,0.9311
2020-03-18,0.0833,0.6289,0.931
2020-03-25,0.0828,0.6378,0.9304
2020-04-01,0.0941,0.6177,0.9209
2020-04-08,0.0924,0.6348,0.9293
2020-04-15,0.0864,0.6506,0.9338
2020-04-22,0.0848,0.6256,0.9318
2020-04-29,0.0891,0.614,0.9272
2020-05-06,0.0825,0.6233,0.9415
2020-05-13,0.089,0.6282,0.9248
2020-05-20,0.1043,0.5925,0.898
2020-05-27,0.0963,0.5964,0.8908
2020-06-03,0.1015,0.6043,0.8744
2020-06-10,0.0982,0.602,0.8646
2020-06-17,0.1027,0.5644,0.8444
2020-06-24,0.1025,0.5731,0.8422
2020-07-01,0.0926,0.5617,0.866
2020-07-08,0.0668,0.6138,0.8968
2020-07-15,0.0499,0.6495,0.9242
2020-07-22,0.0481,0.6137,0.9249
2020-07-29,0.0381,0.6672,0.9459
2020-08-05,0.04,0.6673,0.9545
2020-08-12,0.0369,0.6782,0.9517
2020-08-19,0.04,0.6984,0.9525
2020-08-26,0.0415,0.6799,0.9408
2020-09-02,0.0505,0.689,0.9283
2020-09-09,0.0514,0.6354,0.9265
2020-09-16,0.046,0.6925,0.929
2020-09-23,0.048,0.6966,0.9255
2020-09-30,0.0394,0.7065,0.9384
2020-10-07,0.041,0.6451,0.9376
2020-10-14,0.0427,0.6733,0.9358
2020-10-21,0.0488,0.6507,0.9256
2020-10-28,0.0535,0.6566,0.9213
2020-11-04,0.0623,0.6308,0.9146
2020-11-11,0.0744,0.5983,0.9073
2020-11-18,0.0901,0.5635,0.8773
2020-11-25,0.0992,0.5374,0.8671
2020-12-02,0.1154,0.5109,0.8488
2020-12-09,0.1054,0.5289,0.8812
2020-12-16,0.0895,0.5786,0.9138
2020-12-23,0.0754,0.576,0.9361
2021-01-01,0.0597,0.6995,0.9387
2021-01-08,0.0623,0.6929,0.9344
2021-01-15,0.0667,0.6701,0.9323
2021-01-22,0.0765,0.6608,0.9421
2021-01-29,0.0842,0.6674,0.9382
2021-02-05,0.095,0.6521,0.928
2021-02-12,0.1144,0.6449,0.9281
2021-02-19,0.1046,0.6572,0.9275
2021-02-26,0.0998,0.6184,0.9175
2021-03-05,0.0983,0.6435,0.9285
2021-03-12,0.0911,0.6632,0.9416
2021-03-19,0.1039,0.6515,0.9245
2021-03-26,0.0995,0.6332,0.9218
2021-04-02,0.1007,0.6302,0.9103
2021-04-09,0.1083,0.5982,0.8811
2021-04-16,0.0799,0.6566,0.923
2021-04-23,0.0768,0.6627,0.9215
2021-04-30,0.0807,0.6591,0.916
2021-05-07,0.0837,0.6263,0.9092
2021-05-14,0.0842,0.6273,0.9116
2021-05-21,0.0854,0.6473,0.913
2021-05-28,0.0929,0.6117,0.9
2021-06-04,0.0923,0.5907,0.8989
2021-06-11,0.0953,0.593,0.8715
2021-06-18,0.0793,0.6129,0.8873
2021-06-25,0.0791,0.5976,0.89
2021-07-02,0.0641,0.6262,0.9048
2021-07-09,0.0567,0.6849,0.9266
2021-07-16,0.0585,0.6578,0.9251
2021-07-23,0.0432,0.6971,0.9418
2021-07-30,0.0448,0.6986,0.9415
2021-08-06,0.0339,0.7053,0.9488
2021-08-13,0.0275,0.7352,0.9545
2021-08-20,0.0258,0.7509,0.964
2021-08-27,0.0237,0.7677,0.9632
2021-09-03,0.0267,0.7432,0.969
2021-09-10,0.0217,0.7808,0.9743
2021-09-17,0.0212,0.7712,0.979
2021-09-24,0.0239,0.7539,0.9715
2021-10-01,0.0247,0.7409,0.969
2021-10-08,0.0278,0.7257,0.9615
2021-10-15,0.0321,0.7022,0.9648
2021-10-22,0.0359,0.7041,0.9488
2021-10-29,0.06,0.6965,0.9373
2021-11-05,0.0473,0.6745,0.9463
2021-11-12,0.0717,0.6434,0.921
2021-11-19,0.0989,0.6443,0.9206
2021-11-26,0.1177,0.6355,0.9125
2021-12-03,0.1218,0.6345,0.9069
2021-12-10,0.1079,0.6499,0.9132
2021-12-17,0.109,0.6357,0.9175
2021-12-24,0.11,0.6247,0.921
2022-01-01,0.0496,0.7869,0.9678
2022-01-08,0.0549,0.7775,0.9659
2022-01-15,0.0462,0.7923,0.969
2022-01-22,0.0491,0.7903,0.9741
2022-01-29,0.0746,0.7003,0.9527
2022-02-05,0.0536,0.7841,0.968
2022-02-12,0.0909,0.7009,0.927
2022-02-19,0.0693,0.7487,0.9615
2022-02-26,0.0696,0.7388,0.9508
2022-03-05,0.0631,0.7492,0.9598
2022-03-12,0.086,0.6889,0.9381
2022-03-19,0.1074,0.6347,0.9129
2022-03-26,0.0745,0.7193,0.9462
2022-04-02,0.0637,0.7193,0.9545
2022-04-09,0.0597,0.7454,0.964
2022-04-16,0.0568,0.7199,0.9581
2022-04-23,0.0531,0.7431,0.965
2022-04-30,0.0689,0.6874,0.9431
2022-05-07,0.0701,0.7138,0.9414
2022-05-14,0.0696,0.6976,0.9384
2022-05-21,0.076,0.6676,0.9257
2022-05-28,0.0585,0.707,0.9513
2022-06-04,0.0581,0.6854,0.9545
2022-06-11,0.06,0.6992,0.9388
2022-06-18,0.0587,0.6835,0.9165
2022-06-25,0.0657,0.6432,0.8849
2022-07-02,0.0528,0.6678,0.9083
2022-07-09,0.0442,0.6746,0.9329
2022-07-16,0.04,0.6896,0.9507
2022-07-23,0.0438,0.7114,0.9451
2022-07-30,0.0418,0.7047,0.9512
2022-08-06,0.0374,0.692,0.9552
2022-08-13,0.0363,0.7163,0.9465
2022-08-20,0.0354,0.7077,0.9525
2022-08-27,0.0304,0.7485,0.9662
2022-09-03,0.0244,0.7464,0.9725
2022-09-10,0.0237,0.7266,0.9785
2022-09-17,0.028,0.7268,0.9631
2022-09-24,0.0337,0.7092,0.9608
2022-10-01,0.038,0.7311,0.9561
2022-10-08,0.0469,0.6961,0.9321
2022-10-15,0.0491,0.6894,0.918
2022-10-22,0.0561,0.7026,0.9102
2022-10-29,0.0711,0.6718,0.8902
2022-11-05,0.0818,0.649,0.8938
2022-11-12,0.0765,0.6445,0.897
2022-11-19,0.0788,0.6805,0.9033
2022-11-26,0.0843,0.6437,0.8995
2022-12-03,0.1011,0.611,0.8818
2022-12-10,0.0855,0.6446,0.9111
2022-12-17,0.0868,0.6577,0.9138
2022-12-24,0.0762,0.6508,0.9302
2023-01-01,0.0525,0.7289,0.9425
2023-01-08,0.0583,0.7333,0.953
2023-01-15,0.0632,0.7018,0.9422
2023-01-22,0.0696,0.6601,0.9414
2023-01-29,0.0561,0.7088,0.9493
2023-02-05,0.0547,0.7368,0.9682
2023-02-12,0.0924,0.6504,0.9106
2023-02-19,0.0806,0.67,0.9301
2023-02-26,0.0863,0.6635,0.9189
2023-03-05,0.0538,0.7484,0.9545
2023-03-12,0.0612,0.7387,0.9429
2023-03-19,0.0572,0.7528,0.954
2023-03-26,0.0622,0.7353,0.9545
2023-04-02,0.0668,0.7311,0.9527
2023-04-09,0.0676,0.7273,0.9466
2023-04-16,0.0663,0.7133,0.9472
2023-04-23,0.0685,0.683,0.9346
2023-04-30,0.0632,0.6754,0.9453
2023-05-07,0.0722,0.7072,0.9482
2023-05-14,0.0693,0.7109,0.9504
2023-05-21,0.0687,0.6862,0.9533
2023-05-28,0.0575,0.7251,0.9619
2023-06-04,0.0613,0.7051,0.9562
2023-06-11,0.0584,0.6882,0.9487
2023-06-18,0.057,0.7014,0.9441
2023-06-25,0.0575,0.6954,0.9304
2023-07-02,0.0513,0.6872,0.9294
2023-07-09,0.0424,0.7048,0.9556
2023-07-16,0.0453,0.7014,0.9415
2023-07-23,0.0435,0.6943,0.9458
2023-07-30,0.0426,0.7057,0.9461
2023-08-06,0.0322,0.7128,0.9566
2023-08-13,0.0348,0.7231,0.9608
2023-08-20,0.0306,0.725,0.9617
2023-08-27,0.0298,0.7224,0.9627
2023-09-03,0.025,0.7291,0.9778
2023-09-10,0.0251,0.7287,0.9762
2023-09-17,0.0248,0.7182,0.9701
2023-09-24,0.026,0.7366,0.9745
2023-10-01,0.0302,0.7439,0.9745
2023-10-08,0.0296,0.737,0.971
2023-10-15,0.0314,0.7308,0.9593
2023-10-22,0.0292,0.7638,0.9614
2023-10-29,0.0493,0.7346,0.9433
2023-11-05,0.0612,0.6921,0.9362
2023-11-12,0.0689,0.7254,0.9293
2023-11-19,0.1,0.6616,0.9114
2023-11-26,0.0652,0.7142,0.9402
2023-12-03,0.074,0.6992,0.9439
2023-12-10,0.0806,0.6808,0.9366
2023-12-17,0.069,0.6959,0.9361
2023-12-24,0.0771,0.6906,0.9504
Loading

0 comments on commit e4ab2c2

Please sign in to comment.