Skip to content

taohan10200/WEATHER-5K

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WEATHER-5K Benchmark

Introduction

The WEATHER-5K dataset is a large-scale time series forecasting dataset containing weather data from 5,672 weather stations worldwide. It is a valuable resource for researchers and developers in the field of time-series forecasting, providing a comprehensive evaluation of various methods and models. WEATHER-5K dataset consists of a comprehensive collection of data from 5,672 weather stations worldwide, spanning a 10-year period with one-hour intervals. It includes multiple crucial weather elements (temperature, dewpint temperature, wind speed, wind rate, sea level pressure), providing a more reliable and interpretable resource for forecasting.

Leaderboard of WEATHER-5K benchmark

Until now, we have bnchmarked the following models in this repo:

  • iTransformer - iTransformer: Inverted Transformers Are Effective for Time Series Forecasting [ICLR 2024] [Code].

  • Corrformer - nterpretable weather forecasting for worldwide stations with a unified deep model [NMI 2023] [Code].

  • PatchTST - A Time Series is Worth 64 Words: Long-term Forecasting with Transformers [ICLR 2023] [Code].

  • DLinear - Are Transformers Effective for Time Series Forecasting? [AAAI 2023] [Code].

  • FEDformer - FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting [ICML 2022] [Code].

  • Pyraformer - Pyraformer: Low-complexity Pyramidal Attention for Long-range Time Series Modeling and Forecasting [ICLR 2022] [Code].

  • Autoformer - Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting [NeurIPS 2021] [Code].

  • Informer - Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting [AAAI 2021] [Code].

Benchmarking results

The results are reported at 4 different prediction lengths: 24, 72, 120, and 168, where the input length is 48.

Baselines Lead Time Temperature Dewpoint Wind Speed Wind Direction Sea Level Pressure Overall
MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE
🥇 1st Pyraformer 24 1.75 6.92 1.83 7.88 1.30 3.58 61.8 6930.2 1.90 9.72 13.7 1391.7
72 2.47 13.03 2.67 15.39 1.52 4.97 72.0 8222.4 3.76 33.67 16.5 1657.9
120 2.77 16.04 3.00 18.95 1.59 5.37 75.1 8610.7 4.43 43.91 17.4 1739.0
168 2.95 17.95 3.20 21.06 1.61 5.56 76.4 8773.5 4.77 49.97 17.8 1773.6
🥈 2nd iTransformer 24 1.82 7.49 1.93 8.80 1.32 3.77 63.2 7358.8 1.99 10.84 14.1 1478.0
72 2.60 14.46 2.84 17.5 1.52 4.96 73.2 8713.3 4.14 40.65 16.9 1758.2
120 2.97 18.36 3.24 22.16 1.59 5.42 76.4 9192.2 4.95 54.67 17.8 1858.6
168 3.18 20.64 3.48 24.89 1.64 5.67 78.0 9441.1 5.36 62.31 18.3 1910.9
🥉 3rd Informer 24 1.88 7.51 1.94 8.30 1.30 3.62 60.7 6906.9 2.01 10.56 13.6 1387.4
72 2.75 14.84 2.86 17.24 1.53 4.86 71.5 8251.4 4.24 39.24 16.4 1631.4
120 3.11 18.21 3.25 21.50 1.60 5.38 75.7 8504.5 5.15 54.31 18.3 1720.4
168 3.24 20.24 3.43 24.89 1.63 5.65 76.2 8718.4 5.26 58.42 18.1 1764.4
Autoformer 24 1.93 8.64 2.06 9.57 1.42 3.97 66.5 7710.0 2.26 12.78 15.2 1553.4
72 2.72 15.14 2.97 18.38 1.54 5.14 75.4 9111.5 4.25 42.34 17.8 1846.7
120 3.21 20.27 3.34 23.12 1.58 5.73 79.2 9143.5 4.83 48.88 18.1 1868.3
168 3.43 21.71 3.56 22.55 1.64 5.95 79.8 9435.8 5.32 61.85 18.5 1885.7
FEDformer 24 1.98 8.45 2.02 9.25 1.36 3.91 66.0 7384.1 2.13 11.43 14.7 1483.4
72 2.87 16.50 3.01 18.70 1.59 5.31 76.2 8824.8 4.15 37.60 17.6 1780.6
120 3.19 20.29 3.36 23.10 1.66 5.71 79.0 9143.3 4.81 48.86 18.4 1848.3
168 3.35 22.12 3.54 25.21 1.68 5.88 79.7 9189.2 5.01 53.39 18.7 1859.2
Dlinear 24 2.71 13.82 2.47 12.36 1.44 4.34 66.6 8234.5 3.09 21.34 15.3 1657.3
72 3.55 23.05 3.48 22.85 1.62 5.37 75.0 9250.8 4.64 45.83 17.7 1869.6
120 3.90 27.60 3.89 27.72 1.67 5.70 77.3 9510.6 5.19 56.22 18.4 1925.6
168 4.11 30.38 4.11 30.58 1.69 5.88 78.4 9630.0 5.48 61.73 18.8 1951.7
PatchTST 24 2.05 9.26 2.16 10.58 1.40 4.20 66.2 7765.8 2.19 12.54 14.8 1560.5
72 2.82 16.60 3.06 19.96 1.60 5.39 75.2 9067.8 4.28 42.46 17.4 1830.5
120 3.15 20.32 3.43 24.39 1.66 5.79 77.8 9452.6 5.09 57.29 18.2 1912.1
168 3.33 22.54 3.63 26.94 1.69 6.00 79.0 9638.1 5.51 65.3 18.6 1951.7
Corrformer 24 1.99 8.21 2.09 9.47 1.38 3.83 66.7 7832.3 2.19 12.39 14.9 1584.4
72 2.74 15.16 2.99 18.40 1.56 4.91 75.6 9111.7 4.27 42.36 17.8 1846.7
120 3.06 18.63 3.34 22.48 1.61 5.56 78.0 9477.4 5.08 57.13 18.1 1915.8
168 3.09 18.69 3.36 22.53 1.63 5.69 78.9 9636.0 5.34 61.83 18.4 1938.8
Mamba 24 1.98 8.59 2.01 9.52 1.37 4.02 66.0 7709.5 2.21 12.73 14.7 1548.9
72 2.79 16.00 2.90 18.11 1.55 5.11 75.1 8863.9 4.29 41.88 17.3 1789.0
120 3.03 18.47 3.18 21.02 1.58 5.28 76.7 8931.2 4.93 52.56 17.9 1805.7
168 3.16 19.88 3.32 22.53 1.59 5.35 77.4 8958.8 5.21 57.37 18.1 1812.8
**Compared models of this leaderboard.** ☑ means that their codes have already been included in this repo.

Usage

  1. Install Python 3.8. For convenience, execute the following command.
pip install -r requirements.txt
  1. Prepare Data. You can obtain the well pre-processed datasets from [OneDrive], Then place and unzip the downloaded data in the folder./WEATHER-5K.

  2. Train and evaluate model. We provide the experiment scripts for all benchmarks under the folder ./scripts/. You can reproduce the experiment results as the following examples:

# Global Station Weather Forecasting
bash ./scripts/weather-5k/iTransformer.sh

  1. Develop your own model.
  • Add the model file to the folder ./models. You can follow the ./models/Transformer.py.
  • Include the newly added model in the Exp_Basic.model_dict of ./exp/exp_basic.py.
  • Create the corresponding scripts under the folder ./scripts.

Acknowledgement

This library is constructed based on the Time-Series-Library. We sincerely thank the contributors for their contributions.

About

Large-scale Time-series Dataset Towards Next-Generation Global Station Weather Forecasting Benchmark

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors