Picture: "Comparison between in-situ wind speed, ECMWF wind speed reanalyses and the prediction of wind speed made with three different methods: Multi-modal 4DVarNet, Single-modal 4DVarNet and CatBoost", illustrating the presentation "Monitoring In-Situ Wind Speed Using Underwater Acoustics and Deep Learning"

Organizers

IMT Atlantique (France) and RIKEN (Japan)

programme

Monday 26^th September 2022, afternoon

2pm – introduction

• RIKEN’s activities on fusing AI and data assimilation in numerical weather prediction
  Takemasa Miyoshi⁽³¹⁾, Arata Amemiya⁽³¹⁾, Maha Mdini⁽³¹⁾, and Jianyu Liang^{(31)
  (31)} RIKEN, Japan
  [Read the abstract]

• Rainfall estimation for extreme events from SAR observations using deep learning models
  Aurélien Colin^(12)(17), Ronan Fablet^(12)(14), Pierre Tandeo⁽¹²⁾, Romain Husson⁽¹⁷⁾, and Charles Peureux^{(17)
  (12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
  ⁽¹⁴⁾ INRIA team Odyssey, France
  ⁽¹⁷⁾ Collecte Localisation Satellites, France
  [Read the abstract]

• Algorithm development for the 3D precipitation nowcasting with deep learning
  Shigenori Otsuka^(26)(27)(28), and Takemasa Miyoshi^{(26)(27)(28)(29)(30)
  (26)} RIKEN Center for Computational Science, Japan
  ⁽²⁷⁾ RIKEN Cluster for Pioneering Research, Japan
  ⁽²⁸⁾ RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program, Japan
  ⁽²⁹⁾ University of Maryland, USA
  ⁽³⁰⁾ Japan Agency for Marine-Earth Science and Technology, Japan
  [Read the abstract]

• Ultra short-term wind forecasting for offshore wind operations and maintenance
  Robin Marcille^(12)(23), Pierre Tandeo⁽¹²⁾, Maxime Thiebaut⁽²³⁾, Juan Emmanuel Johnson⁽²⁴⁾, Florence Lafon⁽²³⁾, Pierre Pinson⁽²⁵⁾, and Ronan Fablet^{(12)(14)
  (12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
  ⁽¹⁴⁾ INRIA team Odyssey, France
  ⁽²³⁾ France Energies Marines, France
  ⁽²⁴⁾ Institut des Géosciences de l'Environnement, France
  ⁽²⁵⁾ Technical University of Denmark, Department of Technology, Management and Economics, Denmark
  [Read the abstract]

4pm-4.20pm – coffee break

• Data-driven data assimilation to better characterize climate projections: a case study with an idealized chaotic AMOC model
  Pierre Le Bras^(12)(18), Pierre Ailliot⁽¹⁹⁾, Noémie Le Carrer^(12)(18), Juan Ruiz^(20)(21), Florian Sévellec^(18)(22), and Pierre Tandeo^{(12)
  (12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
  ⁽¹⁸⁾ Laboratoire d’Océanographie Physique et Spatiale, IUEM, Univ. Brest, CNRS, IRD, Ifremer (UMR 6523)
  ⁽¹⁹⁾ Laboratoire de Mathématiques de Bretagne Atlantique, Univ Brest (UMR CNRS 6205), France
  ⁽²⁰⁾ Centro de Investigaciones del Mar y la Atmósfera, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, CONICET-UBA, Argentina
  ⁽²¹⁾ UMI-IFAECI (CNRS-CONICET-UBA), Argentina
  ⁽²²⁾ Ocean and Earth Science, University of Southampton, United Kingdom
  [Read the abstract]

• A Machine Learning Approach to the Observation Operator for Satellite Radiance Data Assimilation
  Jianyu Liang⁽²⁶⁾, Koji Terasaki⁽²⁶⁾, and Takemasa Miyoshi^{(26)
  (26)} RIKEN Center for Computational Science, Japan
  [Read the abstract]

• Fishing gears identification from artificial intelligence applied to geospatial data
  Francois Danhiez⁽¹⁵⁾, Mathieu Woillez⁽¹⁶⁾, and Julien Rodriguez^{(16)
  (15)} Capgemini Engineering Brest, France
  ⁽¹⁶⁾ Ifremer, France
  [Read the abstract]

• Monitoring In-Situ Wind Speed Using Underwater Acoustics and Deep Learning
  Matteo Zambra^(12)(14), Dorian Cazau⁽¹³⁾, Nicolas Farrugia⁽¹²⁾, and Ronan Fablet^{(12)(14)
  (12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
  ⁽¹⁴⁾ INRIA team Odyssey, France
  ⁽¹³⁾ ENSTA Bretagne, Lab-STICC, UMR CNRS 6285, France
  [Read the abstract]

6pm - End of the session

Abstracts

RIKEN’s activities on fusing AI and data assimilation in numerical weather prediction
Takemasa Miyoshi⁽³¹⁾, Arata Amemiya⁽³¹⁾, Maha Mdini⁽³¹⁾, and Jianyu Liang^(31)
(31) RIKEN, Japan

At RIKEN, we have been exploring a fusion of big data and big computation, and now with AI and machine learning (ML). The new Japan’s flagship supercomputer “Fugaku” is designed to be efficient for both double-precision big simulations and reduced-precision machine learning applications, aiming to play a pivotal role in creating super-smart “Society 5.0.” Our group in RIKEN has been pushing the limits of numerical weather prediction (NWP) through two orders of magnitude bigger computations using the previous Japan’s flagship “K computer”. With the new Fugaku, we achieved real-time 30-second-refresh predictions of sudden downpours up to 30 minutes in advance during Tokyo Olympics and Paralympics by fully exploiting big data from a novel Phased Array Weather Radar. Moreover, we have been exploring ideas for improving the predicting capabilities by fusing Big Data Assimilation and AI. The data produced by NWP models become bigger and moving around the data to other computers for ML may not be feasible. Having a next-generation computer like Fugaku, good for both big NWP computation and ML, may bring a breakthrough toward creating a new methodology of fusing data-driven (inductive) and process-driven (deductive) approaches in meteorology. This presentation will introduce the most recent results from data assimilation and NWP experiments, followed by perspectives toward future developments and challenges of DA-AI fusion.

Rainfall estimation for extreme events from SAR observations using deep learning models
Aurélien Colin^(12)(17), Ronan Fablet^(12)(14), Pierre Tandeo⁽¹²⁾, Romain Husson⁽¹⁷⁾, and Charles Peureux^(17)
(12) IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
⁽¹⁴⁾ INRIA team Odyssey, France
⁽¹⁷⁾ Collecte Localisation Satellites, France

Rainfall estimation in coastal areas is an important task to prevent the risk of extreme events. However, this information of rain is difficult to obtain as ground-based weather radars have a limited range over the ocean and that satellite-boarded sensors in low orbit can only provide intermittent observations.

Synthetic Aperture Radar (SAR) is affected by the sea surface roughness and therefore is sensitive to the rain. Recent studies proved that rainfall could be estimated by training a deep learning model on a dataset of colocated data between Copernicus's Sentinel-1 satellites and weather radar from NEXRAD. The performances of this estimation decrease in strong wind speed situations. This is due to the concurrent effect of both the rain and the wind on the ocean surface and the lower quantity of available data for strong winds. Therefore, we aim at increasing the dimension of the dataset by retrieving information from OPERA (pan-European weather radar composite product) and the Japan Meteorological Agency's phased array radars, in addition to NEXRAD. The dual-frequency precipitation radar from the Global Precipitation Measurement satellite enable open-ocean colocations.

A deep learning architecture using the SAR, the incidence angle and priors on the wind direction and speed (the latter provided by a rain-invariant wind speed estimator) is trained on this extended dataset. Cases study on tropical cyclones and strong convective events are performed over the Japanese Archipelago.

Algorithm development for the 3D precipitation nowcasting with deep learning
Shigenori Otsuka^(26)(27)(28), and Takemasa Miyoshi^{(26)(27)(28)(29)(30)
(26)} RIKEN Center for Computational Science, Japan
⁽²⁷⁾ RIKEN Cluster for Pioneering Research, Japan
⁽²⁸⁾ RIKEN Interdisciplinary Theoretical and Mathematical Sciences Program, Japan
⁽²⁹⁾ University of Maryland, USA
⁽³⁰⁾ Japan Agency for Marine-Earth Science and Technology, Japan

Phased-Array Weather Radar (PAWR) is a high-performance observation instrument that can observe three-dimensional (3D) distributions of precipitation within 60 km at high resolution every 30 seconds. PAWR would be effective in detecting and predicting rapidly evolving local storms. We have been developing a 3D precipitation nowcasting system using PAWR based on spatiotemporal extrapolation (Otsuka et al., 2016). Recently, deep learning has been applied to various prediction problems. We have been developing a 3D precipitation nowcasting system based on ConvLSTM (Shi et al., 2015). Here we will present algorithm development to improve its prediction accuracy.

An encoder-decoder structure is constructed by stacking a three-dimensional extension of ConvLSTM (3D ConvLSTM). Increasing the number of 3D ConvLSTM layers increases the representation capability, while it significantly increases its computational cost and memory footprint because of the 3D operations. To suppress the computational cost and memory footprint, down-sampling and up-sampling with skip connections are applied to make U-Net-like structure with 3D ConvLSTM. In addition, input data are preprocessed by an ordinary U-Net-like convolutional neural network. In this study, a Huber loss, which combines L1 and L2, is adopted to improve robustness and to accelerate convergence. In addition, a generalization term is introduced to constrain global metric, rather than pixel-wise metric, to ameliorate so-called double-penalty issue of pixel-wise metric. We will present preliminary results of the proposed algorithm.

References:

Otsuka, S., and Coauthors, 2016: Precipitation nowcasting with three-dimensional space-time extrapolation of dense and frequent phased-array weather radar observations. Wea. Forecasting, 31, 329-340, doi:10.1175/WAF-D-15-0063.1.

Shi, X., Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, and W.-C. Woo, 2015: Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 28, 802-810.

Ultra short-term wind forecasting for offshore wind operations and maintenance
Robin Marcille^(12)(23), Pierre Tandeo⁽¹²⁾, Maxime Thiebaut⁽²³⁾, Juan Emmanuel Johnson⁽²⁴⁾, Florence Lafon⁽²³⁾, Pierre Pinson⁽²⁵⁾, and Ronan Fablet^{(12)(14)
(12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
⁽¹⁴⁾ INRIA team Odyssey, France
⁽²³⁾ France Energies Marines, France
⁽²⁴⁾ Institut des Géosciences de l'Environnement, France
⁽²⁵⁾ Technical University of Denmark, Department of Technology, Management and Economics, Denmark

Operations and maintenance represent major costs for offshore wind energy. Heavy operations in the marine environment depend on met-ocean forecasts to be performed under their operational limitations. Such wind and wave forecasts rely on Numerical Weather Prediction models (NWP) that are very computationally expensive with limited performance for short-term forecasting and uncertainty estimation. Machine learning based models on the other hand, once trained, are computationally cheap. In this sense they can complement NWP forecasts in the short-term, assimilating online data, and deal with the uncertainty quantification.

This paper aims at exploring machine learning techniques for the ultra-short-term probabilistic wind forecasting at sea, using both observational data and model data.  The open-source dataset MétéoNet provided by Météo France will be used to emulate the target situation for offshore operations. Learning in a supervised framework to reproduce an offshore time series, the model will extrapolate both in space and time, to forecast the wind probability density at high temporal resolution, and assimilating observations online. It will explore the use of observations at high frequency and model data as a physical prior of the meteorological situation, comparing the obtained results with state-of-the-art forecasts.

Data-driven data assimilation to better characterize climate projections: a case study with an idealized chaotic AMOC model
Pierre Le Bras^(12)(18), Pierre Ailliot⁽¹⁹⁾, Noémie Le Carrer^(12)(18), Juan Ruiz^(20)(21), Florian Sévellec^(18)(22), and Pierre Tandeo^(12)
(12) IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
⁽¹⁸⁾ Laboratoire d’Océanographie Physique et Spatiale, IUEM, Univ. Brest, CNRS, IRD, Ifremer (UMR 6523)
⁽¹⁹⁾ Laboratoire de Mathématiques de Bretagne Atlantique, Univ Brest (UMR CNRS 6205), France
⁽²⁰⁾ Centro de Investigaciones del Mar y la Atmósfera, Facultad de Ciencias Exactas y Naturales, Universidad de Buenos Aires, CONICET-UBA, Argentina
⁽²¹⁾ UMI-IFAECI (CNRS-CONICET-UBA), Argentina
⁽²²⁾ Ocean and Earth Science, University of Southampton, United Kingdom

The multi-model ensemble aims to combine the outputs of different dynamical models, by weighing them, to improve projection skills. The weighting scheme relies on the choice of a performance metric to measure the closeness of individual model outputs to observations. Model weights can be used to constrain both the mean and the uncertainty in projections of climate models. In this study, we combine different parameterizations of an idealized three-dimensional model of the Atlantic Meridional Overturning Circulation. Each parameterization is evaluated online in a data assimilation framework (EnKF) by comparing the forecast states with observations. Traditional data assimilation procedures induce large computational costs to run multiple model simulations to obtain forecasts at each time step. Here, for each parameterization, the physical model equations are replaced by an already existing catalog of trajectory time evolutions. Using a machine learning approach (analogs forecasting), we are able to emulate the model dynamics. This data-driven methodology retains the benefits given by the classical EnKF at low computational costs. Then, for each parameterization, a local performance metric in time (the contextual model evidence) is computed. This metric, based on the innovation likelihood, is sensitive to differences in the model dynamics and takes into account the uncertainties. The results of the proposed method are discussed with respect to benchmark approaches including the equally weighting approach (“model democracy”) and the climatological distributions comparison.

A Machine Learning Approach to the Observation Operator for Satellite Radiance Data Assimilation
Jianyu Liang⁽²⁶⁾, Koji Terasaki⁽²⁶⁾, and Takemasa Miyoshi^(26)
(26) RIKEN Center for Computational Science, Japan

Evaluating by ERA-interim, the root mean square difference (RMSD) of the temperature (K) from three experiments.

In data assimilation (DA), observation operator (OO) is required to convert model variables to model equivalent of the observations. Many OOs for satellite microwave brightness temperature (BT) from AMSU-A are based on radiative transfer model. Liang et al. (2022, under review) used machine learning as OO (ML-OO). Their work is summarized below. The DA system comprises NICAM model and LETKF. The Radiative Transfer for TOVS (RTTOV) is combined with bias correction as OO to assimilate BT (RTTOV-OO). The conventional observations and BT were assimilated for one month using this system. The forecast data during this period were paired with the observations to train neural network models (NNs) to obtained ML-OO. It was tested in the same month next year by conducting 3 experiments: conventional observations and BT were assimilated using RTTOV-OO (CONV-AMSUA-RTTOV); the same observations were assimilated but using ML-OO (CONV-AMSUA-ML); only conventional observations were assimilated (CONV). The RMS difference (RMSD) from ERA-interim for temperature were used to evaluate the performance. The temperature RMSD in CONV-AMSUA-ML is 2% higher than that in CONV-AMSUA-RTTOV but 4% lower than that in CONV. Therefore, ML-OO works well although it is slightly worse than RTTOV-OO. As ML-OO treats the biases internally, most of the channels have small biases. Two channels have larger biases, which is a result of significant change in the satellite characteristics. If the change is small, ML-OO doesn’t require a separate bias correction procedure so that the DA system design is simplified.

References: Liang, J., Terasaki, K., and Miyoshi, T.: A machine learning approach to the observation operator for satellite radiance data assimilation (Under review)

Fishing gears identification from artificial intelligence applied to geospatial data
Francois Danhiez⁽¹⁵⁾, Mathieu Woillez⁽¹⁶⁾, and Julien Rodriguez^(16)
(15) Capgemini Engineering Brest, France
⁽¹⁶⁾ Ifremer, France

Facing a lack of data to accurately assess the spatial distribution of catches, fishing effort and for the environmental characterization of the fishing areas, Ifremer has implemented since 2005 a project called « Recopesca ». It is based on a sample of voluntary fishing vessels equipped with sensors coupled to on-board GPS monitors recording positions at least every fifteen minutes. The data gathered are used as a benchmark dataset for various research programs regarding to the use of geolocation information to monitor fisheries.    

As an example, a study investigated the use of machine learning to identify fishing gears from geospatial data. The data set gathered is composed of 25 612 trips recorded on 107 vessels from 2006 to 2020. It includes 16 gear types aggregated in 10 gear classes including steaming “SAIL” to be able to identify trips without fishing events. Five different algorithms (KNN, C5.0, SVM, RF and XGBoost) were tested on the basis of motion, speed and time related variables computed on every trip. After the training of these algorithms on 70% of the data set, the algorithms precision was validated on the remaining 30 % of the data set. The model comparison shows that XGBoost achieve the classification of the 10 fishing gear classes with the highest accuracy (94.89 %). After choosing XGBoost as algorithm to predict the gear classes, we conducted an algorithms optimization leading to the building of a final algorithm exploiting only 34 of the 96 features to classify the fishing gears with the same performance.

Monitoring In-Situ Wind Speed Using Underwater Acoustics and Deep Learning
Matteo Zambra^(12)(14), Dorian Cazau⁽¹³⁾, Nicolas Farrugia⁽¹²⁾, and Ronan Fablet^{(12)(14)
(12)} IMT Atlantique, Lab-STICC, UMR CNRS 6285, France
⁽¹⁴⁾ INRIA team Odyssey, France
⁽¹³⁾ ENSTA Bretagne, Lab-STICC, UMR CNRS 6285, France

Comparison between in-situ wind speed, ECMWF wind speed reanalyses and the prediction of wind speed made with three different methods: Multi-modal 4DVarNet, Single-modal 4DVarNet and CatBoost

Wind speed retrieval at sea surface is an aspect of primary importance for scientific and operational applications. Besides weather models, in situ measurements and remote sensing technologies, especially satellite sensors, provide complementary means to monitor wind speed. As sea surface winds impact underwater sound propagation, underwater acoustics recordings can also deliver fine-grained wind-related information. Whereas model-driven schemes, especially data assimilation approaches, are the state-of-the-art schemes to address inverse problems in geoscience, machine learning techniques become more and more appealing to fully exploit the potential of observation data sets. Here, we introduce a deep learning approach for the retrieval of wind speed time series from underwater acoustics possibly complement by other data sources such weather model reanalyses.

Our approach bridges data assimilation and learning-based  frameworks in such a way to benefit from prior physical knowledge and computational efficiency.

Numerical experiments for real data demonstrate that we outperform the state-of-the-art with a relative gain up to 12% in terms of RMSE. Interestingly, these results support the relevance of the time dynamics of underwater acoustic data to better inform the time evolution of wind speed. They also show that multimodal data, here underwater acoustics data complemented by ECMWF reanalysis data, may further improve the reconstruction performance, including the robustness to the sampling strategy of underwater acoustics data.