Introduction:

The midterm project for my CS7150: Deep Learning class, my team developed an end-to-end deep learning system for predicting cloud formations from satellite imagery, tackling one of meteorology's most challenging problems. Cloud forecasting has wide-ranging impacts on agriculture, aviation, and renewable energy, but remains difficult due to clouds' dynamic evolution and complex spatial patterns. 

​

Our Approach:

• Processed 70,080+ satellite images from the CloudCast dataset (768x768 resolution, 10 cloud classes, 15-minute intervals).

• Designed a vanilla autoencoder for dimensionality reduction, compressing images 6x while preserving fine-grained spatial details.

• Implemented and compared multiple architectures: baseline CNN, CNN-LSTM, and GAN with ConvLSTM.

• Developed ConvLSTM layers that preserve 2D spatial structure during temporal processing, replacing flatten-and-project operations that lose critical spatial information

​

Key Innovations:

• ConvLSTM architecture using Hadamard products on 2D feature maps for joint spatial-temporal learning

• GAN training stabilization through balanced discriminator/generator learning rates and reconstruction loss weighting

• Context window preprocessing for sequential satellite image analysis

​

Results:

Our GAN with ConvLSTM achieved superior performance in capturing temporal cloud patterns, as measured by Structural Similarity Index (SSIM). The model successfully predicted cloud movements and formations for 15-minute forecast intervals, with the ConvLSTM approach significantly outperforming traditional LSTM methods in preserving spatial coherence.
 

Collaborative project with George Wang and Jacob Krucinski for CS7150: Deep Learning at Northeastern University​​​