I've spent the month downloading/processing data from five weather models. There are 25 weather recording stations within 50 miles of Columbia, Missouri (where I live). This is the area that I want to forecast for.
I'm downloading all of the model runs/forecasts for all of the models for the time period 1/1/2021 to 12/31/2025. If I were to just download all data for all of North America, I'd never be able to hold it all. I download the data for a day, save more variables with more data points within 150 miles of CoMo, and then fewer variables with less densely packed data points outside of that. Finally, I save nothing more than 3000 miles from CoMo. Once this all has been "cropped" from the full model, the full model data for that day is discarded. The model/data point reductions are more complicated than this, but that's the gist of it.
When I have all of the model data downloaded/processed, I'm going to use chatgpt.com to help me use my GPU (NVIDIA GeForce RTX 4060 Ti 16 GB) to ingest the data (in pieces) and write software that learns which models are most reliable in what types of weather situations and will eventually try to out-forecast them.
Mid-Missouri is a very hilly area, and I'm hoping to integrate detailed topographic data into the resultant model and be able to micro-forecast for this area. I'm Class of '94 and have a Meteorolgy degree that I've never used.
Download/processing progress:
This is an example of data being "reduced" into certain variables for certain locations:
System resources aren't really being pushed outside of RAM. RAM usage had gotten up to 55 GB but has since fallen back into the high 40s.
I'm downloading all of the model runs/forecasts for all of the models for the time period 1/1/2021 to 12/31/2025. If I were to just download all data for all of North America, I'd never be able to hold it all. I download the data for a day, save more variables with more data points within 150 miles of CoMo, and then fewer variables with less densely packed data points outside of that. Finally, I save nothing more than 3000 miles from CoMo. Once this all has been "cropped" from the full model, the full model data for that day is discarded. The model/data point reductions are more complicated than this, but that's the gist of it.
When I have all of the model data downloaded/processed, I'm going to use chatgpt.com to help me use my GPU (NVIDIA GeForce RTX 4060 Ti 16 GB) to ingest the data (in pieces) and write software that learns which models are most reliable in what types of weather situations and will eventually try to out-forecast them.
Mid-Missouri is a very hilly area, and I'm hoping to integrate detailed topographic data into the resultant model and be able to micro-forecast for this area. I'm Class of '94 and have a Meteorolgy degree that I've never used.
Download/processing progress:
This is an example of data being "reduced" into certain variables for certain locations:
System resources aren't really being pushed outside of RAM. RAM usage had gotten up to 55 GB but has since fallen back into the high 40s.
