Great questions.

1. The truth is we still have to investigate the the numerical stability of these models. Our GFT forecast rollouts are around 2 weeks (~60 steps) long and things are stable in in that range. We're working on longer-ranged forecasts internally.

2. The compute requirements are extremely favorable for ML methods. Our training costs are significantly cheaper than the fixed costs of the supercomputers that government agencies require and each forecast can be generated on 1 GPU over a few minutes instead of 1 supercomputer over a few hours.

3. There's a similar floating-point story in deep learning models with FP32, FP16, BF16 (and even lower these days)! An exciting area to explore

Our training cutoff date was the end of 2022. Here's our blogpost on the 2024 hurricane season https://silurian.ai/posts/001/hurricane_tracks

I just don't understand how can your produce new knowledge which it don't have access to. Are you you folks claiming the future weather is a function of previous weather and the model is capable of replicating the function?

No one is claiming that there is "new knowledge" here.

The entire class of deep learning or AI-based weather models involves a very specific and simple modeling task. You start with a very large training set which is effectively a historical sequence of "4D pictures" of the atmosphere. Here, "4D" means that you have "pixels" for latitude, longitude, altitude, and time. You have many such pictures of these for relevant atmospheric variables like temperature, pressure, winds, etc. These sequences are produced by highly-sophisticated weather models run in what's called a "reanalysis" task, where they consume a vast array of observations and try to create the 4D sequence of pictures that are most consistent with the physics in the weather model and the various observations.

The foundation of AI weather models is taking that 4D picture sequence, and asking the model how to "predict" the next picture in the sequence, given the past 1 or 2 pictures. If you can predict the picture for 6 hours from now, then you can feed that output back into the model and predict the next 6 hours, and so on. AI weather models are trained such that this process is mostly stable, e.g. the small errors you begin to accumulate don't "blow up" the model.

Traditionally, you'd use a physics-based model to accomplish this task. Using the current 3D weather state as your input, you integrate the physics equations forward in time to make the prediction. In many ways, today's AI weather models can be thought of as a black box or emulator that reproduces what those physics-based models do - but without needing to be told much, if any of the underlying physics. Depending on your "flavor" of AI weather model, the architecture of the model might draw some analogies to the underlying physics. For example, NVIDIA's models use Fourier Neural Operators, so you can think of them as learning families of equations which can be combined to approximate the state of the atmosphere (I'm _vastly_ over-simplifying here). Google DeepMind's GraphCast tries to capture both local and non-local relationships between fields through it's graph attention mechanisms. Microsoft Aurora' (and Silurian's, by provenance, assuming it's the same general type of model) try to capture local relationships through sliding windows passed over the input fields.

So again - no new knowledge or physics. Just a surprisingly effective of applying traditional DL/AI tools to a specific problem (weather forecasting) that ends up working quite well in practice.

Thanks for the explanation. I am still a bit confused how this takes care of the errors? I can see how the weather prediction for tomorrow might have less errors. But shouldn't the errors accumulate as you feed the predicted weather as the input for the model? Wouldn't the results start diverging from reality pretty soon? Isn't that the reason why the current limit is close to 6 days? How exactly does this model fixed this issue?

It doesn't take care of the errors. They still "accumulate" over time, leading to the same divergence that traditional physics-based weather models experience. In fact, the hallmark that these AI models are _doing things right_ is they show realistic modes of error growth when compared with those physics-based models - and there is already early peer-reviewed literature suggesting this is the case.

This _class_ of models (not Aurora, or Silurian's model specifically) can potentially improve on this a bit by incorporating forecast error at longer lead times in their core training loss. This is already done in practice for some major models like GraphCast and Stormer. But these models are almost certainly not a magical silver bullet for 10x'ing forecast accuracy.

Thanks for the explanation.

Yes, 100%! We'll still take a statistical/distributional approach to long-ranged climate behavior rather than trying to predict exact atmospheric states. Keep an eye out for more news on this

If there is sufficient data, we can train on it!

Would geolocated historical seismographic data do?

I suspect (possibly incorrectly) that earthquakes are a chaotic phenomenon resulting from a multilayered complex system, a lot like a lottery ball picker.

Essentially random outputs from deterministic systems are unfortunately not rare in nature…. And I suspect that because of the relatively higher granularity of geology vs the semicohesive fluid dynamics of weather, geology will be many orders of magnitude more difficult to predict.

That said, it might be possible to make useful forecasts in the 1 minute to 1 hour range (under the assumption that major earthquakes often have a dynamic change in precursor events), and if accuracy was reasonable in that range, it would still be very useful for major events.

Looking at the outputs of chaotic systems like geolocated historical seismographic data might not be any more useful than 4-10 orders of magnitude better than looking at previous lottery ball selections in predicting the next ones…. Which is to say that the predictive power might still not be useful even though there is some pattern in the noise.

Generative AI needs a large and diverse training set to avoid overfitting problems. Something like high resolution underground electrostatic distribution might potentially be much more predictive than past outputs alone, but I don’t know of any such efforts to map geologic stress at a scale that would provide a useful training corpus.

They’re empiricists — the only ~~real~~ conclusive way to answer that question is to try it, IMO!

The old ML maxim was “don’t expect models to do anything a human expert couldn’t do with access to the same data”, but that’s clearly going to way of Moore’s Law… I don’t think a meteorologist could predict 11km^2 of weather 10 days out very accurately, and I know for sure that a neuroscientists couldn’t recreate someone’s visual field based on fMRI data!

Hi, this looks really cool! Can we meet? Shoot us an email at contact@silurian.ai

We want to branch out to industries which are highly dependent on weather. That way we can integrate their data together with our core competency: the weather and climate. Some examples include the energy grid, agriculture, logistics, and defense.

you'll have trouble simulating the grid, but for energy data you might want to look at (or get in touch with) these people: https://app.electricitymaps.com/map

They're a cool little team based in Copenhagen. Would be useful, for example, to look at the correlation between your weather data and regional energy production (solar and wind). Next level would be models to predict national hydro storage, but that is a lot more complex.

My advice is to drop the grid itself to the bottom of the list, and I say this as someone who worked at a national grid operator as the primary grid analyst. You'll never get access to sufficient data, and your model will never be correct. You're better off starting from a national 'adequacy' level and working your way down based on information made available via market operators.

Actually, it seems like a great time to get involved with the grid (at least in the US). In order to comply with FERC Order 881, all transmission operators need to adjust their line ratings based on ambient temperatures with hourly predictions 10 days into the future by mid 2025. Seems like that would present a great opportunity to work directly with the ISOs (which have regional models and live data) on improving weather data.

These are great resources, thank you. If you're open to it, we'd love to meet and chat about the energy space since we're newcomers to that arena. Shoot us an email at contact@silurian.ai

If their weather forecast is really the best, power traders would pay them large amounts just for the forecast.

You'd be surprised :D

I was surprised: https://arxiv.org/abs/1912.09363

It's not exactly an LLM but it works in a similar fashion.

We explored several examples from the 2024 hurricane season in our blog post: https://silurian.ai/posts/001/hurricane_tracks. We overlaid the true paths of the hurricane over our predictions for everyone to see!

I'm finding the posts confusing -- is the prediction the images?

What exactly is predicted and what is the actual path in those videos?

In the videos the true path is the dashed line and the government prediction is the solid line. Our prediction, from our GFT model, is the animation which plays in the background.

Hi, it totally is. That's one of our favorite weather visualization projects. We're using Cameron Beccario's open source version of nullschool for our forecasts. We cited him above in the blurb and also on our about page (https://hurricanes2024.silurian.ai/about.html)

Hi, Nikhil here. We haven't done a head-to-head comparison of GFT vs GraphCast, but our internal metrics show GFT improves on Aurora and published metrics show Aurora improves on GraphCast. You can see some technical details in section 6 of the Aurora paper (https://arxiv.org/pdf/2405.13063)