Research Summary: No One Knows the State of the Art in Geospatial Foundation Models

An analysis of the geospatial foundation model (GFM) literature reveals significant challenges in assessing the current state of the art. Despite GFMs being proposed for high-stakes Earth-observation tasks such as disaster response and food security, there is a critical lack of standardized evaluations, training protocols, and transparency regarding model weights and pretraining controls. This deficiency makes it impossible for practitioners and researchers to objectively compare or select suitable models for specific applications.

Why it matters

This situation poses a significant risk to the reliable application of advanced geospatial technologies in critical domains. Without clear benchmarks and transparent reporting, investment in GFM development and deployment may be misdirected, hindering progress in areas vital for global resilience and resource management.

Key insights

• Published work on Geospatial Foundation Models (GFMs) lacks sufficient information for reviewers or users to determine model suitability for specific tasks.
• A significant gap exists in understanding the current state of the art in GFM capabilities due to inconsistent reporting.
• The GFM literature does not standardize evaluations, training/testing protocols, released weights, or pretraining controls.
• An audit of 152 papers identified 46 cross-paper disagreements of at least 10 points for the same model and benchmark, indicating inconsistencies in reporting or methodology.

Source

arXiv — Computers and Society — https://arxiv.org/abs/2605.12678