What is Shadow Detection in Remote Sensing Applications?
Shadows are ubiquitous in satellite and aerial imagery, naturally arising when sunlight is obstructed by elevated structures such as buildings, vegetation, or mountainous terrain. Depending on the application, these shadows may serve as valuable geometric cues or complicate accurate scene interpretation. Shadow detection refers to the process of identifying and segmenting these shadowed areas from images captured by Earth observation platforms.
Remote sensing systems—whether mounted on satellites, drones, or aircraft—routinely capture high-resolution imagery for urban planning, disaster monitoring, agricultural assessment, and climate science applications. In many tasks, shadowed areas can severely affect downstream processing like classification, change detection, and 3D reconstruction. Being able to detect shadows automatically is particularly important to create efficient systems dedicated to Earth Observation.
Despite its practical significance, shadow detection remains a challenging computer vision task, especially in remote sensing. Unlike ground-level images, which can capture a much smaller area or rely on artificial illumination, nearly all satellite images capture shadows. Shadows in satellite imagery are affected by terrain irregularities, varying sun angles, and diverse surface materials.
Figure 1: Examples of shadows in remote sensing imagery
Why Does It Matter?
Most available shadow detection datasets predominantly consist of ground-level imagery, including indoor environments and street-level photographs. These datasets inherently differ from satellite imagery in terms of viewing angles, illumination conditions, and scene compositions, making them inadequate for capturing the specific characteristics of shadows observed from satellite and aerial platforms. In particular, existing datasets typically suffer from:
- Limited domain generalization: Models trained on everyday photos don’t generalize well to satellite imagery.
- Sparse annotation: High-quality shadow annotations are time-consuming to produce and rare in remote sensing contexts.
- Neglect of physical context: Many datasets ignore geometric properties like sun elevation and camera tilt, which are critical to shadow formation.
Consequently, models trained on such datasets often exhibit limited performance when applied to real-world remote sensing scenarios, underscoring the need for specialized datasets explicitly designed for satellite and aerial imagery contexts.
The Digital Sense R&D team—working alongside academic partners—has co-authored a landmark paper (presented in CVPR 2025) introducing the S-EO dataset, a large-scale, high-resolution dataset specifically tailored to advance machine learning models in shadow detection for remote sensing. This contribution addresses a critical gap in the remote sensing community, providing researchers and practitioners with the first comprehensive, publicly available dataset explicitly designed for satellite and aerial shadow detection tasks.
By making this dataset accessible, we enable the development of robust shadow detection models, particularly benefiting urban modeling applications. Accurate shadow detection is crucial for urban planning and infrastructure monitoring, supporting improved analysis of building heights, structural density, and urban development patterns.
Moreover, our research demonstrates how shadows can be exploited as valuable geometric cues. Leveraging shadows in our machine learning models allows more precise interpretation of 3D geometry, significantly enhancing the quality and accuracy of 3D reconstructions. Our state-of-the-art shadow detector, trained using the S-EO dataset, integrates seamlessly with advanced photogrammetry methods, such as EO-NeRF, resulting in improved altitude estimation and sharper geometric details in the resulting 3D models. This advancement not only pushes the frontier of remote sensing capabilities but also showcases the practical benefits of our approach for diverse real-world applications.
The S-EO Dataset: A Geometry-Aware Shadow Detection Dataset
The S-EO dataset (named after Shadow-aware Earth Observation) comprises 702 georeferenced tiles across the USA covering 500 x 500 m each. Each tile includes multi-date, multi-angle WorldView-3 pansharpened RGB images, panchromatic images, and a ground-truth DSM of the area obtained from LiDAR scans. For each image, we provide a shadow mask derived from geometry and sun position, a vegetation mask based on the NDVI index, and a bundle-adjusted RPC model. With approximately 20,000 images, the S-EO dataset aims to establish a new public resource for shadow detection in remote sensing. This enables researchers and practitioners to train models that detect shadows and reason about them physically and geometrically.
Key Features of the S-EO Dataset:
Figure 2: Sample from the S-EO dataset
Evaluating S-EO: Enhanced Model Performance and 3D Reconstruction Improvements
To demonstrate the practical impact of the S-EO dataset, we trained a widely-used U-Net architecture—a versatile encoder-decoder network commonly employed for image segmentation tasks—using our large-scale satellite imagery data. Despite its simplicity, our U-Net trained on the S-EO dataset showed exceptional performance when fine-tuned on the AISD dataset, a well-established aerial shadow detection benchmark.
Specifically, our approach involved fine-tuning the U-Net model pre-trained on S-EO using only 10% (41 images) of the AISD training data. Remarkably, this lightly fine-tuned model outperformed DSSDNet—the reference AISD model explicitly designed and trained on the full AISD training set—in key metrics such as AUC-ROC (0.990 vs. 0.985) and achieved a comparable F-score (91.52% vs. 91.79%).
Table 1: Shadow Detection Performance on AISD dataset
Additionally, we leveraged our trained shadow detector to enhance EO-NeRF—a state-of-the-art model for multi-view 3D reconstruction from satellite imagery. Incorporating shadow masks predicted by our model provided geometric supervision that notably improved altitude accuracy and geometric detail in the resulting 3D reconstructions.
Table 2: Average EO-NeRF 3D Reconstruction Altitude Error (meters)
Innovation at Digital Sense
This research exemplifies the commitment of Digital Sense to the scientific foundation of AI solutions. With more than 200 peer-reviewed publications and partnerships with academic leaders like ENS Paris-Saclay and Universidad de la República, we routinely bridge research and application. The co-authors of this paper are:
- Elías Masquil, Senior R&D Engineer at Digital Sense
- Roger Marí, Senior Researcher at Eurecat
- Thibaud Ehret, Senior Researcher at AMIAD
- Enric Meinhardt-Llopis, Professor at ENS Paris-Saclay
- Pablo Musé, Co-Founder at Digital Sense, Full Professor at Universidad de la República
- Gabriele Facciolo, Senior Scientific Consultant at Digital Sense and Professor at ENS Paris-Saclay
Their joint work on S-EO strengthens our position as a leading provider of AI and Image Processing services for geospatial analytics, as shown in our successful collaborations with CNES, Satellogic, and CESBIO.
For decision-makers evaluating AI partners in sectors such as earth observation, infrastructure monitoring, and agritech, the S-EO project showcases our cutting-edge capabilities in data-centric AI, geospatial modeling, and scientific rigor.
Next Steps: Bring Shadow Intelligence to Your Geospatial Stack
At Digital Sense, we specialize in turning advanced AI research into real-world solutions. With over a decade of experience and a team that includes PhDs, Master's holders, and experienced engineers, we’ve delivered over 100 successful AI projects across industries—including collaborations with Satellogic, CNES, CESBIO, and others in the space sector.
Whether you're optimizing satellite data pipelines, developing geospatial AI systems, or building scalable 3D modeling platforms, we provide the scientific rigor and engineering talent to help you succeed.
We invite you to learn more about our projects and get in touch.
📞 Schedule a consultation with our team at www.digitalsense.ai
📚 Explore more case studies and technical insights on our blog
📩 Contact us for collaboration opportunities or custom solutions
Reference
Masquil, E., Marí, R., Ehret, T., Meinhardt-Llopis, E., Musé, P., & Facciolo, G. (2025). S-EO: A Large-Scale Dataset for Geometry-Aware Shadow Detection in Remote Sensing Applications. arXiv preprint arXiv:2504.06920. https://arxiv.org/abs/2504.06920
